Experience
10+ Years
Key Skills
Hadoop, AWS EMR (Must have)
Job Description - We need a very experienced and hands-on Hadoop admin to help with the following:
- Maintain the AWS EMR production cluster
- Install software on Hadoop cluster to support data science and analytics users. This includes but not lmited to R, Python packages, Spark, H2O and other Big Data Open source components/services
- Maintain user accounts/system security and cluster resource management to optimize user query execution time and data loading time
- Help with data migration and issue resolution while migrating the existing Hadoop/Java applications from on-premise cluster to cloud distribution
Minimum Skills Required:
- Minimum 3 years of experience as Hadoop admin in a large (multi-Petabyte) production environment
- Experience supporting data science teams, super users and analytics teams on complex code deployment, debugging and performance optimization problems
- Experience with Hadoop, HDFS, Hive, Spark, R, Python, Java and UNIX
- System monitoring and controls
- Experience managing CPU, memory, storage resources for a large Hadoop cluster with 100s of users.