Hadoop Administrator (3-5 yrs)

Job Summary :

We are seeking a skilled and experienced Hadoop Administrator with 3-5 years of experience to join our growing team. You will be responsible for the installation, configuration, maintenance, and performance tuning of our Hadoop clusters.

This role requires a strong understanding of Hadoop architecture, ecosystem components, and best practices for cluster management.

You will work closely with data engineers, analysts, and other stakeholders to ensure the availability and reliability of our big data platform.

Key Responsibilities :

- Cluster Management : Install, configure, and maintain Hadoop clusters, including HDFS, YARN, MapReduce, and other ecosystem components.

- Performance Tuning : Monitor cluster performance, identify bottlenecks, and implement optimizations to improve efficiency and scalability.

- Capacity Planning : Forecast cluster resource needs and plan for capacity expansion.

- Security Management : Implement and maintain security measures to protect data and the Hadoop environment. Manage user access and permissions.

- Software Upgrades : Plan and execute software upgrades and patches for Hadoop and related components.

- Troubleshooting : Diagnose and resolve issues related to Hadoop clusters, including hardware failures, software bugs, and performance problems.

- Monitoring & Alerting : Set up and maintain monitoring and alerting systems to proactively identify and address potential issues.

- Backup & Recovery : Implement backup and recovery strategies to ensure data availability and disaster recovery.

- Automation : Automate routine tasks using scripting languages like Bash or Python.

- Documentation : Maintain accurate documentation of Hadoop cluster configurations, procedures, and troubleshooting steps.

- Collaboration : Work closely with data engineers, analysts, and other stakeholders to understand their requirements and provide support.

Required Technical Skills :

- Hadoop Ecosystem : Deep understanding of the Hadoop ecosystem, including HDFS, YARN, MapReduce, Hive, Pig, Spark, and other related components.

- Linux : Strong Linux system administration skills, including command-line proficiency, shell scripting, and package management.

- Networking : Basic understanding of networking concepts, including TCP/IP, DNS, and firewalls.

- Scripting : Proficiency in scripting languages like Bash or Python for automation and system administration tasks.

- Monitoring Tools : Experience with monitoring tools like Ganglia, Nagios, or Cloudera Manager.

- Troubleshooting : Strong troubleshooting skills and the ability to diagnose and resolve complex technical issues.

Preferred Skills :

- Cloud Platforms : Experience with cloud-based Hadoop distributions (e.g. , AWS EMR, Azure HDInsight, Google Cloud Dataproc).

- Big Data Technologies : Familiarity with other big data technologies like NoSQL databases (e.g. , HBase, Cassandra) or data streaming platforms (e.g. , Kafka).

- Security : Knowledge of Hadoop security best practices, including Kerberos authentication and authorization.

- Configuration Management : Experience with configuration management tools like Ansible, Chef, or Puppet.

- Database Administration : Basic understanding of database administration concepts.

Required Experience & Qualifications :

- Experience : 3-5 years of experience as a Hadoop Administrator.

- Education : Bachelor's degree in Computer Science, Information Technology, or a related field preferred.

Personal Attributes :

- Strong problem-solving and analytical skills.

- Excellent communication and interpersonal skills.

- Ability to work effectively in a team environment.

- Proactive, self-motivated, and results-oriented.

- Attention to detail and a commitment to quality