Hadoop Administrator - Cloudera/Hortonworks (2-4 yrs)

Job Description :

We are seeking an experienced Hadoop Administrator with 4 years of hands-on expertise in managing and maintaining large-scale Hadoop clusters. The ideal candidate will be responsible for ensuring the high availability, security, and performance of Hadoop ecosystems while collaborating with data engineers and developers to optimize big data workflows.

Key Responsibilities :

- Install, configure, and maintain Hadoop clusters (Cloudera, Hortonworks, Apache Hadoop, etc.).

- Monitor cluster performance and troubleshoot issues to ensure high availability and reliability.

- Implement security measures such as Kerberos authentication, SSL, and role-based access control.

- Perform regular cluster maintenance, upgrades, and patches.

- Manage HDFS storage, replication, and optimization.

- Implement cluster tuning, capacity planning, and performance monitoring.

- Work with development teams to optimize MapReduce, Spark, and Hive jobs.

- Automate administrative tasks using Shell scripting, Python, or Ansible.

- Ensure data backup and disaster recovery plans are in place.

- Troubleshoot system errors, hardware failures, and connectivity issues.

- Maintain and manage YARN, HBase, Hive, Kafka, and Spark as part of the ecosystem.

Required Skills and Qualifications :

- 4+ years of experience as a Hadoop Administrator.

- Hands-on experience with Hadoop distributions (Cloudera, Hortonworks, or Apache Hadoop).

- Strong understanding of Linux administration (RedHat, CentOS, Ubuntu).

- Experience with HDFS, YARN, Hive, HBase, Spark, and Kafka.

- Knowledge of cluster security using Kerberos, Ranger, or Knox.

- Familiarity with automation tools (Ansible, Puppet, Chef).

- Proficiency in Shell scripting, Python, or Perl for automation.

- Experience with monitoring tools like Nagios, Prometheus, Grafana, or Cloudera Manager.

- Strong problem-solving skills and the ability to work independently.

Preferred Qualifications :

- Certifications like Cloudera Certified Administrator (CCA) or Hortonworks HDPCA.

- Experience in cloud-based Hadoop deployments (AWS EMR, Azure HDInsight, GCP Dataproc).

- Knowledge of SQL and NoSQL databases.