Kafka Administrator - Cluster Management (8-12 yrs)

Job Description :

As a Kafka Administrator, you will play a crucial role in developing and maintaining our corporate messaging infrastructure, ensuring the smooth delivery, security, retention, and recoverability of topics and messages.

You will be responsible for designing, building, testing, and deploying high-quality messaging and streaming solutions that align with project objectives, technology architecture standards, and departmental best practices. In addition to administering the enterprise-wide messaging architecture, you will actively manage the Kafka eco-system and implement event sourcing strategies to enhance system performance and reliability.

Requirements :

Manage Large-Scale Kafka Clusters :

- Oversee and manage multi-node Kafka cluster environments residing on AWS and on-prem OpenShift platforms.

- Lead Kafka environment builds, including design, capacity planning, cluster setup, performance tuning, and ongoing monitoring.

Operational Maintenance and Support :

- Perform high-level, day-to-day operational maintenance, support, and upgrades for the Kafka Cluster.

- Collaborate closely with product managers and lead engineers to ensure effective communication and alignment on project requirements.

Standardization and Procedures :

- Provide guidance in the creation and modification of standards and procedures related to Kafka administration.

- Implement best practices to optimize Kafka cluster performance and ensure seamless operation.

Topic Management and Monitoring :

- Create topics and set up redundancy clusters to enhance system resilience.

- Deploy monitoring tools and alerts for proactive monitoring of the Kafka Cluster and supporting hardware, ensuring system health and maximum availability.

Technical Expertise and Production Support :

- Offer technical expertise and guidance to production support teams, facilitating issue resolution and continuous improvement.

Requirements :

- Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).

- Proven experience as a Kafka Administrator or similar role with hands-on expertise in managing large-scale Kafka clusters.

- Solid understanding of messaging and streaming technologies, Kafka internals, and event sourcing strategies.

- Strong proficiency in AWS services and OpenShift platform for deploying Kafka clusters.

- Familiarity with capacity planning, performance tuning, and cluster optimization techniques.

- Excellent problem-solving skills and the ability to troubleshoot complex issues in a timely manner.

- Effective communication and collaboration skills, with the ability to work closely with cross-functional teams.

Preferred Qualifications :

- Certification in Kafka Administration or related technologies.

- Experience with other distributed data streaming platforms and messaging systems.

- Knowledge of DevOps practices and tools for automating deployment and monitoring tasks.

- Understanding of security best practices for securing Kafka clusters and data.

- Familiarity with other big data technologies and their integration with Kafka.