Kafka Administrator - Prometheus/Grafana (6-10 yrs)

Kafka admin required with below skill sets :

Cluster Management :

- Install, configure, and maintain Kafka clusters across various environments (development, testing, production).

- Perform upgrades and patching of Kafka and related components (e.g., Zookeeper).

- Perform Kafka migration and related components

- Ensure optimal performance and reliability of Kafka clusters.

Monitoring and Troubleshooting :

- Monitor Kafka cluster health and performance using tools like Prometheus, Grafana, or proprietary monitoring solutions.

- Diagnose and resolve Kafka brokers, topics, partitions, and consumer issues.

- Implement proactive measures to prevent potential issues.

Security and Compliance :

- Implement and manage security protocols for Kafka, including SSL/TLS encryption, Kerberos authentication, and access control policies.

- Ensure compliance with organizational and industry standards for data security and privacy.

- Apply network security principles to protect Kafka infrastructure.

Capacity Planning and Scalability :

- Perform capacity planning to ensure the Kafka infrastructure can handle current and future workloads.

- Optimize Kafka configurations for performance and scalability based on application requirements.

Backup and Recovery :

- Develop and maintain disaster recovery plans for Kafka environments.

- Implement and test backup and restore procedures to ensure data integrity and availability.

Collaboration and Support :

- Work closely with development teams to understand Kafka usage patterns and provide guidance on best practices.

- Provide support for Kafka-related issues, including on-call support as needed.

- Document Kafka infrastructure, configurations, and operational procedures.

Automation and Scripting :

- Develop automation scripts for routine tasks such as cluster provisioning, monitoring, and maintenance using tools like Ansible, Puppet, or custom scripts.

- Implement terraform code(IAC) for Kafka cluster creation.

- Implement CI/CD pipelines for Kafka-related deployments and updates.

Client Communication and Reporting :

- Maintain regular communication with clients to provide updates and gather feedback.

- Prepare and present weekly and monthly status reports to stakeholders.

- Present proof-of-concept (POC) solutions and designs to clients and internal teams.

- Hands on with Kubernetes deployment and debugging

Nice to Have - GCP Knowledge