Streaming Platform Engineer

What We re Looking For:
We re looking for a talented engineer with expertise in managing and scaling Kafka clusters. Your role will involve designing, deploying, and maintaining streaming data systems for high performance and reliability. Key qualifications include:

• Kafka Expertise:

• Hands-on experience with Kafka setup, configuration, and management in production environments.

• Knowledge of Kafka Connect, Kafka Streams, and handling schema registry.

• Proficiency in tuning Kafka for scalability and performance in high-throughput environments.

• Scalability and High Availability:

• Experience designing multi-region Kafka clusters with disaster recovery capabilities.

• Proficiency in managing replication, partitioning, and leader election.

• Kubernetes Knowledge:

• Experience deploying Kafka on Kubernetes (K8s) using operators like Strimzi or Confluent.

• Understanding of persistent storage and stateful workload configurations.

• Coding and Automation:

• Strong scripting or programming skills for automating Kafka deployments and operational tasks.

• Familiarity with CI/CD pipelines and infrastructure-as-code tools (e.g., Terraform, Ansible).

• Cloud Expertise:

• Experience with AWS cloud platform

• Collaboration and Communication:

• Ability to work with teams to design streaming architectures and educate others on Kafka best practices.

• Strong troubleshooting skills with a focus on ensuring system reliability.

What You ll Be Learning and Doing:

• Building Streaming Architectures: Design and deploy scalable Kafka clusters to handle real-time data processing needs.

• Optimizing Kafka: Work on performance tuning and ensuring high availability in distributed systems.

• Automating Operations: Develop scripts and tools to streamline Kafka management and reduce manual effort.

• Cloud-Native Systems: Deploy and manage Kafka clusters on Kubernetes and public cloud platforms.

• Educating Teams: Collaborate with customers and internal teams to share Kafka best practices and scalable solutions.

• Open Source Contributions: Create tools or contribute to projects that enhance Kafka operations.