Senior Data Engineer - Python/Java (8-15 yrs)

About the Role :

We are seeking a highly skilled and experienced Senior Data Engineer to join our data engineering team.

In this role, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines to support critical business decisions.

You will work closely with data scientists, analysts, and engineers to build and maintain data infrastructure that enables data-driven insights and innovation.

Key Responsibilities :

- Design, develop, and maintain data pipelines using Python (preferred), Java, or Scala.

- Leverage cloud platforms like AWS (preferred), Azure, or GCP for data storage, processing, and analysis.

- Experience with data engineering tools like Snowflake, Apache Airflow/Prefect/Dagster, Kafka (preferred), AWS SQS, Google Cloud Pub/Sub.

- Implement stream processing solutions using Apache Flink (preferred), Apache Spark Streaming, or Kafka Streams.

- Build and maintain data lakes using Apache Iceberg (preferred) or AWS Lake Formation.

- Develop and maintain data quality checks and monitoring systems.

- Collaborate with data scientists and analysts to understand data requirements and translate them into technical solutions.

- Work closely with DevOps teams to implement CI/CD pipelines (nice to have).

- Stay up-to-date with the latest advancements in data engineering technologies.

Qualifications :

- 5+ years of experience in data engineering or a related field.

- Strong proficiency in Python (preferred), Java, or Scala.

- Experience with cloud platforms like AWS (preferred), Azure, or GCP.

- Experience with data engineering tools like Snowflake, Apache Airflow/Prefect/Dagster, Kafka (preferred), AWS SQS, Google Cloud Pub/Sub.

- Experience with stream processing technologies like Apache Flink (preferred), Apache Spark Streaming, or Kafka Streams.

- Experience with data lakes like Apache Iceberg (preferred) or AWS Lake Formation.

- Experience with containerization and orchestration tools like Docker and Kubernetes (nice to have).

- Understanding of CI/CD pipelines (nice to have).

- Excellent problem-solving, analytical, and critical thinking skills.

- Strong communication and interpersonal skills.

- Ability to work independently and as part of a team.

- Bachelor's degree in Computer Science, Engineering, or a related field.

Bonus Points :

- Experience with data warehousing and data modeling.

- Experience with NoSQL databases (e.g, Cassandra, MongoDB).

- Experience with data visualization and business intelligence tools.

- Contributions to open-source projects