Data Engineer - Python/SQL (4-5 yrs)

Requirements:

- 3+ years of experience in data engineering, big data processing, or related roles (6+ years for the Senior Data Engineer role)

- Strong programming skills in Python, Java, or Scala.

- Experience with distributed data processing frameworks such as Apache Spark, Flink, or Hadoop.

- Strong understanding of SQL and NoSQL databases for data storage and retrieval.

- Experience with real-time data streaming technologies like Apache Kafka or Pulsar.

- Knowledge of containerization and orchestration tools like Docker and Kubernetes is a plus.

- Strong problem-solving skills and ability to work in a fast-paced environment.

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Key Responsibilities :

- Design, develop, and maintain scalable data engineering pipelines

- Build and optimize APIs for data processing and integration

- Work with various data stores including Snowflake, PostgreSQL, and SQL databases

- Create and implement efficient ETL processes to transform raw data into actionable insights

- Collaborate with data scientists and analysts to understand data requirements

- Ensure data quality, integrity, and security across all data systems

- Develop testing protocols for data pipelines and APIs

- Document technical processes and maintain system architecture diagrams

- Troubleshoot and resolve data-related issues