Data Engineer - Python/ETL (2-5 yrs)

Job Description :

As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines to extract, transform, and load (ETL) data from various sources into target systems.

You will work closely with data analysts and data scientists to ensure data quality, accuracy, and consistency.

Key Responsibilities :

- Design, develop, and maintain ETL processes using Python, Java, and other relevant tools.

- Extract, transform, and load data from various sources (databases, files, APIs) into target systems.

- Ensure data quality, integrity, and consistency throughout the ETL process.

- Design and implement efficient data models to support business needs.

- Create and maintain data mappings, transformations, and data flows.

- Integrate data from multiple sources, including databases, files, and APIs.

- Identify and resolve performance bottlenecks in ETL processes.

- Implement data governance policies and procedures to ensure data quality and security.

Required Skills and Experience :

- Strong proficiency in Python and Java programming languages.

- Experience with data engineering tools and frameworks (e., Apache Spark, Apache Airflow, Kafka).

- Knowledge of SQL and database concepts.

- Understanding of data warehousing and data modeling concepts.

- Experience with cloud platforms (AWS, Azure, GCP).

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration skills.

- CS Background

Preferred Skills :

- Experience with machine learning and AI frameworks (e., TensorFlow, PyTorch).

- Knowledge of data visualization tools (e., Tableau, Power BI).

- Experience with data streaming and real-time processing