ADTECH Corp - Data Engineer - SQL/Python (6-8 yrs)

Role : Data Engineer

Experience : 5+

Location : Bangalore

Must have : Python or C#

Key Responsibilities :

- Develop and maintain ETL pipelines using Azure Data Factory to ingest data from various sources (e. , databases, files, APIs) into Azure Blob Storage.

- Implement data validation and cleansing techniques to ensure data quality.

- Use Azure Databricks to develop and optimize data transformations using Python or SQL.

- Create complex data transformations, aggregations, and joins to derive insights from data.

- Load transformed data into data warehouses and data lakes for analysis and reporting.

- Ensure data integrity and consistency throughout the data pipeline.

- Implement data governance policies and procedures to maintain data quality and security.

- Collaborate with IT and IT Security teams to adhere to organizational security standards.

- Monitor data pipelines for performance and errors.

- Troubleshoot and resolve issues in a timely manner.

- Perform regular maintenance activities to optimize data pipelines.

- Work closely with data analysts, data scientists, and business stakeholders to understand their data needs.

- Communicate effectively with technical and non-technical audiences.

Required Skills and Experience :

- 4+ years of experience in data engineering.

- Strong proficiency in Python or C#.

- Experience with SQL and data warehousing concepts.

- Experience with Azure Data Factory, Azure Databricks, and Azure Blob Storage.

- Understanding of data modeling and data warehousing concepts.

- Experience with data quality and data governance practices.

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration skills.

Desired Skills :

- Experience with other cloud platforms (AWS, GCP).

- Knowledge of data visualization tools (Power BI, Tableau).

- Experience with machine learning and AI