Senior Data Engineer - Azure Cloud (6-10 yrs)

Job Title : Senior Data Engineer

Experience : 6 to 10 Years

Location : Indore, India

About the Job :

We are seeking a highly skilled and motivated Senior Data Engineer with deep expertise in Microsoft Azure cloud technologies to join our dynamic data team in Indore. This role will be instrumental in designing, building, and maintaining robust and scalable data pipelines and warehousing solutions to support our growing data-driven initiatives. You will leverage your extensive experience with Azure Data Factory, Azure Synapse, SQL, and Power BI to empower our business with actionable insights.

Responsibilities :

Azure Data Pipeline Development and Optimization :

- Design, develop, and deploy complex ETL/ELT pipelines using Azure Data Factory (ADF) for data ingestion, transformation, and loading.

- Optimize ADF pipelines for performance, scalability, and reliability, including monitoring and troubleshooting.

- Implement data quality checks and validation processes within ADF pipelines.

Azure Synapse Analytics Implementation and Management :

- Design and implement data warehousing solutions using Azure Synapse Analytics, including dedicated SQL pools and serverless SQL pools.

- Develop and optimize SQL queries and stored procedures for data retrieval and analysis within Synapse.

- Manage and maintain Synapse data models and schemas.

SQL Development and Optimization :

- Write complex SQL queries, stored procedures, and functions for data manipulation and analysis.

- Optimize SQL queries for performance and efficiency.

- Design and implement database schemas and data models.

Power BI Dashboard Development and Reporting :

- Develop interactive and insightful Power BI dashboards and reports to visualize data and communicate key business metrics.

- Design and implement data models within Power BI for efficient data analysis.

- Ensure data accuracy and consistency in Power BI reports.

Data Modeling and Warehousing :

- Design and implement scalable and efficient data models for data warehousing and analytics.

- Apply data warehousing principles and techniques (e.g., star schema, snowflake schema) to optimize data storage and retrieval.

- Ensure data integrity and consistency across data platforms.

ETL/ELT Methodology Implementation :

- Apply and adapt ETL/ELT methodologies to manage and transform large datasets from various sources.

- Design and implement data integration solutions for diverse data sources.

- Automate data integration and transformation processes.

Collaboration and Communication :

- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver data solutions.

- Communicate technical concepts and solutions effectively to both technical and non-technical audiences.

- Participate in code reviews and contribute to team knowledge sharing.

Performance Monitoring and Troubleshooting :

- Monitor the performance of data pipelines and data warehousing solutions.

- Troubleshoot and resolve data-related issues.

- Implement monitoring and alerting solutions for data infrastructure.

Data Governance and Security :

- Adhere to data governance and security policies and procedures.

- Implement data access controls and ensure data privacy.

Required Skills and Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. -

- 6 to 10 years of experience in data engineering, with a strong focus on Azure technologies.

- Extensive hands-on experience with Azure Data Factory (ADF), Azure Synapse Analytics, SQL Server, and Power BI.

- Strong proficiency in SQL programming and database design.

- Deep understanding of ETL/ELT methodologies and data warehousing concepts.

- Experience in designing and implementing scalable data models and data pipelines.

- Excellent problem-solving and analytical skills.

- Strong communication and collaboration skills.

- Understanding of data governance and security best practices.

- Experience with other azure data services like Azure Databricks, Azure data lake storage is a plus.

- Experience in CI/CD pipelines for data engineering is a plus.