Senior Data Engineer

Roles & RESPONSIBILITIES

As an Azure Data Engineer, you will be responsible for designing, building, and maintaining data pipelines to support business and analytics needs. You will work closely with data analysts and business stakeholders to understand their needs and develop solutions to meet those needs.

• 6+ Years of Experience in Azure with Data Engineering.
• 4+ Hands on experience in Azure technologies such as Azure Data Factory, Azure Databricks and Azure Synapse Analytics, Azure Data Lake
• Design, develop, manage data flows that integrate information from various sources into a common data lake using Microsoft Azure data Factory.
• Extract, transform and load data from variety of sources, including on-premises databases, cloud-based data stores and API calls.
• 6+ years of working with SQL, implement data processing and transformation logic using SQL, handling complex windowing functions and joins.
• Optimize and tune data pipelines for performance, scalability, and cost-efficiency, leveraging best practices and automation tools.
• Ensure data quality, integrity, and security throughout the data lifecycle, implementing data governance policies and compliance standards.
•  Collaborate with data analysts and business stakeholders to understand their data needs and develop data models and solutions to meet those needs.

Monitor and troubleshoot data pipelines, identifying and resolving issues in a timely manner to minimize impact on business operations.

Competencies & Experience Required/Desired

• Any Degree
• Proven experience as a data engineer, with a focus on designing and building data solutions on Azure.
• Knowledge of Azure data services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL Datawarehouse, and Azure Data Lake Storage.
• Strong proficiency in SQL.
• Experience with data modeling, schema design, and performance optimization techniques.
• Expert in data integration patterns, ETL/ELT processes, and data warehousing concepts.
• Excellent problem-solving and analytical skills, with the ability to troubleshoot complex data issues and drive resolution.
• Effective communication and collaboration skills, with the ability to work effectively in a remote and cross-functional team environment.

To summarize: Experience working on ETL, Microsoft Azure SQL Data Warehouse, SQL Scripting, with exposure to other services on the Azure landscape, API Data Extraction