Azure Data Engineer

Must Have Capabilities

Good Verbal and Written communication skills
Design, develop, and maintain scalable and robust data pipelines to ingest, process, transform, and store large volumes of data from various sources.
Design and implement data models, schemas, and data architecture to support the needs of the business, ensuring data integrity, consistency, and performance.
Integrate data from disparate sources and systems, ensuring compatibility, consistency, and accuracy across the data ecosystem.
Implement data quality checks and validation processes to ensure data accuracy, completeness, and consistency throughout the data lifecycle.
Implement data orchestration framework and setting up CI/CD pipelines.
Write complex SQL queries and scripts to query large datasets and assist in data analysis.

Tools

Azure - Data Factory, Data Lake, Synapse, Databricks [Must have]
Spark, Python, Hive, HDFS, Sqoop, Kafka
MS SQL, Postgres, MySQL, Teradata, Oracle, DB2, HBase,
GitHub, Bit Bucket, Control M, JIRA, Jenkins, Confluence
Azure Cloud
Familiarity with Power BI for visualizing and reporting data insights.

Optional Capabilities

Develop consumers for stream processing platforms such as Kafka.
Implement Big Data solutions for Data Ingestion, Transformation and Analytics, including ACID transactions and Change Data Capture (CDC) scenarios.

Certification

Databricks Certified Data Engineer
Microsoft Certified Azure Data Engineer

Skills:

• Azure Data Factory
• Azure Active Directory
• Azure Data Lake