Senior Data Engineer

Details of the role:

Experience: 8+ years

We are seeking a skilled and detail-oriented Azure Data Engineer to join our team. In this role, you will design and implement robust data architectures and workflows using Azures data services. The ideal candidate will play a critical role in developing centralized data storage solutions and analytics frameworks to meet business requirements.

Key Responsibilities

1. Data Architecture Design
   • Develop scalable data architecture using Azure Data Lake and Azure Synapse Analytics for centralized data storage and management.
   • Ensure the architecture supports structured and unstructured data processing needs.
2. ETL Workflow Development
   • Design and implement ETL workflows for processing and transforming data from diverse sources.
   • Automate and optimize data ingestion, transformation, and loading processes for real-time and batch scenarios.
3. Analytics Framework Definition
   • Define and implement data analytics frameworks that integrate seamlessly with the data architecture.
   • Support advanced analytics, reporting, and machine learning use cases.
4. Data Governance and Security
   • Establish data governance policies to ensure data quality, compliance, and security.
   • Implement robust access controls and monitoring mechanisms across the data ecosystem.
5. Collaboration and Support
   • Work closely with data scientists, business analysts, and stakeholders to understand data requirements and deliver tailored solutions.
   • Provide technical guidance and troubleshooting support for data infrastructure issues.

Required Skills and Qualifications

• Technical Expertise:
  • Strong hands-on experience with Azure Data Lake, Azure Synapse Analytics, and related Azure data services.
  • Proficiency in designing and optimizing ETL workflows using tools like Azure Data Factory or Databricks.
  • Experience working with structured (SQL, relational databases) and unstructured data (JSON, NoSQL databases).
  • Knowledge of data modeling, schema design, and query optimization.
• Programming and Tools:
  • Proficiency in Python, Scala, or SQL for data manipulation and transformation.
  • Familiarity with big data technologies and distributed processing frameworks (e.g., Spark, Hadoop).
• Analytics and Reporting:
  • Understanding of data visualization tools like Power BI, Tableau, or similar.
  • Ability to integrate data pipelines with analytics platforms for actionable insights.
• Soft Skills:
  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration abilities to work with cross-functional teams.

Preferred Qualifications

• Certification in Microsoft Azure Data Engineer Associate or equivalent.
• Experience with real-time data streaming using Azure Event Hubs, Kafka, or Azure Stream Analytics.