Data Engineer

Data Ingestion and Integration:
- Design and implement efficient data ingestion processes from various data sources into the Azure data platform.
- Collaborate with cross-functional teams to understand data requirements and ensure data quality and reliability during the ingestion process.

Data Validation and Transformation (ETL):
- Develop robust data validation and transformation workflows (ETL) to ensure data integrity and consistency.
- Utilize Azure Data Factory, Apache Airflow, and other relevant tools to orchestrate complex data workflows.

Data Governance and Compliance:
- Implement data governance practices to ensure data security, privacy, and compliance with industry regulations and company policies.
- Collaborate with the data governance team to enforce data quality standards and data lineage tracking.

Data Analysis and Reporting:
- Work closely with data analysts and data scientists to support data analysis initiatives and create actionable insights from the data platform.
- Develop data models and structures optimized for analytical and reporting purposes.

Azure Data Platform Management:
- Manage and optimize the Azure data platform, including Azure Synapse, Azure Data Lake Store Gen 2, CosmosDB, and other components.
- Utilize Apache Airflow for workflow scheduling, monitoring, and orchestration.

Data Architecture and Strategy:
- Contribute to the data platforms strategy and architecture, ensuring scalability, flexibility, and alignment with business objectives.
- Participate in design discussions and provide technical expertise to guide data-related decisions.

Documentation and Knowledge Sharing:
- Maintain comprehensive documentation of data processes, architecture, and solutions.
- Share knowledge and best practices with team members to foster a collaborative and learning-oriented culture.