Acceskeys - Azure Data Lake Professional - Data Factory (4-8 yrs)

Key Responsibilities :

Azure Data Lake Development :

- Design and develop scalable and efficient data storage solutions using Azure Data Lake Gen1/Gen2.

- Implement data ingestion, transformation, and processing pipelines.

Data Integration and Management :

- Develop and maintain ETL processes using Azure Data Factory (ADF).

- Integrate structured, semi-structured, and unstructured data from multiple sources into Azure Data Lake.

Data Governance and Security :

- Implement data governance policies, including role-based access control (RBAC).

- Ensure compliance with security and regulatory requirements for data storage and processing.

Optimization and Troubleshooting :

- Optimize data lake storage and processing for cost and performance efficiency.

- Troubleshoot and resolve technical issues related to data pipelines and storage.

Collaboration and Documentation :

- Work with data scientists, analysts, and other stakeholders to understand data requirements.

- Document processes, configurations, and best practices for team reference.

Required Skills and Qualifications :

- Strong experience with Azure Data Lake (Gen1/Gen2).

- Proficiency in Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.

- Hands-on experience with big data processing frameworks like Spark or Hadoop.

- Strong programming skills in Python, Scala, or SQL.

- Understanding of data modeling, data partitioning, and schema design.

- Familiarity with data governance, security best practices, and monitoring tools in Azure.

- Knowledge of Azure Blob Storage and Azure DevOps for CI/CD pipelines.

- Strong analytical and problem-solving skills.

Preferred Qualifications :

- Microsoft Azure certifications (e.g., Azure Data Engineer Associate, Azure Developer Associate).

- Experience with real-time data processing tools like Kafka or Event Hub.

- Background in data warehousing and analytics solutions.

Educational Requirements :

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.