Azure Data Engineer - ETL/PySpark (5-8 yrs)

Responsibilities :

- Develop and maintain data solutions on Databricks and Azure cloud platforms (Data Lake, Data Factory, Synapse, and SQL Database).

- Build and optimize large-scale data processing pipelines using Apache Spark (PySpark).

- Design and implement advanced SQL queries for data integration and analytics.

- Create and maintain efficient data models, with expertise in data modeling principles and data warehousing concepts.

- Lead and manage ETL/ELT workflows for large datasets and data transformations.

- Ensure data security and compliance on Databricks and Azure.

- Stay updated with emerging trends in big data technologies, cloud-based architectures, and best practices.

Qualifications :

- Proficiency in Azure Databricks, Apache Spark, and cloud platforms (specifically Azure).

- Strong skills in Python, Scala, or SQL for data processing and transformation.

- Strong understanding of data modeling principles, ETL/ELT workflows, and data warehousing concepts.

- Experience with large datasets, distributed computing systems, and big data technologies.

- Familiarity with DevOps practices, CI/CD pipelines, and version control tools like Git.

- Knowledge of cloud-based architectures and solutions, especially on Azure.

- Experience in handling data security measures and policies on Databricks and cloud platforms