Lead Azure Databricks Developer - Data Modeling & Management (5-8 yrs)

Job Description :

Roles & Responsibilities :

Design and Develop Data Solutions :

- Architect, develop, and optimize scalable data pipelines and ETL processes using Azure Data bricks.

- Implement data integration, data transformation, and data storage solutions.

Data Modeling and Management :

- Design and implement data models and structures that support business intelligence and analytics requirements.

- Manage and maintain the Data bricks environment, ensuring optimal performance and reliability.

Collaboration :

- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

- Collaborate with other engineering teams to integrate Data bricks solutions with existing systems.

Optimization and Performance Tuning :

- Perform data quality checks and ensure data accuracy and consistency.

- Optimize performance of Data bricks jobs and troubleshoot any issues.

Security and Compliance :

- Implement data security and compliance best practices.

- Ensure all solutions adhere to company policies and regulatory requirements.

Documentation and Training :

- Document data processes, solutions, and configurations.

- Provide training and support to end-users and team members as needed.

Technical Skills :

- Proficiency in programming languages such as Python, Scala, or SQL.

- Experience with Azure cloud services (e.g., Azure Data Lake, Azure Data Factory).

- Knowledge of data warehousing concepts and tools.

- Familiarity with CI/CD pipelines and version control systems (e.g., Git) Soft Skills:

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities.

- Ability to work in a fast-paced, dynamic environment.

Preferred Qualifications :

- Experience with other cloud platforms (e.g., AWS, Google Cloud).

- Certifications in Azure or Data bricks.

- Experience with machine learning and data science projects.