Databricks Developer

About the Role We are seeking a highly motivated and skilled Databricks Engineer to join our growing Data Engineering team. In this role, you will play a critical part in designing, developing, and maintaining data pipelines and solutions on the Databricks platform. You will work closely with cross-functional teams to deliver high-quality data solutions that drive business insights and support key initiatives. Responsibilities: • Design, develop, and deploy data pipelines using Databricks, including data ingestion, transformation, and loading (ETL) processes. • Develop and maintain high-quality, scalable, and maintainable Databricks notebooks using Python. • Work with Delta Lake and other advanced features. • Leverage Unity Catalog for data governance, access control, and data discovery. • Develop and optimize data pipelines for performance and cost-effectiveness. • Integrate with various data sources, including but not limited to databases and cloud storage (Azure Blob Storage, ADLS, Synapse), and APIs. • Experience working with Parquet files for data storage and processing. • Experience with data integration from Azure Data Factory, Azure Data Lake, and other relevant Azure services. • Perform data quality checks and validation to ensure data accuracy and integrity. • Troubleshoot and resolve data pipeline issues effectively. • Collaborate with data analysts, business analysts, and business stakeholders to understand their data needs and translate them into technical solutions. • Participate in code reviews and contribute to best practices within the team. Qualifications: • 10+ years of experience as a Data Engineer with a strong focus on Databricks. • Strong proficiency in Python and SQL. • Hands-on experience with Databricks features such as delta lake, clusters, notebooks, jobs, and workspaces. • Experience with data warehousing concepts and best practices. • Strong understanding of data modeling and data quality principles. • Experience with Agile development methodologies. • Excellent communication and collaboration skills. • Ability to work independently and as part of a team. Preferred Qualifications: • Experience with Azure cloud services (e.g., Azure Data Factory, Azure Synapse Analytics) • Experience with CI/CD pipelines for Databricks workloads. • Experience with data visualization tools (e.g., Power BI, Tableau) • Experience with data security and privacy best practices Role & responsibilities

Preferred candidate profile

Perks and benefits