Azure Databricks, ADF, Azure DevOps, Python, Pyspark

No of years' experience 6 8 years Detailed job description Skill Set:.

Must have excellent coding skills either Python or Scala, preferably Python.

Must have experience in Data Engineering domain.

Must have knowledge of Databricks cluster optimization and its integration with various cloud services.

Must have strong performance optimization skills to improve efficiency and reduce cost.

Must have strong communication skills and have worked with cross platform team.

Must have great attitude towards learning new skills and upskilling the existing skills.

Responsible to set best practices around Databricks CI/CD.

Must understand composable architecture to take fullest advantage of Databricks capabilities.

Strong expertise in Apache Spark, Delta Lake, and other Databricks components for data processing and analytics.

Delta lake.

db API 2-0.

SQL Endpoint Photon engine.

Unity Catalog.

Security management.

Platform governance.

Data Security.

Serverless.

Good to have Rest API knowledge.

Good to have understanding around cost distribution.

Mandatory Skills Azure Databricks, ADF, Azure DevOps, Python, pyspark.