Principal Consultant - Azure Databricks (7-10 yrs)

Role : Principal Consultant- Databricks Developer Azure!

In this role, the Databricks Developer is responsible for solving the real-world cutting-edge problem to meet both functional and non-functional requirements.

Responsibilities :

- Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.

- Work with architect and lead engineers for solutions to meet functional and non-functional requirements.

- Demonstrated knowledge of relevant industry trends and standards.

- Demonstrate strong analytical and technical problem-solving skills.

- Must have experience in Data Engineering domain.

Qualifications we seek in you!

Minimum qualifications :

- Bachelor's Degree or equivalency (CS, CE, CIS, IS, MIS, or engineering discipline) or equivalent work experience.

- Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.

- Work with architect and lead engineers for solutions to meet functional and non-functional requirements.

- Demonstrated knowledge of relevant industry trends and standards.

- Demonstrate strong analytical and technical problem-solving skills.

- Must have excellent coding skills either Python or Scala, preferably Python.

- Must have experience in Data Engineering domain .

- Must have implemented at least 2 project end-to-end in Databricks.

Must have at least experience on databricks which consists of various components as below :

- Delta lake

- dbConnect

- db API 2.0

- Databricks workflows orchestration

- Must be well versed with Databricks Lakehouse concept and its implementation in enterprise environments.

- Must have good understanding to create complex data pipeline

- Must have good knowledge of Data structure & algorithms.

- Must be strong in SQL and sprak-sql.

- Must have strong performance optimization skills to improve efficiency and reduce cost.

- Must have worked on both Batch and streaming data pipeline.

- Must have extensive knowledge of Spark and Hive data processing framework.

- Must have worked on any cloud (Azure, AWS, GCP) and most common services like ADLS/S3, ADF/Lambda, CosmosDB/DynamoDB, ASB/SQS, Cloud databases.

- Must be strong in writing unit test case and integration test

- Must have strong communication skills and have worked on the team of size 5 plus

- Must have great attitude towards learning new skills and upskilling the existing skills.

Preferred Qualifications :

Good to have :

- Unity catalog and basic governance knowledge.

- Good to have Databricks SQL Endpoint understanding.

- Good To have CI/CD experience to build the pipeline for Databricks jobs.

- Good to have if worked on migration project to build Unified data platform.

- Good to have knowledge of DBT.

- Good to have knowledge of docker and Kubernetes.