Evalueserve - Databricks Engineer - ETL/Python (4-6 yrs)

What you will be doing :

We are looking for a highly motivated and experienced Databricks Developer to design, develop, and optimize our data infrastructure and pipelines. The ideal candidate will have a deep understanding of Apache Spark, Databricks, and big data processing.

Responsibilities :

- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.

- Optimize and tune Spark applications for performance and scalability.

- Implement data processing workflows for ETL, batch, and streaming data.

- Ensure data quality, integrity, and security throughout the data lifecycle.

What we're looking for :

- 4 to 6 years in Data Engineer function applied to finance, ideally Market Risk.

- Proficiency in programming languages such as Python.

- Experience with data warehousing concepts and SQL.

- Familiarity with Delta Lake and data lake architecture.

- Knowledge of Azure cloud platforms and its data services.

- Databricks certification (e.g., Databricks Certified Associate Developer for Apache Spark) is a plus.

- Familiarity with CI/CD processes and tools.

- Ability to quickly absorb financial concepts and apply them to real world situations.