Aeonovatech - Senior Big Data Engineer - Azure Databricks & Spark (4-7 yrs)

Key Responsibilities :

- Lead and mentor a team of Big Data Engineers.

- Participate in the design and implementation of the big data analytics architecture on Azure.

- Develop and implement big data pipelines using Spark Scala and Python.

- Write complex Hive queries to extract valuable insights from data lakes.

- Collaborate with cross-functional teams (QA, Development) to ensure successful project delivery.

- Provide technical guidance and resolve queries from team members.

- Deliver projects on time and within budget following Agile methodologies (Scrum).

- Maintain a strong understanding of big data best practices and stay up-to-date on the latest advancements in Azure Databricks and Spark.

Required Skills :

- 6+ years of experience in big data development and implementation.

- 4+ years of experience as a lead developer in a big data environment.

- Proven experience with Azure Databricks, Databricks Notebooks, and Spark Scala.

- Strong understanding of big data concepts (Hadoop ecosystem, data lakes, data warehousing).

- Experience in building and tuning Spark pipelines.

- Proficiency in writing complex Hive queries.

- Excellent communication, collaboration, and leadership skills.

- Ability to work effectively in an Agile development environment