Azimuth GRC - Senior Databricks Developer - Big Data Technologies (8-15 yrs)

Job Description :

Responsibilities :

- Conceptualize and communicate Data architecture strategy, technical design and technology roadmap for data platform solutions and services.

- Lead design and development of data architecture to support implementation of large-scale data solutions in Databricks to support multiple use cases (delta lake, cloud Datawarehouse migrations, reporting and analytics).

- Guiding the organization and development teams on the overall data processes, architectural approaches for data strategy design and implementation.

- Providing data solutions that enable business intelligence data integration, data services, self-service analytics, and data-driven digital products and services.

- Articulating the value proposition of cloud modernization/transformation to stakeholders; creating detailed documentation that empowers other engineers and customers.

- Architecting the solutions for Databricks-specific use cases (streaming and batch) as well as integrating it into other solution components in the customer architecture landscape.

- Translating advanced business data, integration and analytics problems into technical approaches that yield actionable recommendations, across multiple, diverse domains.

- Implement architectural improvements for existing solutions using legacy modernization and cloud adoptions strategies.

- Creating Azure Data Factory processes for SQL and NoSQL data pipelines, and working in Azure SQL, to create tables, views, stored procedures, and other database objects.

- Creating applications through Scala, Python, and Java.

- Developing Spark tasks for data aggregation and transformation

- Writing Scala style documents with each code.

- Identifying technical issues (performance bottlenecks, data discrepancies, system defects) in data and software, perform root cause analysis, and communicate results effectively to the development team and other stakeholders.

- Providing recommendations on opportunities and improvements and participating in technology and analytics proof-of-concepts

- Managing a team

Desired Profile :

- 8 to 10+ years of experience working with Databricks, Apache Spark, and big data technologies.

- 8+years of experience with Azure Databricks, Azure Data Factory, and Pyspark

- Highly proficient in use of Spark and Azure Databricks

- 8 years' experience working in Microsoft Azure

- Proven experience in designing and implementing data solutions and ETL pipelines.

- Strong understanding of data architecture, data modeling, and data warehousing concepts.

- Proficiency in Databricks and Spark, including cluster management, job scheduling, and performance tuning.

- Experience with cloud platforms such as AWS, Azure, or Google Cloud.

- Hands-on working knowledge of topics such as data security, messaging patterns, ELT, Data wrangling and cloud computing and proficiency in data integration/EAI and DB technologies, sophisticated analytics tools, programming languages or visualization platforms

- Knowledge of the Databricks ecosystem

- Team Management Experience

- Strong programming skills in languages such as Python, Scala, or SQL.

- Experience with data integration tools and technologies (e.g., Kafka, Delta Lake).

- Knowledge of data governance, security, and compliance best practices

- Bachelor's degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees or certifications are a plus