IN_Senior Associate _Azure DE + Databricks Developer

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics

We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology

We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge

Creating business intelligence from data requires an understanding of the business, the data, and the technology used to store and analyse that data

Using our Rapid Business Intelligence Solutions, data visualisation and integrated reporting dashboards, we can deliver agile, highly interactive reporting and analytics that help our clients to more effectively run their business and understand what business questions can be answered and how to unlock the answers

Why PWC

At PwC, you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes for our clients and communities

This purpose-led and values-driven work, powered by technology in an environment that drives innovation, will enable you to make a tangible impact in the real world

We reward your contributions, support your wellbeing, and offer inclusive benefits, flexibility programmes and mentorship that will help you thrive in work and life

Together, we grow, learn, care, collaborate, and create a future of infinite experiences for each other

Learn more about us

At PwC, we believe in providing equal employment opportunities, without any discrimination on the grounds of gender, ethnic background, age, disability, marital status, sexual orientation, pregnancy, gender identity or expression, religion or other beliefs, perceived differences and status protected by law

We strive to create an environment where each one of our people can bring their true selves and contribute to their personal growth and the firm’s growth

To enable this, we have zero tolerance for any discrimination and harassment based on the above considerations

"

Responsibilities

Data Lake and Lakehouse Implementation:

Design, implement, and manage Data Lake and Lakehouse architectures

(Must have)

Develop and maintain scalable data pipelines and workflows

(Must have)

Utilize Azure Data Lake Services (ADLS) for data storage and management

(Must have)

Knowledge on Medalion Architecture, Delta Format

(Must have)

Data Processing and Transformation:

Use PySpark for data processing and transformations

(Must have)

Implement Delta Live Tables for real-time data processing and analytics

(Good to have)

Ensure data quality and consistency across all stages of the data lifecycle

(Must have)

Data Management and Governance:

Employ Unity Catalog for data governance and metadata management

(Good to have)

Ensure robust data security and compliance with industry standards

(Must have)

Data Integration:

Extract, transform, and load (ETL) data from multiple sources (Must have) including SAP (Good to have), Dynamics 365 (Good to have), and other systems

Utilize Azure Data Factory (ADF) and Synapse Analytics for data integration and orchestration

(Must have)

Performance Optimization of the Jobs

(Must have)

Data Storage and Access:

Implement and manage Azure Data Lake Storage (ADLS) for large-scale data storage

(Must have)

Optimize data storage and retrieval processes for performance and cost-efficiency

(Must have)

Collaboration and Communication:

Work closely with data scientists, analysts, and other stakeholders to understand data requirements

(Must have)

Provide technical guidance and mentorship to junior team members

(Good to have)

Continuous Improvement:

Stay updated with the latest industry trends and technologies in data engineering and cloud computing

(Good to have)

Continuously improve data processes and infrastructure for efficiency and scalability

(Must have)

Required Skills And Qualifications

Technical Skills:

Proficient in PySpark and Python for data processing and analysis

Strong experience with Azure Data Lake Services (ADLS) and Data Lake architecture

Hands-on experience with Databricks for data engineering and analytics

Knowledge of Unity Catalog for data governance

Expertise in Delta Live Tables for real-time data processing

Familiarity with Azure Fabric for data integration and orchestration

Proficient in Azure Data Factory (ADF) and Synapse Analytics for ETL and data warehousing

Experience in pulling data from multiple sources like SAP, Dynamics 365, and others

Soft Skills:

Excellent problem-solving and analytical skills

Strong communication and collaboration abilities

Ability to work independently and as part of a team

Attention to detail and commitment to data accuracy and quality

Certifications Required

Certification in Azure Data Engineering or relevant Azure certifications

DP203 (Must have)

Certification in Databricks

Databricks certified Data Engineer Associate (Must have)

Databricks certified Data Engineer Professional (Good Have)

Mandatory skill sets: Azure DE, Pyspark, Databricks

Preferred skill sets: Azure DE, Pyspark, Databricks

Years of experience required: 5-10 Years

Educational Qualification: BE, B

Tech, MCA, M

Tech

Education (if blank, degree and/or field of study not specified)

Degrees/Field of Study required: Master of Engineering, Bachelor of Engineering

Degrees/Field Of Study Preferred

Certifications (if blank, certifications not specified)

Required Skills

Databricks Platform, Microsoft Azure

Optional Skills

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Available for Work Visa Sponsorship?

Government Clearance Required?

Job Posting End Date