Data Engineer - Python/PySpark (4-8 yrs)

Job Titile : Data Enineer

Experince : 4 to 8 years

Primary Skills : Azure Data Factory, Synapse Analytics, Databricks, ADLS, SQL, PySpark, and Python

Objective :

A "Data Engineer" is mid-level professional leading the design, build and evolution of the inhouse data platforms. You lead the construction of datasets requested by various stakeholders, the build and maintenance of the resulting data pipelines. You closely work in collaboration with our data analysts, data scientist and business application development teams in delivering data solutions.

Productivity and Quality :

- Provide subject matter expertise for data engineering and implementation of high-performing, secure and cost-efficient data solutions and pipelines

- Provide guidance on how to sensibly evolve solutions within the Business Application Group's platform(s) of your expertise

- Develop and drive data engineering best practices to further elevate the Business Applications Group's standing as a reliable data intelligence innovation partner

Collaboration :

- Actively participate in the development and review of data standards, solution documentation and implementation within the Business Application Group's platform(s) of your expertise as well as providing guidance to junior team members in form of active peer reviews

- Establish strong relationship with business stakeholders, team members and work closely with platform teams across Business Application Group's chapters

Leadership

- Stay on the leading edge of data engineering practices and remain current on new technologies, available vendor packages and proactively take on initiative to optimize legacy data services/applications

- Mentor junior colleagues on how to drive skills/knowledge development through advice, coaching, knowledge transfer and training opportunities

Compliance :

- Timely complete all compliance obligations and trainings, including relevant professional certification if applicable

- Thoroughly understand and guide team members on the compliance, security and operational responsibilities in your area of work

Hiring guidance :

- The typical education and professional experience required for a Senior Data Engineer is a master's degree plus at least 4 years of working experience or working in similar role.

- Experience in Azure Data Factory, Synapse Analytics, Databricks, ADLS, SQL, PySpark, and Python Programming.

- A successful candidate has proven experience leading a minimum of one large scale and 3 mid-scale projects or managed complex data pipelines/environments in the past.

- Related certifications or relevant accreditations are preferred.