AU Small Finance Bank - Lead Data Engineer - ETL/Data Warehousing (9-12 yrs)

Job Description :

This role is for a Lead Data Engineer who specializes in building and managing Data Warehouses (DWH) on the AWS cloud platform. You'll be responsible for designing, developing, and maintaining data solutions that support the company's business needs through reporting, integrations, and analytics.

Key Responsibilities :

- Lead DWH projects from end to end, including technical solution design, team management, and timely delivery.

- Design and architect data solutions aligned with strategic business goals.

- Manage all aspects of the data lifecycle : extraction, transformation, loading (ETL/ELT), data cleansing, data quality, and data visualization.

- Build, maintain, and monitor batch and real-time ETL pipelines using AWS services like Kinesis, DMS, S3, EMR, and Redshift.

- Develop a data strategy encompassing data source, flow, storage, and usage.

- Perform data validation and quality assurance.

- Present technical solutions to stakeholders.

- Provide ongoing support for the DWH and data lake environments.

- Collaborate with Data Analytics teams to understand their needs.

Preferred Skills and Experience :

Technical Skills :

- 9+ years of experience in the data field.

- 5+ years of experience in Data Warehousing concepts.

- 3+ years of hands-on experience with AWS.

- 5+ years of building ETL pipelines.

- 3+ years of experience with job scheduling frameworks.

- 3+ years of experience in performance tuning SQL queries.

- Knowledge of Spark (DataFrames, Spark-Shell, Lazy evaluation).

- Strong proficiency in SQL, especially Spark SQL.

- 2+ years of Python skills.

- 1+ year of experience with data visualization tools.

- 3+ years of experience in data modeling, architecture, and designing data systems.

Soft Skills :

- Experience handling client/business teams.

- Excellent communication skills across technical and non-technical audiences.

- Strong documentation skills.

Education : Bachelor's or Master's degree in Engineering (BE, B.Tech, M.Tech) preferred