21 Lead Data Engineer Jobs in Hyderabad / Secunderabad

Job Summary

With the acceleration of digital transformation (DX) and rapid changes in business models and IT systems at firms nowadays, we are focusing on the importance of data and promote the utilization of data to provide more value to our customers with speed.
In this regard, from a data engineering perspective, our IT organization is rebuilding and integrating legacy on-premises data infrastructures into the cloud as one data platform that seamlessly connects various internal and external data sources.

This role will lead to solve business challenges with a proven track record as experienced data engineer, in collaboration with business and IT teams.
The ideal candidate should have a technical backbone with hands on experience in various disciplines of Data Engineering leading an army of Engineers.

Responsibilities:

• Leads the creation of architectural specifications, ensuring feasibility, functionality, and integration with existing systems/platforms. Works towards enabling self service capabilities for business partners.
• Lead the collection and documentation of user requirements as well as lead the development of user stories and estimates into technology solutions for large complex features in collaboration with Product Owner.
• Lead the detailed technical design and development of applications/platforms. Partner with Product owner in planning. Responsible for ensuring disaster recovery, privacy and security are aligned to enable

application/platform stability.

• Leads peer reviews of solution designs and related code for complex features. Ensures code meets best practice guidelines set by application/platform vendor.
• Investigate and resolve production management incidents, problems, and service requests. Manages prioritization of problems, incidents, and service requests with Product Owner.
• Leads team in production support activities of complex nature and/or Product. Leads the effort in upgrading or patching applications/platforms as needed.
• Maintain awareness of trends and issues in area of technical expertise, evaluate new technologies or technology opportunities, and provide analysis of their potential impact to champion the customer
• Lead the analysis and design of data pipelines that bring in various internal and external data into the data lake in the cloud based on the business use case.
• Lead the design and development of data marts that make large volumes of data from the data lake accessible to business analysts and data scientists.
• As Lead data engineer , contribute to the development of each team member and the maturation of the team.
• Understand and align with strategy of company-wide / IT organization. Drive projects and delivery.
• Lead delivery teams in an agile delivery model.
• Solution definition and architecture design for assigned areas.
• Project management or PMO (including budget and vendor management)
• BAU management of data engineering teams (e.g., system availability, change release, incident and problem management
• Alignment to global IT strategy, collaboration with MetLife global or regional teams
  
  Qualifications
  Experience:
  
• 10+ years data engineering or application development experience (ETL/ELT/BI)
• 5+ years' experience in project management or PMO
• 5+ years' experience in BSA for application development or Data Analyst
• 3+ years cloud-based application or data platform development
• 2+ years of experience developing or participating in Agile, Scrum, or SAFe
• 5+ years' experience leading project with a few members
• 5+ years' experience managing offshore teams of around 10 people.

Basic Expectations:

• Expert level knowledge on building Data Pipelines on Azure Platform using various services including but not limited to

Azure Data Factory/ Synapse
 Data Bricks / Synapse Spark pool
 Cosmos DB
 ADLS
 Dedicated SQL Pool / Azure SQL
 Logical Apps

• Ingestion of structured / unstructured data / semi-structured data into the data lake
• Hands on experience doing data Transformation/Cleansing using programming languages such as Spark/Python/R/SQL
• Hands on DevOps and CI/CD using Azure DevOps
• Working experience on Project delivery using Agile methodologies.

Desirable Skills:

• Planning, evaluating and piloting use cases using Generative AI
• Experience working with Event Hubs and Streaming solutions
• Domain knowledge for Life Insurance