Lead/Manager Data Engineer

You will be in the role of xcelhires working with end customer.

• Proven experience leading technical teams in designing, building, and implementing end [1] to-end data pipelines.
• Strong experience leading large-scale projects with multiple project pods, collaborating with cross-functional teams such as Data Science, BI, and Infrastructure teams.
• Demonstrated ability to lead engineering at all levels, providing technical direction, mentoring team members, and ensuring alignment across architecture, development, and deployment processes.
• Strong experience in managing senior client stakeholder expectations, with the ability to bridge technical and business discussions effectively.
• Demonstrated success in designing, building, and deploying analytical data infrastructure, including defining end-to-end data architecture and delivering robust solutions.
• Comprehensive understanding of large-scale data concepts such as data warehouses, data lakes, and machine learning solutions, including best practices for their implementation and management.
• Experience with at least one big data platform such as Data bricks, Snowflake, or AWS Big Data services (e.g., Redshift, EMR, Glue, Athena, S3).
• Experience in MLOps practices, including the development and orchestration of model training pipelines and real-time or batch model inference pipelines.
• Hands-on experience with model tracking and versioning tools (e.g., ML flow, Tensor Board), ensuring visibility and reproducibility of machine learning models.
• Ability to implement techniques for monitoring model drift and building processes for automatic model retraining and updates to ensure sustained model performance.
• Experience working across development, test, and production environments, with a solid understanding of CI/CD principles and tools for automated deployments and testing, including for machine learning workflows.
• Extensive experience in ETL technical design, development, optimization, and support, ensuring seamless data flow and transformation processes.
• Expert-level proficiency in Python, with the ability to design and implement complex data engineering and machine learning solutions.
• Advanced SQL skills, including query optimization, complex joins, and handling large datasets.
• Strong expertise with at least one cloud platform (AWS, Azure, or GCP), including cloud-native services for data processing, storage, compute, and ML/AI services.
• Deep understanding of enterprise-level coding best practices, including modular design, code reviews, and test-driven development, with experience using top-tier source code repositories (e.g., Git).
• Ability to establish and implement data governance frameworks, ensuring security, compliance, and data quality throughout the pipeline lifecycle.
• Self-starter mind set with excellent problem-solving abilities and a proactive approach challenges.

• Tech Stack : Total Experience : 8+ Years with minimum 4 Year s experience into at least one big data platform such as Data bricks, Snowflake, or AWS Big Data services
• Machine Learning: Hands-on experience with model tracking and versioning tools (e.g., ML flow, Tensor Board), ensuring visibility and reproducibility of machine learning models.
• Cloud Experience: Strong expertise with at least one cloud platform (AWS, Azure, or GCP), including cloud-native services for data processing, storage, compute, and ML/AI services.
• Programming Language : Expert-level proficiency in Python and Advanced SQL skills
• People Management : Proven experience leading technical teams in designing, building, and implementing end- to-end data pipelines.