Lead Data Engineer - Python/SQL (7-12 yrs)

Job Description :

Requirements :

- Proven experience leading technical teams in designing, building, and implementing end-to-end data pipelines.

- Strong experience leading large-scale projects with multiple project pods, collaborating with cross-functional teams such as Data Science, BI, and Infrastructure teams.

- Demonstrated ability to lead engineering at all levels, providing technical direction, mentoring team members, and ensuring alignment across architecture, development, and deployment processes.

- Strong experience in managing senior client stakeholder expectations, with the ability to bridge technical and business discussions effectively.

- Demonstrated success in designing, building, and deploying analytical data infrastructure, including defining end-to-end data architecture and delivering robust solutions.

- Comprehensive understanding of large-scale data concepts such as data warehouses, data lakes, and machine learning solutions, including best practices for their implementation and management.

- Experience with at least one big data platform such as Databricks, Snowflake, or AWS Big-Data services (e. g., Redshift, EMR, Glue, Athena, S3).

- Experience in MLOps practices, including the development and orchestration of model training pipelines and real-time or batch model inference pipelines.

- Hands-on experience with model tracking and versioning tools (e. g., MLflow, TensorBoard), ensuring visibility and reproducibility of machine learning models.

- Ability to implement techniques for monitoring model drift and building processes for automatic model retraining and updates to ensure sustained model performance.

- Experience working across development, test, and production environments, with a solid understanding of CI/CD principles and tools for automated deployments and testing, including for machine learning workflows.

- Extensive experience in ETL technical design, development, optimization, and support, ensuring seamless data flow and transformation processes.

- Expert-level proficiency in Python, with the ability to design and implement complex data engineering and machine learning solutions.

- Advanced SQL skills, including query optimization, complex joins, and handling large datasets.

- Strong expertise with at least one cloud platform (AWS, Azure, or GCP), including cloud-native services for data processing, storage, computing, and ML/AI services.

- Deep understanding of enterprise-level coding best practices, including modular design, code

reviews, and test-driven development, with experience using top-tier source code repositories

(e. g., Git).

- Ability to establish and implement data governance frameworks, ensuring security, compliance, and data quality throughout the pipeline lifecycle.

- Self-starter mindset with excellent problem-solving abilities and a proactive approach to challenges.