CoStrategix - Lead Data Engineer - ETL/Amazon Athena (9-14 yrs)

We are seeking an experienced and motivated Lead Data Engineer to join our team.

This role will be instrumental in shaping and enhancing our data infrastructure, driving innovative data solutions, and ensuring data quality and reliability across the organization.

The Lead Data Engineer will work closely with cross-functional teams, including data science, analytics, software engineering, and product management, to deliver robust data solutions that support business objectives.

In addition to technical leadership, this role will also focus on people leadership, ensuring the growth, development, and productivity of the data engineering team members.

You will cultivate a collaborative and inclusive environment that promotes technical excellence and empowers the team to achieve their full potential.

Roles & Responsibilities :

- Design, build, and maintain scalable, reliable, and performant data pipelines and architectures.

- Implement best practices for data management, including data integrity, security, and governance.

- Oversee architectural decisions for data-related projects, ensuring they align with the company's goals and scalability requirements.

- Drive technical direction and be hands-on when necessary to support complex problem-solving efforts.

- Lead, mentor, and develop a team of data engineers, fostering a collaborative and high-performance culture.

- Conduct regular one-on-ones, set clear performance goals, provide constructive feedback, and support career development paths.

- Partner with HR and senior management in hiring, onboarding, and retaining top talent for the data engineering team.

- Develop and maintain frameworks for data quality, monitoring, and observability to ensure data reliability.

- Proactively identify and resolve data-related issues and bottlenecks to maintain a smooth data flow.

- Collaborate with stakeholders to understand business needs and translate them into technical requirements.

- Analyze and optimize performance of data pipelines and queries to improve efficiency and speed.

- Implement data partitioning, indexing, and caching strategies where applicable.

- Contribute to the evolution of our data strategy and technology stack.

- Monitor and troubleshoot data pipelines, ensuring timely resolution of issues in production environments.

- Actively participate in code reviews and ensure best practices are maintained.

- Takes ownership of projects, ensuring timely delivery and adherence to quality standards.

- Demonstrates a proactive approach to identifying challenges and delivering solutions that enhance system robustness and operational efficiency.

- Ability to see the big picture, understand business needs, and align data solutions with business goals.

Technical Skills :

- Extensive experience with Azure cloud services (e.g , Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL) or AWS cloud services (e.g , AWS Glue, Redshift, S3, Lambda, EMR).

- Strong experience in ETL processes, data modeling, and data warehousing.

- Expert level knowledge of Azure Data Factory / AWS Glue.

- Expert level knowledge of SQL.

- Expert level knowledge on Cloud based DW solutions like Snowflake or Azure Synapse Analytics or AWS Redshift.

- Strong knowledge of programming languages such as Python or C#.

- Should be able to analyze and understand data.

- Experience with version control systems like Git.

- Experience with other Azure services such as- Azure Function App, Azure Data Lake, Azure Synapse Azure DevOps or Amazon Athena, Lambda, Kinesis, or CloudWatch.

Soft Skills :

- Proven ability to lead data engineering teams and projects.

- Excellent problem-solving and analytical skills.

- Strong communication and interpersonal skills with the ability to foster collaboration and teamwork.

- Ability to inspire and lead a team in a fast-paced environment.

- Commitment to fostering a diverse and inclusive work culture.

Qualifications & Experience :

Must have :

- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

- Overall 9 years to14 years of experience with 5+ years in data engineering or a related field, with demonstrated leadership and people management experience.

- Strong proficiency in Python, SQL, and cloud platforms (preferably Azure/AWS).

- Expertise in designing and maintaining data warehouses and lakes (e.g Snowflake, Azure Synapse Analytics).

- Strong proficiency in SQL and programming languages such as Python, Scala, or Java.

- Experience with ETL tools and frameworks (e.g, Apache Airflow, dbt).

- Proficiency in cloud-based data solutions (e.g , AWS, Azure, GCP).

- Familiarity with data observability tools (e.g , Monte Carlo, Databand).

- Experience with data warehousing solutions like Snowflake, Redshift, or BigQuery.

- Strong understanding of data governance, data quality frameworks, and data security.

Nice to Have :

- Certifications like Azure Data Engineer Associate, Azure Administrator Associate, Azure Solutions Architect ,Snow Pro Core or AWS Certified Solutions Architect.

- Familiarity with CI/CD practices.

- Familiarity with Power BI or similar visualization tools.

- Familiarity with Microsoft Fabric.