Data Engineering Lead

Responsibilities:

• Lead and manage an offshore team of data engineers, providing strategic guidance, mentorship, and support to ensure the successful delivery of projects and the development of team members.
• Collaborate closely with onshore stakeholders to understand project requirements, allocate resources efficiently, and ensure alignment with client expectations and project timelines.
• Drive the technical design, implementation, and optimization of data pipelines, ETL processes, and data warehouses, ensuring scalability, performance, and reliability.
• Define and enforce engineering best practices, coding standards, and data quality standards to maintain high-quality deliverables and mitigate project risks.
• Stay abreast of emerging technologies and industry trends in data engineering, and provide recommendations for tooling, process improvements, and skill development.
• Assume a data architect role as needed, leading the design and implementation of data architecture solutions, data modeling, and optimization strategies.
• Demonstrate proficiency in AWS services such as:
• Expertise in cloud data services, including AWS services like Amazon Redshift, Amazon EMR, and AWS Glue, to design and implement scalable data solutions.
• Experience with cloud infrastructure services such as AWS EC2, AWS S3, to optimize data processing and storage.
• Knowledge of cloud security best practices, IAM roles, and encryption mechanisms to ensure data privacy and compliance.
• Proficiency in managing or implementing cloud data warehouse solutions, including data modeling, schema design, performance tuning, and optimization techniques.
• Demonstrate proficiency in modern data platforms such as Snowflake and Databricks, including:
• Deep understanding of Snowflake's architecture, capabilities, and best practices for designing and implementing data warehouse solutions.
• Hands-on experience with Databricks for data engineering, data processing, and machine learning tasks, leveraging Spark clusters for scalable data processing.
• Ability to optimize Snowflake and Databricks configurations for performance, scalability, and cost-effectiveness.
• Manage the offshore team's performance, including resource allocation, performance evaluations, and professional development, to maximize team productivity and morale.

Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field; advanced degree preferred.
• 10+ years of experience in data engineering, with a proven track record of leadership and technical expertise in managing complex data projects.
• Proficiency in programming languages such as Python, Java, or Scala, as well as expertise in SQL and relational databases (e.g., PostgreSQL, MySQL).
• Strong understanding of distributed computing, cloud technologies (e.g., AWS), and big data frameworks (e.g., Hadoop, Spark).
• Experience with data architecture design, data modeling, and optimization techniques.
• Excellent communication, collaboration, and leadership skills, with the ability to effectively manage remote teams and engage with onshore stakeholders.
• Proven ability to adapt to evolving project requirements and effectively prioritize tasks in a fast-paced environment.