At Think Right Technologies we are seeking a highly skilled and motivated Lead Data Engineer to spearhead our data initiatives, design scalable architectures, and manage data pipelines using Azure or AWS cloud platforms. The ideal candidate will have a strong technical background, proven leadership skills, and the ability to collaborate across teams to deliver impactful data solutions.
Key Responsibilities
Leadership and Strategy:
Lead and mentor a team of data engineers in building and maintaining robust data infrastructure.
Define and implement the data engineering roadmap aligned with organizational goals.
Advocate for best practices in data management, security, and governance.
Data Architecture and Pipeline Development:
Design and implement scalable, efficient, and reliable data pipelines and ETL/ELT processes.
Optimize existing data architectures for performance and scalability in Azure or AWS environments.
Manage large-scale data lakes, warehouses, and real-time streaming data.
Collaboration:
Partner with data scientists, analysts, and other stakeholders to understand data requirements.
Translate business needs into technical solutions that align with industry standards.
Cloud Expertise:
Utilize Azure or AWS cloud services, including but not limited to Databricks, Synapse, Redshift, Glue, and S3.
Monitor and manage cloud resources for cost and efficiency optimization.
Automation and Monitoring:
Develop automation scripts for deploying and maintaining data workflows.
Implement monitoring tools to ensure data pipeline reliability and system health.
Required Qualifications:
Education: Bachelor s or Master s degree in Computer Science, Engineering, or a related field.
Experience: 9 - 12 years of experience in data engineering, with 2 - 4 years leading technical teams.
Required Technical Skills:
Proficiency in programming languages such as Python, Scala, or Java.
Expertise in SQL and database systems (e.g., PostgreSQL, Snowflake, SQL Server).
Hands-on experience with Azure (e.g., Data Factory, Synapse, Data Lake) or AWS (e.g., Glue, Redshift, S3).
Strong understanding of data modelling, warehousing, and big data technologies (e.g., Spark, Kafka).
Experience with CI/CD pipelines, Terraform, and version control systems like Git.
Required Leadership Qualities:
Leadership : Ability to inspire and guide a team towards achieving goals.
Ownership : Demonstrates a strong sense of responsibility for their tasks and outcomes. Takes initiative to identify and solve problems without waiting for direction.
Problem-Solving : Creative and analytical thinking to overcome challenges.
Decision-Making : Strong ability to make informed and timely decisions.
Communication : Ability to convey ideas and instructions clearly and concisely. Capable of persuading and motivating others to achieve common goals. Provides constructive feedback in a way that encourages growth and improvement.
Preferred Qualifications
Certifications in Azure or AWS (e.g., Azure Data Engineer Associate, AWS Certified Data Analytics).
Experience with machine learning workflows and AI/ML tools.
Familiarity with orchestration tools like Apache Airflow or Azure Data Factory pipelines.