Atacama DQ Tech Lead - Python/Big Data (9-18 yrs)

Responsibilities :

- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements

- Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets

- Collaborate with other teams to design, develop and deploy data tools that support both operations and product use cases

- Work with the team in solving problems in big data technologies and prototype solutions to improve our data processing architecture

Must Have :

- Ensuring proper execution of tasks and alignment with business vision and objectives

- Oversee activities of the junior data engineering teams

- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements

- Experience in building and maintaining reliable and scalable ETL pipeline on big data/Cloud platform through the collection, storage, processing, and transformation of large data-sets

- Experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark

- Proficiency in scripting languages in PySpark, python

- Experience in AWS cloud & DevOps

- Experience in Databricks

- Experience in Database design/data modelling

- Must have strong experience in data warehouse concepts, understanding Data Lake, Lake House

- Concepts, any new Big Data eco system.

- Experience in testing and validation in order to support the accuracy of data transformations and data verification

- Should be able to independently drive the requirements, have solutioning mindset facing business stakeholders

Soft Skills :

- Excellent Verbal and Communication skills needed.

- Should be a excellent team player.

- Good knowledge on Agile principles and experience working in scrum teams using Jira

- Should be comfortable to mentor junior engineers in the team (For Senior Data Engineers)

- Should be able to operate in an ambiguous environment with minimal guidance.