Joining time / Notice Period : Immediate to 30 days
Must have: Experience in Atacama is mandatory. Experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark . Proficiency in scripting languages in PySpark, python . Experience in AWS cloud & DevOps . Experience in Databricks, Database design/data modelling . Must have strong experience in data warehouse concepts, understanding Data Lake, Lake House . Mode of work: Hybrid (3 days WFO) .
Responsibilities:
Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements.
Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets
Collaborate with other teams to design, develop and deploy data tools that support both operations and product use cases. Work with the team in solving problems in big data technologies and prototype solutions to improve our data processing architecture
Qualification:
Ensuring proper execution of tasks and alignment with business vision and objectives
Oversee activities of the junior data engineering teams, Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements
Experience in building and maintaining reliable and scalable ETL pipeline on big data/Cloud platform through the collection, storage, processing, and transformation of large data-sets
Experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark. Proficiency in scripting languages in PySpark, python
Experience in AWS cloud & DevOps. Experience in Databricks
Experience in Database design/data modelling. Must have strong experience in data warehouse concepts, understanding Data Lake, Lake House Concepts, any new Big Data eco system.
Experience in testing and validation in order to support the accuracy of data transformations and data verification
Should be able to independently drive the requirements, have solutioning mindset facing business stakeholders