1 Miles Ahead Consultants Job
Senior Engineer - Atacamma DQ (5-14 yrs)
Miles Ahead Consultants
posted 1mon ago
Key skills for the job
Responsibilities :
- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements
- Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets
- Collaborate with other teams to design, develop and deploy data tools that support both operations and - product use cases
- Work with the team in solving problems in big data technologies and prototype solutions to improve our data processing architecture
Must Have :
- Ensuring proper execution of tasks and alignment with business vision and objectives
- Oversee activities of the junior data engineering teams
- Works closely with the business owner / Product owner / Cross functional Team and gathering technical requirements
- Experience in building and maintaining reliable and scalable ETL pipeline on big data/Cloud platform through the collection, storage, processing, and transformation of large data-sets
- Experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark
- Proficiency in scripting languages in PySpark, python
- Experience in AWS cloud & DevOps
- Experience in Databricks
- Experience in Database design/data modelling
- Must have strong experience in data warehouse concepts, understanding Data Lake, Lake House
- Concepts, any new Big Data eco system.
- Experience in testing and validation in order to support the accuracy of data transformations and data verification
- Should be able to independently drive the requirements, have solutioning mindset facing business stakeholders
Soft Skills :
- Excellent Verbal and Communication skills needed.
- Should be a excellent team player.
- Good knowledge on Agile principles and experience working in scrum teams using Jira
- Should be comfortable to mentor junior engineers in the team (For Senior Data Engineers)
- Should be able to operate in an ambiguous environment with minimal guidance.
Functional Areas: Other
Read full job description