Data Engineer - AWS Services (8-12 yrs)

Role : AWS Data Engineer.

Good To Have Skills : AWS Certifications.

Must Have Skills : Client Management, Pyspark, Glue, EMR Cluster, Lamda, DMS, ETL, AWS.

Experience Level : 8-12 Years.

Employee Type : Salaried.

The Data platform associate will work in a variety of settings to build the data systems including design, implementation, ongoing maintenance, and management of the businesses data assets to assist the business in achieving its goals regarding reporting and mining. These data systems are responsible to collect, convert raw data into usable information for data scientists, business analyst and other stakeholders in organization responsible to manage end to end data.

Additional responsibilities include collaborating with Architect and DBA in architecture, development, and maintenance of all system databases.

Their goal is to make data accessible easy so that organizations can use it to evaluate and optimize in decision-making.

Responsibilities :

- Assist in the design and implementation of Snowflake-based analytics solution (data lake and data warehouse) on AWS.

- Requirements definition, source data analysis and profiling, the logical and physical design of the data lake and data warehouse as well as the design of data integration and publication pipelines.

- Develop Snowflake deployment and usage best practices.

- Build and maintain data pipelines adhering to suggested enterprise architecture principles and guidelines.

- Design, build, test, and maintain data management systems.

- Work in sync with internal and external team members like data architects, data scientists, data analysts to handle all sorts of technical issue.

- Working in Agile/Lean model.

- Deliver quality deliverables on time.

- Troubleshoot and resolve pipeline and infrastructure issues promptly.

- Able to understand and translate complex functional requirements into technical solutions with the help of product managers/business analysts.

External Skills And Expertise :

Required Skills :

- 6-9 years of experience in Data Engineering.

- Creation and maintenance of optimum data pipeline architecture for ingestion, processing of data.

- Creation of necessary infrastructure for ETL jobs from a wide range of data sources using, S3, Snowflake and AWS data services.

- Experience in Data storage technologies like Amazon S3, SQL, NoSQL.

- Hands-on experience in writing complex SQL.

- Hands on experience in developing pipelines using AWS data services like Glue, Lambda.

- Develop, manage, and optimize data pipelines using AWS Glue and PySpark.

- Hands on experience in Optimizing AWS services for cost-effectiveness and performance.

- Data modeling technical awareness.

- Experience in working with stakeholders working in different time zones.

Good to have :

- Working knowledge on using Bigdata technologies.

- Experience in collaborating with data quality and data governance team.

- Payments domain knowledge.

- CRM, Accounting, etc. in depth understanding.

- Regulatory reporting exposure.

Other skills :

- Good Communication skills.

- Team Player.

- Problem solver.

- Willing to learn new technologies, share your ideas and assist other team members as needed.

- Strong analytical and problem-solving skills; ability to define problems, collect data, establish facts, and draw conclusions.