AWS Data Engineer (3-5 yrs)

Overview :

The AWS Data Engineer plays a vital role in our organization by designing and implementing data-driven solutions using Amazon Web Services. This position is crucial for ensuring that data pipelines are efficient, scalable, and aligned with business goals. The Data Engineer will work closely with data scientists and business analysts to optimize data collection, storage, and processing.

The successful candidate will leverage AWS tools like Glue, Redshift, and Apache Airflow to create robust data workflows. As we continue to grow our data capabilities, this role will be instrumental in enhancing our data processing frameworks and data quality while fostering data-driven decision-making across various departments.

Key Responsibilities :

- Design, build, and maintain scalable data pipelines using AWS Glue and Redshift.

- Implement ETL processes to transform and prepare data for analysis.

- Develop and optimize SQL queries for efficient data retrieval and manipulation.

- Collaborate with data scientists and analysts to understand data requirements and deliver solutions.

- Utilize Apache Airflow for workflow management and to schedule data jobs.

- Ensure data quality and integrity across all data pipelines.

- Monitor performance and troubleshoot issues in data workflows.

- Document data models, workflows, and architecture for knowledge sharing.

- Stay updated with industry trends and AWS services to enhance data architecture.

- Work on data security protocols and ensure compliance with regulations.

- Participate in code reviews and contribute to best practices in data engineering.

- Assist in capacity planning and scaling of the data infrastructure.

- Create dashboards and reports to visualize data trends and insights.

- Train and mentor junior team members on data engineering best practices.

- Support data governance initiatives and contribute to a data-driven culture.

Required Qualifications :

- Bachelor's degree in Computer Science, Information Technology, or a related field.

- 3-5 years of experience in data engineering role with a focus on AWS technologies.

- Strong proficiency in SQL and data modeling techniques.

- Hands-on experience with AWS Glue and Redshift.

- Familiarity with Apache Airflow for orchestration of data workflows.

- Knowledge of data warehousing concepts and methodologies.

- Experience with ETL tools and data integration processes.

- Proficiency in programming languages such as Python or Java.

- Ability to work with large data sets and big data technologies.

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration abilities.

- Experience with data visualization tools is a plus.

- Knowledge of data governance frameworks and best practices.

- Hands-on experience with source control tools such as Git.

- Familiarity with Agile methodologies.

- AWS certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect) are preferred.