The AWS Data Engineer will design and implement robust data solutions using AWS services, ensuring seamless integration, storage, processing, and analysis of data. The role involves collaborating with cross-functional teams to support business objectives and optimize data workflows.
Design, build, and manage scalable and reliable data pipelines on AWS.
Extract, transform, and load (ETL) data from various sources into AWS storage solutions like S3, Redshift, and RDS.
Implement data lake and warehouse solutions for structured and unstructured data.
Write and optimize SQL queries and scripts for data transformation and aggregation.
Monitor data workflows for performance, availability, and accuracy.
Optimize data pipelines for cost-efficiency and scalability.
Work closely with data scientists, analysts, and other engineers to understand data needs and ensure end-to-end solutions.
Assist in the integration of third-party tools and APIs.
Ensure data security by implementing IAM policies, encryption, and other AWS security best practices.
Automate repetitive tasks to improve efficiency and reduce errors.
Strong programming skills in Python.
Familiarity with workflow orchestration tools like Apache Airflow or AWS Step Functions.
Strong problem-solving and analytical skills.
Effective communication to collaborate with diverse teams.
Attention to detail and a commitment to delivering high-quality solutions.