AWS Data Engineer - PySpark/ETL (5-8 yrs)

We are an innovative and growing organization seeking an experienced AWS Data Engineer to join our dynamic data team. This role is perfect for someone with 5+ years of experience in cloud-based data engineering, particularly in AWS services like Glue, PySpark, SQL, and Streaming Data. If you're passionate about building efficient and scalable data pipelines and want to play a key role in a cutting-edge environment, we'd love to have you on board.

Key Responsibilities :

• Design, develop, and maintain data pipelines using AWS Glue, PySpark, and other AWS services (e.g., S3, Redshift, Lambda, Kinesis).
• Implement ETL processes for extracting, transforming, and loading data into data lakes and warehouses.
• Work with streaming data platforms (e.g., AWS Kinesis) to build real-time data processing systems.
• Optimize data workflows for performance, scalability, and cost-effectiveness in AWS cloud.
• Write efficient, high-quality SQL queries to interact with relational and non-relational databases.
• Ensure data quality, integrity, and reliability across various systems and data pipelines.
• Collaborate with data scientists, analysts, and business teams to define data requirements and deliver insights.
• Monitor the health and performance of data pipelines, and troubleshoot or resolve any issues.
• Document data pipelines, architectures, and processes to support knowledge sharing and long-term maintainability.
• Implement security best practices to ensure the privacy and protection of sensitive data in AWS.

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong hands-on experience with AWS Glue, PySpark, and SQL for data engineering tasks.
• Proven experience in working with streaming data and real-time data processing technologies (e.g., AWS Kinesis).
• Expertise in designing, building, and optimizing ETL processes in AWS cloud environments.
• Experience with AWS S3, AWS Redshift, and other AWS data services.
• Solid knowledge of data modeling, data architecture, and data warehousing.
• Strong SQL skills, with the ability to write complex queries and interact with both structured and unstructured data.
• Ability to troubleshoot and resolve issues in data pipelines, ensuring reliability and scalability.
• Knowledge of data security practices and working with sensitive data in cloud environments.
• Strong collaboration skills and the ability to work in a team-oriented, fast-paced environment.
• Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent work experience).

• Experience with containerization technologies like Docker and Kubernetes.
• Experience with Apache Kafka or other similar stream processing technologies.
• AWS Certifications (e.g., AWS Certified Data Analytics - Specialty, AWS Certified Solutions Architect).
• Experience working with Agile methodologies and using tools like JIRA.
• Work Environment:
• Hybrid work schedule: 3 days a week in the office, 2 days remote.
• Collaborative, fast-paced environment with opportunities for career growth and skill development.
• Competitive compensation package with benefits.