Receive all updates via Facebook. Just Click the Like Button Below... ML - AWS Glue
2024-09-06
Responsibilities
Design and implement scalable and efficient data pipelines and servers for processing, cleaning, and transforming large volumes of structured and unstructured data.
Ensure data quality and integrity throughout the entire data lifecycle.
Optimize and tune existing data infrastructure for performance, scalability, and reliability.
Troubleshoot and resolve data-related issues in a timely manner.
Collaborate with data scientists, engineers, and business stakeholders to understand requirements and provide technical solutions.
Create and maintain comprehensive documentation for data pipelines, ML models, and related processes.
Stay up to date on emerging technologies and trends in data engineer, machine learning, and natural language processing.
Skill Required
Qualifications
Minimum of 8-10 years of experience in data engineering or similar role.
Strong proficiency in Python and SQL for data manipulation and analysis i.e. numpy, pandas, scikit-learn
Solid foundation of data engineering frameworks used for data ingestion, transformation, and consolidation.
Experience with cloud-based platforms (AWS) and associated services (S3, EC2, Glue )
Experience with version control systems (Git) and CI/CD pipelines a plus.
Familiarity with agile processes and tools such as Jira for project management.
Excellent problem-solving skills and attention to detail.
Effective communication and collaboration skills.
Education Required
Bachelors or Masters degree in Computer Science, Engineering, or a related field.