Brillio - Senior/Lead Data Engineer - AWS Cloud (7-9 yrs)

Skills - AWS ecosystem, especially AWS ECS, AWS EMR, AWS SageMaker, CodeBuild, and CodePipeline and Expertise in CI/CD processes , ready to work on and learn new skills like Dagster and/or Kedro , Spark/Scala.

Required Skills & Experience :

- Experience in developing and managing ETL pipelines.

- Demonstrated experience with AWS ecosystem, especially AWS ECS, AWS EMR, AWS SageMaker, CodeBuild, and CodePipeline.

- Strong understanding and practical experience with GitHub for version control.

- Extensive experience with Docker for containerization and managing container-based applications.

- Expertise in CI/CD processes, including automated testing and deployment pipelines.

- Proven track record in unit testing and integration testing for maintaining code and infrastructure quality.

Preferred Skills :

- Hands-on experience with developing interactive notebook-based workflows.

- Familiarity with data engineering and MLOps best practices to drive continuous improvement in development processes.

- Strong collaboration skills, with the ability to work closely with data scientists, engineers, and other stakeholders.