Senior Data Engineer

Key Responsibilities

• Develop and maintain scalable pipelines for data and ML models, containerization, validation and approval.
• Develop and maintain model training datasets.
• Collaborate with Data Scientists to design, validate, deploy, monitor and maintain production scale ML models.
• Build and maintain solutions on Databricks, AWS, EKS and Kubeflow.
• Scale up existing data and ML pipelines to work with larger datasets and optimize resource usage.
• Build and maintain infrastructure as code (IaC) in the cloud, that is resilient and can scale when needed.
• Work with stakeholders across the organization to identify opportunities utilizing big data from many different sources

About you

• Minimum of 4 years of years of experience as a software engineer with a Bachelors/ Minimum of 3 years of experience as a software engineer with a masters degree, with expertise in building ML and data pipelines (document pipelines or RAG development plus)
• Bachelors degree in IT, Software Engineering, or Computer Science is preferred.
• Proficient in Python, Linux shell scripting, and tools like PySpark, Pandas, and NumPy.
• Hands-on experience with Databricks, big data management, and core AWS services (EC2, S3, CloudWatch, Lambda, etc.).
• Good to have knowledge of containerization (Kubernetes/EKS) and infrastructure-as-code tools (Terraform or CloudFormation).
• Good to have prior experience in building Api's
• Skilled in debugging computational bottlenecks, troubleshooting, software testing, and optimizing data pipelines.
• Self-motivated, strong problem-solving skills, and comfortable collaborating remotely with international teams.

Additional Attributes

• Excellent problem-solving and analytical abilities.
• Strong communication and interpersonal skills.
• Self-driven, proactive, and adaptable to evolving project requirements.
• Team-oriented mindset and ability to thrive in a collaborative environment.