Data Platform Engineer

• Work as a member of a Data Platform Engineering team that uses Cloud and Big Data technologies to design, develop, implement and maintain solutions to support various functional areas like Manufacturing, Commercial, Research and Development.
• Collaborate with lead architect, Business SMEs, and Data Scientists to architect data solutions
• Serve as lead engineer for technical implementation of projects including planning, architecture, design, development, testing, and deployment in agile setting
• Overall management of the Enterprise Data Lake on AWS environment to ensure that the service delivery is cost effective and business SLAs around uptime, performance and capacity are met
• Help define guidelines, standards, strategies, security policies and change management policies to support the Data Lake
• Proactively work on challenging data integration problems by implementing optimal ETL patterns, frameworks for structured and unstructured data.
• Automate and Optimize data pipeline and framework for easier and cost-effective development process.
• Advice and support project teams (project managers, architects, business analysts, and developers) on cloud platforms (AWS, Databricks preferred), tools, technology, and methodology related to the design, build scalable, efficient and maintain Data Lake and other Big Data solutions
• Experience developing in an Agile development environment, and comfortable with Agile terminology and ceremonies.
• Familiarity with code versioning using GITLAB, and code migration tools.
• Provide mentorship and technical guidance to junior engineers and team members.
• Stay up to date with the latest data technologies and trends

• Masters degree in computer science or engineering field and 4 to 6 years of relevant experience OR
• Bachelors degree in computer science or engineering field and 6 to 8 years of relevant experience OR
• Diploma in computer science and Minimum of 10+ years of relevant work experience

Must-Have Skills:

• Demonstrated hands-on experience with cloud platforms (AWS, Azure, GCP)
• Proficiency in Python, PySpark, SQL. Hands on experience with ETL performance tuning.
• Development knowledge in Databricks.
• Good analytical and problem-solving skills to address sophisticated data challenges.
• Programming experience preferably in Python, Scala , Java.
• Experience building Bigdata pipelines; Hands-on experience with SQL
• Experience in API development and deployment.
• Experience working with Apache Spark, Pyspark, Apache Airflow, Databricks
• Experience with Software engineering best-practices, including but not limited to version control, CI/CD, automated testing
• Experience with Docker and AWS services: EKS, EC2, S3, EMR, RDS, Redshift/Spectrum, Lambda, Glue
• Exposure to Jira or Jira Align.

• Experience in SQL/NOSQL database, vector database for large language models
• Experience with AWS services: EKS, EC2, S3, EMR, RDS, Redshift/Spectrum, Lambda, Glue, Athena
• Understanding of relational/ dimensional data modeling techniques
• Experience with timely engineering, model fine tuning
• Ability to work in a startup-like environment with little supervision
• Ability to effectively present information for all levels, and respond to questions from managers, clients, and customers
• AWS Certified Data Engineer
• Databricks Certification
• SAFe Agile certification

• Excellent analytical and solve skills
• Excellent written and verbal communications skills (English) in translating technology content into business-language at various levels
• Ability to work effectively with global, virtual teams
• High degree of initiative and self-motivation
• Ability to manage multiple priorities successfully
• Team-oriented, with a focus on achieving team goals
• Strong problem-solving and analytical skills.
• Strong time and task leadership skills to estimate and successfully meet project timeline with ability to bring consistency and quality assurance across various projects.