Data Engineer

• Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions.
• Take ownership of data pipeline projects from inception to deployment, managing scope, timelines, and risks.
• Ensure data quality and integrity through rigorous testing and monitoring.
• Leverage cloud platforms (AWS, Databricks preferred) to build scalable and efficient data solutions.
• Work closely with data analysts, data scientists, and business collaborators to understand data requirements.
• Identify and resolve complex data-related challenges.
• Adhere to data engineering practices and standards.
• Experience developing in an Agile development environment, and comfortable with Agile terminology and ceremonies.
• Familiarity with code versioning using GIT, Jenkins and code migration tools.
• Exposure to Jira or Rally.
• Identifying and implementing opportunities for automation and CI/CD.
• Help define guidelines, standards, strategies, security policies and change management policies to support the Data Lake
• Automate and Optimize data pipeline and framework for easier and cost-effective development process.
• Stay up to date with the latest data technologies and trends.

• Masters degree in computer science or engineering field and 4 to 6 years of relevant experience OR
• Bachelors degree in computer science or engineering field and 6 to 8 years of relevant experience OR
• Diploma in computer science and Minimum of 10+ years of relevant work experience

Must-Have Skills:

• Demonstrated hands-on experience with cloud platforms (AWS, Azure, GCP)
• Proficiency in Python, PySpark, SQL. Hands on experience with ETL performance tuning.
• Development knowledge in Databricks.
• Good analytical and problem-solving skills to address sophisticated data challenges.
• Experience building Bigdata pipelines; Hands-on experience with SQL
• Experienced with data modeling and performance tuning for both OLAP and OLTP databases
• Experience working with Apache Spark or Pyspark, Apache Airflow, and Databricks
• Experienced with software engineering best-practices, including but not limited to version control (GitLab, Subversion, etc.), CI/CD (Jenkins, GITLab etc.), automated unit testing, and Dev Ops

• Experience in SQL/NOSQL database, vector database for large language models
• Experience with AWS services: EKS, EC2, S3, EMR, RDS, Redshift/Spectrum, Lambda, Glue, Athena
• Understanding of relational/ dimensional data modeling techniques
• Experience with timely engineering, model fine tuning
• Ability to work in a startup-like environment with little supervision
• Ability to effectively present information for all levels, and respond to questions from managers, clients, and customers
• AWS Certified Data Engineer
• Databricks Certification
• SAFe Agile certification

• Excellent analytical and solve skills
• Excellent written and verbal communications skills (English) in translating technology content into business-language at various levels
• Ability to work effectively with global, virtual teams
• High degree of initiative and self-motivation
• Ability to manage multiple priorities successfully
• Team-oriented, with a focus on achieving team goals
• Strong problem-solving and analytical skills.
• Strong time and task leadership skills to estimate and successfully meet project timeline with ability to bring consistency and quality assurance across various projects.