Quality Automation Engineer (SDET)

What you will do:

• Design and implement automated test frameworks for ETL jobs running on Google Cloud Dataflow and Dataproc

• Develop test suites for data pipelines written in Java, Scala, Spark, and SQL

• Create test automation for Cloud Composer (Apache Airflow) workflows

• Write shell scripts for test orchestration and environment setup

• Validate data transformations, reconciliations, and business logic across pipeline stages

• Implement mock services and test data generation for pipeline testing

• Build performance testing frameworks to evaluate ETL job efficiency

• Maintain CI/CD pipelines for test automation

What experience you need:

• 3+ years experience in SDET/Automation Testing roles

• Strong programming skills in Java or Scala

• Expertise in testing ETL processes and data pipelines

• Experience with Apache Spark and SQL

• Proficiency in GCP services: Dataflow, Dataproc, Cloud Storage, BigQuery

• Knowledge of Apache Airflow/Cloud Composer

• Strong shell scripting skills (Bash/Shell)

• Working knowledge of Apache Hive and HiveQL

• Experience with test automation frameworks and tools

• CI/CD implementation experience

What could set you apart:

• Experience with data quality testing and validation

• Knowledge of distributed computing concepts

• Python programming skills

• Experience with data privacy and security compliance

• Prior experience in financial services industry

• Cloud certification (GCP Professional Data Engineer preferred)

• Google Cloud Platform

• Languages: Java, Scala, Python, SQL

• Apache Spark, Apache Beam, Apache Hive

• Shell scripting (Bash/Shell)

• Cloud Composer/Airflow

• Version Control: Git

• CI/CD Tools