Data Engineer - Python/SQL (5-10 yrs)

Job Description :

We are looking for a skilled Data Engineer with expertise in ETL, Python, PySpark, SQL, and Google Cloud Platform (GCP) - BigQuery to join our team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and ensuring data integrity, performance, and reliability.

Key Responsibilities :

- Design, develop, and maintain ETL pipelines to extract, transform, and load data efficiently.

- Work with BigQuery to design scalable and optimized data solutions.

- Develop and optimize PySpark scripts for large-scale data processing.

- Write complex SQL queries for data extraction, transformation, and analysis.

- Collaborate with data scientists, analysts, and business teams to understand data needs.

- Implement best practices for data governance, security, and compliance.

- Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability.

- Automate data workflows and optimize query performance for cost efficiency.

Required Skills & Qualifications :

- 5 to 10 years of experience in ETL development and data engineering.

- Strong proficiency in Python and PySpark for data processing.

- Expertise in SQL with hands-on experience in BigQuery (GCP).

- Experience with data warehousing, data modeling, and performance tuning.

- Hands-on experience with Google Cloud Platform (GCP) services.

- Knowledge of CI/CD pipelines, Airflow, or other workflow orchestration tools is a plus.

- Experience working with structured and unstructured data.

- Strong analytical and problem-solving skills.

- Ability to work independently and collaborate effectively with cross-functional teams.