Data Engineer - BigQuery/Big Data (5-6 yrs)

Job Description :

Key Responsibilities :

1. Lead the migration of data from Oracle to BigQuery, ensuring a seamless transition and modernization of our data warehouse.

2. Develop and maintain data pipelines using BigQuery, DataProc, and PySpark.

3. Utilize Google Cloud Storage (GCS) for efficient data storage and retrieval.

4. Work within the Hadoop ecosystem to process large datasets and derive insights.

5. Write and optimize complex SQL queries, with a strong emphasis on performance.

6. Collaborate with cross-functional teams to identify data needs and deliver actionable insights.

7. Document processes, best practices, and design patterns.

8. Implement and manage workflows with Apache Airflow.

Qualifications :

1. Proven experience in data engineering, specifically with Oracle databases and PL/SQL.

2. Proficiency in Google Cloud Platform (GCP) tools, particularly BigQuery, DataProc, and GCS.

3. Strong expertise in PySpark and the Hadoop ecosystem.

4. Excellent SQL skills, with the ability to write complex queries and optimize performance.

5. Familiarity with data migration strategies and data warehousing concepts.

6. Strong problem-solving skills and attention to detail.

7. Ability to work collaboratively in a fast-paced environment.

Preferred Qualifications :

1. Experience with data visualization tools.

2. Knowledge of ETL processes and data modeling.