Data Engineer - Google Cloud Platform (3-5 yrs)

Job Title : Data Engineer

Experience: 3-5 Years

Job Location : Hyderabad

Responsibilities :

- Design, develop, and maintain robust and scalable data pipelines using GCP services (e., Dataflow, Dataproc, Composer).

- Implement data ingestion processes from various sources, including relational databases, NoSQL databases, APIs, and streaming platforms.

- Perform data cleaning, transformation, and enrichment.

- Develop and optimize SQL queries for data analysis and reporting in BigQuery.

- Design and implement efficient data models in BigQuery.

- Work with NoSQL databases (Cassandra, MongoDB, Elasticsearch) for data storage and retrieval.

- Design and implement data models for NoSQL databases.

- Utilize programming languages like Python, Java, or C to automate data processing tasks, build data pipelines, and develop data applications.

- Implement data quality checks and monitoring to ensure data accuracy and consistency.

- Collaborate with data scientists, analysts, and other engineers to understand data requirements and deliver data solutions.

- Work with GCP services related to data storage, processing, and management.

- Monitor and optimize data pipeline performance and query efficiency.

Must-Have Skills :

- Strong hands-on experience writing complex SQL queries for data manipulation, analysis, and transformation.

- Hands-on experience working with BigQuery, including data loading, querying, and performance tuning.

- Experience with at least one NoSQL database (Cassandra, MongoDB, Elasticsearch, or similar).

- Proficiency in at least one programming language (Python, Java, C, or similar).

Nice-to-Have Skills :

- Experience with Apache Airflow for workflow orchestration and scheduling.

- Experience with ETL tools (Talend, Informatica, SSIS, Datastage, or similar).

Qualifications :

- Bachelor's degree in Computer Science, Engineering, or a related field.

- 3-5 years of experience in data engineering or a related role