Data Engineer - Google Cloud Platform (4-8 yrs)

Job Description :

- Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity.

- Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.

- Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.

- Writes unit/integration tests, contributes to engineering wiki, and documents work.

- Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.

- Works closely with a team of frontend and backend engineers, product managers, and analysts.

- Design and build scalable data products for our business stakeholders.

- Agile development through story planning, estimation and retrospectives.

- Continual improvement mind-set i.e. innovation / process improvements / automation opportunities / security standards.

- Collaborate with architects, designers, analysts, product specialists, and other software engineers.

- Continual improvements using engineering excellence best practices.

- Demos and knowledge transfers to stakeholders around product developments.

- Improve automated testing and deployments using CI/CD pipelines.

Key skills & experience :

- Strong development skills in the following : Python, SQL.

- Designing & building data products from inception through to production.

- GCP including Big Query & Dataflow

- Can demonstrate software engineering best practices.

- Lead on software and data product designs with other engineers to come up with the best solutions.

- Hands on software development in a story driven agile environment.

- Be proactive about continuous improvement and innovation.

Desirable skills & experience :

- Defines company data assets (data models), spark, spark SQL, and hive SQL jobs to populate data models.

- Designs data integrations and data quality framework.

Preferred candidate profile :

- BS or MS degree in Computer Science or a related technical field

- 4+ years of Python or Java development experience

- 4+ years of SQL experience (No-SQL experience is a plus)

- 4+ years of experience with schema design and dimensional data modelling

- 4+ years of experience with Big Data Technologies like Spark, Hive

- 2+ years of experience on data engineering on GCP services like big query.