Data Engineer

Core Skills:

Exp: 5-10 yearsMust have - Python, Java, Bigquery, PySpark

Extensive experience with Google Cloud Platform (GCP) data services such as BigQuery, Cloud Storage, and Dataflow.

Expertise in ETL (Extract, Transform, Load) processes and data integration on GCP.

Strong SQL and database querying optimization skills on GCP.

Experience with data warehousing and data architecture on GCP.

Responsibilities:

Design, code, and develop new features/fix bugs/add enhancements.

Analyze and improve the efficiency, scalability and stability of various system resources.

Lead and Mentor junior engineers and drive a culture of technical perfection.

Drive creative and innovative solutions to complex problems, exemplifying good technical discernment.

Drive improvements and new approaches to address potential systemic pain points and technical debt, anticipate and avoid problems.

Take a hands-on approach in developing prototypes, independently and with others, to establish design decisions and/or technical feasibility.

Evaluate, Install, Setup, Maintain and Upgrade, Data Engineering, Machine Learning and CI/CD infrastructure tools hosted on Cloud (GCP/AWS)

Drive the CI/CD infrastructure tooling-related work in collaboration with various internal teams to get the user stories, epics, and goals to closure.

Propose, participate, and implement architecture-level enhancements/changes strategically through Dev/Stage/Prod environments.

Design, evangelize, deliver comprehensive best practices & efficient usage of available tooling resources/capabilities to run high-performance systems.

Provide innovative & and strategic solutions, along with cost & risk analysis, to improve the stability, scalability, and performance of the tools" infrastructure.

Perform troubleshooting, analysis, and resolution of environmental issues.

Innovate and automate processes/tasks to improve operational efficiency

Document and maintain application setup, runbooks, administration, and troubleshooting guides.

Programming Skills o Algorithms o Design Skills o Refactoring Skills o Debugging Skills o Unit testing

Skills: Database o SQL Databases (Oracle/MySQL/MSSQL etc.,) o No SQL Databases o Data Lake, Snowflake