Data Engineer - Python/Spark (5-7 yrs)

Job Description :

Mandatory Skills :

- Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.

- Hands-on on technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, Hadoop Platform, Hive, airflow

- Experience in GCP Cloud Composer, Big Query, DataProc

- Offer system support as part of a support rotation with other team members.

- Operationalize open source data-analytic tools for enterprise use.

- Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.

- Understand and follow the company development lifecycle to develop, deploy and deliver the solutions.

Minimum Qualifications :

- Bachelor's degree in Computer Science, CIS, or related field

- 5-7 years of IT experience in software engineering or related field

- Experience on project(s) involving the implementation of software development life cycles (SDLC)