Senior Data Engineer

Job Responsibilities:

The data engineers are responsible for the data on-boarding and hence custom integration work. Therefore, a solid knowledge of AWS infrastructure is a must in order to build different connectors such as FTP, API or JDBC integrations.

Making the data available in the data lake through AWS Glue, Appflow and Lakeformation are responsibilities as well as writing unit/data tests and monitoring the quality of the overall on-boarding process.

Required experience and qualifications:

Experience building data pipelines in Python (experience with PySpark is a plus)

Understanding on AWS Cloud fundamentals (AWS certification is a plus)

Solid knowledge in infrastructure as code -> cdk

Git & CI/CD knowledge is a must

Experience with common data Python libs (pandas, awswrangler,...)

Understands REST APIs from the consumer perspective

ML knowledge is a plus (kubeflow)