Senior Data Engineer - Python/SQL (10-15 yrs)

Responsibilities :

- Design and build parts of our data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources using the latest Big Data technologies.

- Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

- Work with machine learning, data, and analytics experts to drive innovation, accuracy and greater functionality in our data system.

Qualifications :

- Bachelor's degree in Engineering, Computer Science, or relevant field.

- 10+ years of experience in a Data Engineer role.

- 3+ years experience with Apache Spark and solid understanding of the fundamentals.

- Deep understanding of Big Data concepts and distributed systems.

- Strong coding skills with Scala, Python, Java and/or other languages and the ability to quickly switch between them with ease.

- Advanced working SQL knowledge and experience working with a variety of relational databases such as Postgres and/or MySQL.

- Cloud Experience with DataBricks

- Experience working with data stored in many formats including Delta Tables, Parquet, CSV and JSON.

- Comfortable working in a linux shell environment and writing scripts as needed.

- Comfortable working in an Agile environment