Senior Platform Data Engineer

Engineer will work within a product team to design, develop, maintain, and evolve an internal developer platform (IDP). This platform provides tools to accelerate and standardize the delivery of digital, data, and AI products within the Azure ecosystem.

Responsibilities:

Platform Development :

Implement reusable modules in Python.

Develop command-line features for deploying and managing code in Azure.

Maintain and improve code generation templates.

Release Process Maintenance :

Set up CI pipelines to automate testing and documentation.

Set up CD pipelines to automate delivery with release notes.

Implement testing strategies to ensure adequate coverage.

User Support :

Create documentation and knowledge bases.

Participate in demonstrations of new features.

Assist and troubleshoot user issues.

Collaboration with Data Experts :

Assist Data Scientists and MLEs with model experiments and deployments.

Help Data Engineers with Databricks tasks.

Support Data Analysts in industrializing their DBT workflows.

Security and Compliance :

Adhere to industry standards and privacy regulations.

Integrate automated security and compliance checks.

Continuous Improvement :

Stay up-to-date on technological advancements.

Propose and implement optimizations.

Proactively identify areas for improvement and automation.

Desired Skills:

Languages and Tools : Python, CLI, Cookiecutter, Databricks, DBT.

DevOps and CI/CD : Azure DevOps, Docker.

Cloud : Azure, Azure Databricks, Azure Machine Learning Services