Azure Data Engineer

Role & responsibilities

Create and maintain data pipelines, data storage solutions, data processing and data integration
Handle and analyze enormous volumes of data using big data technologies like Azure Synapse and Apache Spark.
Software engineering skills: Coding (Python and MS SQL) and source control versioning requirement spec, architecture, and design
Experience with API tools such as Postman
Required Functional/Domain Knowledge

Design, implement, and manage data solutions on the Microsoft Azure cloud platform
• Azure App Services
• Azure-Data bricks
• Azure Functions
• Azure-Logic Apps
• Azure Synapse Analytics
• Azure Cosmos db

Work in Agile environment with agile ceremonies with prioritization from Product Owner
Required to be on-call for 3-4 days, 6-8 times per year- outside regular working hours
Work hours overlapping Pacific Time Zone
Designing, implementing, and maintaining data solutions on the Microsoft Azure cloud platform
Create and maintain data pipelines, data storage solutions, data processing, and data integration to enable data-driven decision-making in the company.
Develop and implement data integration processes, frequently using Azure Databricks
Ensure that the data is efficiently cleaned, converted, and loaded
Create data processing workflows and pipelines to support data analytics, machine learning, and other data-driven applications.
Designs data storage systems that scale, perform, and are economical enough to satisfy the organization's requirements
Assess & implement ways to lower costs
Required Technical Skills
Familiar with data modeling, reviewing Entity Relation Diagram and updating logical / physical models.
Experience in working with Ansible scripts to update/alter ansible playbooks as part of the CI/CD deployments.
Experience in Azure DevOps to manage deployments / code repositories
Experience in coding in Azure Databricks using python programming language. Able to work with both batch and streaming data processes.

Preferred candidate profile

Perks and benefits