Azure and Databricks Data Engineer

• Utilizing the Azure ecosystem and Databricks platform, create, build, and maintain data engineering solutions
• Build scalable and effective data pipelines, perform data transformation and integration, and enable sophisticated analytics and machine learning workflows while collaborating closely with cross-functional teams that include data scientists, analysts, and software engineers
• Utilize your knowledge of Databricks and Azure services to support business intelligence efforts and deliver data-driven insights
• On the Azure platform with Databricks, help to design, create, and deploy end-to-end data engineering solutions
• With the help of stakeholders, collect and interpret business requirements into technical data engineering solutions
• Execute data ingestion operations, which involve gathering data from diverse sources, processing it to suit business requirements, and feeding it into data lakes or data warehouses
• Utilizing Azure Data Factory, Databricks, and other pertinent tools, create and maintain scalable data pipelines and workflows
• For effective data storage and retrieval, create and improve data models and schemas
• Implement security measures, and make sure all legal and regulatory standards are met regarding data protection

Job Requirements:

• Bachelor s/Master s degree in Engineering, Computer Science (or equivalent experience)
• At least 6-8+ years of relevant experience as a data engineer
• Prolific experience working with Azure and Databricks
• In-depth knowledge of the principles, procedures, and best practices of data engineering
• Demonstrable experience leveraging Azure Data Factory, Databricks, and other pertinent technologies to develop and implement data pipelines
• Proficient knowledge of programming languages like Python, SQL, and Scala
• Thorough knowledge of data processing and storage technologies like Apache Spark, Azure Data Lake Storage, and Azure SQL Data Warehouse
• Familiarity with ETL/ELT procedures, schema design, and data modeling
• Proven problem-solving and troubleshooting abilities, including the ability to recognize and fix data-related problems
• Outstanding teamwork and communication abilities to work well with cross-functional teams.
• Data engineering-related Azure certificates or Databricks certifications are nice to have
• Azure credentials in Machine Learning or DevOps are desirable
• Prior remote project management experience is crucial
• Fluent in spoken and written English communication