Senior Data Engineer

• Combine strong data modeling, integration, and data engineering abilities in Python and SQL
• Utilize the Azure stack and stay informed about the Data team's engineering efforts
• Possess an opinion on information architecture and how it should be developed to meet functional needs and company goals
• Participate actively in the data transformation effort by liaising with the teams responsible for reporting, data science, governance, and business partnerships
• Work with the newest technologies in a cloud-based environment in a multidisciplinary team comprising analysts, data scientists, data engineers, and SMEs to improve the caliber of data-based decision-making throughout the company
• Participate in the creation of the platform and support for data analytics, guaranteeing standardized and simple data exchange and access
• Become familiar with and utilize Databricks, Azure Events Hub, Azure Data Lake, and Synapse
• Assist with the development, construction, and upkeep of data pipelines for various data sources and communicate with system owners
• Be proficient at ingesting and transforming data in real-time to meet our use cases
• Make that all data interfaces are consistent with governance and provide high-quality data tools and capabilities
• Make certain that the transformation backlog is sent to the data products, derived tables, and source systems
• Analyze complicated data and present your findings
• Collaborate closely with other senior and lead engineers to support BAU efforts and develop new products
• Troubleshoot BAU-related problems, create a JIRA, and work on process improvements
• Encourage others to build on your work and share your knowledge with your teammates
• Engage in on-call duties to assist with any significant malfunctions.
• Engage in a rotating on-call schedule for production support
• Assist with sporadic data requests and operational problems

Job Requirements:

• Bachelor s/Master s degree in Engineering, Computer Science (or equivalent experience)
• At least 10+ years of relevant experience as a data engineer
• 10+ years of experience in a data engineering capacity with exceptional hands-on abilities
• Data Engineering certification as an Azure Data Engineer or Google Certified Professional
• Ability to oversee initiatives on an individual basis
• Extensive knowledge of Python, Pyspark, and SQL
• Working knowledge of and comprehension of Azure Databricks, Azure Events Hub, Azure Data Lake, and Azure Synapse
• Strong experience in data warehousing, big data processing, ETL workflow design, and data pipeline architecture
• Proven background in creating Python frameworks
• Extensive background in analytics solutions and data mapping for business processing.
• Substantial expertise in the concepts of corporate data modeling
• Strong debugging and troubleshooting abilities
• Excellent English communication skills, both spoken and written