We are seeking a Data Engineer with 3+ years experience to assist with the architecture and development of the Internal Data Lake as well as derivation of analytics. Reporting to the Head of Data & Analytics you will work closely with key stakeholders to gather and analyze business requirements, leverage existing data and ingest new sources to help build a unified, expansive and effective data lake.
Key Responsibilities
Develop and maintain the Azure Databricks hosted data lake by creating pipelines to extensive internal and external data sources, including writing APIs.
Convert unprocessed data into mastered, easy to use tables for data analysts through ETL.
Promote best practices and help setting up standardized, effective processes to improve your work and work of the team.
Work collaboratively with the data engineering team to envision and layout the roadmap for the architecture of the data lake.
Drive data governance with high standards of quality.
Coordinate with the wider data and analytics team to ensure that the output meets the requirements of data science, reporting and data analysis workstreams.
Contribute to driving the strong data culture behind ION Analytics digital transformation.
Actively build in depth knowledge in a range of domains:
o Commercial: Contracts and client relationships
o Accounting: Financials of ION and External companies
o Markets: M&A, LCM and ECM Deals
o Content: Articles and Journalism
o Product front end: Functionalities and platform offering
o Product back end: Usage and other archived information
Required skills, experiences, and qualifications
- Working knowledge of Machine Learning (ML) or Artificial Intelligence (AI) preferred.
- Proficiency in Python and SQL, Pyspark is a plus.
- Experience with Azure.
- Experience with Databricks.
- Experience with version control in Azure DevOps or GitHub.
- Experience of agile software development methodologies (Scrum, Kanban etc.).
- Strong analytical and problem-solving skills.
- Strong written and verbal communication skills.
- Proactive team player, attention to detail and quality conscious.
- Ability to work to deadlines and manage expectations.
- Be a quick learner and demonstrate ability to adapt to change