Azure Data Engineer

Role & responsibilities

• A good understanding of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ELT and reporting/analytic tools
• Azure experience, including expert understanding of Azure Data Lakes
• Experience in the extraction of data from SQL servers
• Must have experience in at least two programming languages among Java, Scala or Python
• Experience in wider financial services or insurance sector is preferred.
• High level of accuracy and attention to detail and the ability to simplify problems into component parts and deal with them systematically
• Understanding of Meta-data driven data ingestion using Kafka
• Experience in coaching / mentoring less experienced engineers through technical problem solving
• Strong customer / value focus
• Driven to deliver business value
• Desirable but not essential - understanding of Postgres databases
• Desirable but not essential - experience of data visualization tools (QlikSense etc)

Preferred candidate profile

• Degree, MSc or PhD in a numerate or scientific discipline (e.g. Computer Science, Physics, Mathematics etc)
• 4+ years of experience in developing scalable solutions using big data technologies
• Hands-on development experience on Hadoop, spark, Hive, H-Base, Kafka
• Experience in ELK stack
• Experience in NoSQL databases like Cassandra, MongoDB etc
• Cloud experience (Azure(preferred) or AWS)
• Experience in application, data, and infrastructure architecture disciplines
• Translate, load, and exhibit unrelated data sets in various formats and sources like JSON, text files, Kafka queues, and log data.
• Proven design, coding, testing, and debugging skills and experience.
• Experience in end-to-end release of highly reliable applications including design, development, and testing.
• Understanding of Linux OS core principles, performance and tuning
• A solid understanding of database design and data manipulation
• Experience in building data processing, ETL and data pipelining workflows

Perks and benefits