Analyst, Data Engineer, Future Ready Technology

Business Function

Group Technology and Operations (TO) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality control, technology, people capability and innovation. In Group TO, we manage the majority of the Banks operational processes and inspire to delight our business partners through our multiple banking delivery channels

Roles Responsibilities

Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and translate them effectively using the Spark/SCALA Framework
Should be able to extract the data from source systems (Open/Mainframes) to Data Lake (Hive/Parquet)
Understand the complex transformation logic and translate them to Hive/Spark-SQL queries to load them onto Enterprise Data Domain tables and Data Marts
Hands on experience on dealing with live streaming datasets using Kafka/Spark Streaming
Excellent oral and written communication skills, On-time delivery and good team player

Requirements

2 to 4 years hands on experience on Spark Core, Spark-SQL, Scala-Programming and Streaming datasets in Big Data platform
Should have extensive working experience in Hive and other components of Hadoop eco system(HBase, Zookeeper, Kafka and Flume)
Should be able to understand the complex transformation logic and translate them to Spark-SQL queries
Unix Shell Scripting and setting up CRON jobs
Should have worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler), Jenkins (or any version controller)
Prior experience in Consumer Banking Domain is an advantage
Prior experience in agile delivery method is an advantage
Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions
Familiar with Data Warehouse concepts and Change Data Capture (CDC and SCD Types)
Core Java skillset, Micro services understanding is an added advantage. Airflow is plus to have
Good Knowledge and experience in any database (Teradata or Data Lake or Oracle or SQL Server) is a plus
Bachelor of Engineering, Bachelor of Technology, MCA (or equivalent master s degree)

-en