Data Engineer - Streaming Architecture (6-8 yrs)

Job Description :

Key Skills : Apache Beam/ Flink, Kafka, Spark

Experience : 5+ Years

Description :

- Minimum 5+ years of development and design experience in Java/Scala/Python with Flink, Beam (or similar streaming toolset) and Kafka.

- Extensive coding experience and knowledge in Event driven and streaming architecture

- Experience in JVM tuning for performance

- Knowledge on Containerisation using Docker and Kubernetes.

- Working knowledge on Caching systems, with particular experience using Apache Ignite will be preferable. [Nice to have]

- Linux OS configuration and use, including shell scripting.

- Good hands on experience with design patterns and their implementation.

- Well versed with CI/CD principles (GitHub, Jenkins etc.), and actively involved in solving, troubleshooting issues in distributed services ecosystem

- Experience working with SQL and NoSQL databases

- Familiar with Distributed services resiliency and monitoring in a production environment.

- Experience in designing, building, testing and implementing security systems - including identifying security design gaps in existing and proposed architectures and recommend changes or enhancements.

- Responsible for adhering to established policies, following best practices, developing and possessing an in-depth understanding of exploits and vulnerabilities, resolving issues by taking the appropriate corrective action.

- Knowledge on security controls designing Source and Data Transfers including CRON, ETLs, and JDBC-ODBC scripts.

- Understand basics of Networking including DNS, Proxy, ACL, Policy and troubleshooting

- High level knowledge of compliance and regulatory requirements of data including but not limited to encryption, anonymization, data integrity, policy control features in large scale infrastructures

- Understand data sensitivity in terms of logging, events and in memory data storage- such as no card numbers or personally identifiable data in logs.

- Implements wrapper solutions for new/existing components with no/minimal security controls to ensure compliance to bank standards.

- Experience in Agile methodology.