Data Engineer

Our Story

Vegapay Technology is a financial technology company. It partners with banks and financial institutions to digitize its financial infrastructure. It provides users with a credit suite featuring a wide breadth of modules and no-code configuration to design, deploy, and direct their credit programs. It provides access to build financial asset products including Card Management System, LOS, LMS, Co-lending and more.

Founded in 2022 by Gaurav Mittal, Himanshu Agrawal, Puneet Sharma and Abhinav Garg, the startup is a B2B digital lending and Card Management Platform. Vegapay s vision is to liberate financial institutions and fintech enterprises from every technical barrier which hinders offering a lending programme.

Meet the Team

Gaurav Mittal - Gaurav is the Co-Founder and the CEO of the company. He is having more than 18 yrs of experience and has worked with organisations like Zeta, Matchmove, MasterCard, Amex and ICICI Bank.

Abhinav Garg - Abhinav is the Co-Founder and the Head of engineering. He is from IIT Roorkee and has more than 12 yrs of experience working with organisations like Podeum and WalmartLabs.

Himanshu Agrawal - Himanshu is the Co-Founder and the Head of Technology. He is from IIT Kanpur and has more than 12 yrs of experience working with organisations like Amazon and DE Shaw.

Puneet Sharma - Puneet is the Co-Founder and the Head of Product. He is from IIT Roorkee and has more than 8 years of experience working with organisations like BharatPe, Avail Finance

Why This Role Matters

The role of a Data Engineer is at the heart of transforming raw data into valuable insights that drive business decisions and innovation. By designing and maintaining robust data pipelines, warehouses, and real-time processing systems, Data Engineers enable seamless access to reliable, scalable, and secure data. They optimize processes, ensure data integrity, and create tools that empower data scientists, analysts, and business teams to uncover trends and opportunities. As organizations increasingly rely on data to stay competitive, the Data Engineer s work ensures the foundation for all data-driven strategies is strong, efficient, and future-ready. This role is vital in shaping the company s data ecosystem into a true engine of growth and innovation.

The Hats You Will Wear

• Create and maintain optimal data pipeline architecture.
• Extract, Transform and Load data from multiple sources and multiple formats using Big Data Technologies.
• Develop and maintain robust data warehouses and lakes on AWS/GCP, ensuring architecture supports efficient data retrieval and storage.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Author complex SQL queries and design data models to optimize for performance and scalability, addressing product and business requirements.
• Implement real-time data processing pipelines within a micro-services architecture, ensuring timely data availability and integrity.
• Collaborate with cross-functional teams (Data Science, Product, Business) to support data infrastructure and analytics initiatives.
• Advocate for and implement best practices in data engineering, including Agile, TDD (Test-Driven Development), and CI/CD (Continuous Integration/Continuous.
• Deployment) to enhance team productivity and product quality.
• Stay abreast of emerging data technologies and evaluate their application to continually improve the data ecosystem within the company.
• Create a conceptual data model to identify key business entities and visualize their relationships.
• Create detailed logical models using business intelligence logic by identifying all the entities, attributes, and their relationships.
• Create a taxonomy/data dictionary to communicate data requirements that are important to business stakeholders work on acquiring external data sets through APIs and/or WebSockets and prepare physical data models on top of that.

The Perfect Fit

• At least 3 years of experience in a Data Engineer role.
• Experience with big data tools: HDFS/S3, Spark/Flink, Hive, Hbase, Kafka/Kinesis, etc.
• Experience with relational SQL and NoSQL databases, including Elastic search and Cassandra/MongoDB.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS /GCP cloud services
• Experience with stream-processing systems: Spark-Streaming/Flink etc.
• Experience with object-oriented/object function scripting languages: Java, Scala, etc.
• Experience building and optimizing big data data pipelines, architectures and data sets.
• Strong analytic skills related to working with structured/unstructured datasets.
• Build processes supporting data transformation, data structures, dimensional modeling, metadata, dependency, schema registration/evolution and workload management.
• Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
• Experience supporting and working with cross-functional teams in a dynamic environment
• Solid understanding of best practices in software development, including Agile methodologies, TDD, and CI/CD processes.