Senior Data Engineer - Java/Python/SQL/Spring-boot

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries, enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog, ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build external-facing conversational BI chatbots to handle customer requests, while also significantly reducing data exploration efforts for internal data analysts by more than 90%.

 

Key Responsibilities:

Basic Qualification
4 yrs+ of experience in architecture design and development of large-scale
data management platforms and data application with simple solutions
Bachelors or master s degree in Computer Science or related technical
discipline required
Must have extensive hands-on coding and designing skills on
Java/Python for backend
MVC (model-view-controller) for end-to-end development
SQL/NoSQL technology. Familiar with Databases like Oracle, DB2, SQL Server,
etc.
Web Services (REST/ SOAP/gRPC)
React/Angular for front-end (UI front-end nice to have)
Expertise in design and management of complex data structures and data
processes
Expertise in efficiently leveraging the power of distributed big data systems,
including but not limited to Hadoop Hive, Spark, Kafka streaming, etc.
Strong service architecture and development experience with high
performance and scalability
Strong on driving for results and self-motivated, strong learning mindset, with
good understanding of related advanced/new technology. Keep up with the
technology development in the related areas in the industry, which could be
leveraged to enhance current architectures and build durable new ones.
Strong leadership and team player.
Strong skills on mentoring/growing junior people
Payment industry experience is a plus.

Preferred Qualification
Deep knowledge and hands on experience on big data and cloud computing
technologies.
Experience with LLM / GenAI tools / applications and Prompt Engineering
Experience with ETL/ELT tools / applications
Experience with Apache NiFi and Apache Spark for processing large data sets
Experience on Elastic Search
Knowledge on Data Catalog tools
Experience in building Data Pipeline development tools
Experience with Data Governance and Data Quality tools