Data Engineer - ETL/SQL (3-4 yrs)

Role : Data Engineer

Job Description :

We are looking for a Data Engineer to join our Data Science team. As a Data Engineer, you will play a key role in designing, implementing and maintaining the data infrastructure of our organization. Your primary responsibility will be to develop and optimize data pipelines, databases, and ETL processes, ensuring the efficient and reliable flow of data across systems.

The role offers a challenging and exciting opportunity to be a part of industry-disrupting solutions and do cutting-edge work in data science.

Job responsibilities :

- Design and implement data infrastructure and processing workflows required to support data science, machine learning. Support data science team by building industry level data lake environment / data warehouse ecosystem that supports AI products at scale

- Design, implement and optimize distributed computing systems for big data processing.

- Utilize Hadoop, Spark, MapReduce, HDFS for data storage, retrieval and preprocessing.

- Drive the collection of new data and refinement of existing data sources.

- Share data needs and data sources, understand specific requirements and build scalable and efficient pipelines.

- Design and optimize SQL queries, joins, stored procedures and relational schemas

- Knowledge of No-SQL databases like MongoDB for specific use-cases

- Apply ETL techniques and frameworks for data transformations and integration

- Own the data expertise and data quality for the pipelines

- Perform performance tuning of Spark jobs to optimize data processing and analytics

- Familiarity with data governance and data security practices

Requirements :

- Advanced Degree in Computer Science, Data Engineering, Software, IT or a related technical field

- 3+ year experience in Big Data and related tech.

- Knowledge of ETL techniques and frameworks & experience in distributed computing principles, schema design and dimensional data modelling

- Knowledge in integrating data from multiple sources, including RDBMS

- Experience in SQL and NoSQL databases

- Knowledge of Spark & familiarity with Hadoop, MapReduce, HDFS & columnar databases

- Knowledge of implementing data lake systems using specific AWS tech such as Glue, Redshift, S3 etc.

Behavioral Competencies :

- Willingness to learn, self-motivated, self-driven

- Ownership and team-sprit

- A quick learner with analytical rigor, curiosity, open-mindedness and creativity.

- Excellent communication skills to convey ideas and information clearly and accurately to others

We solve complex, completely new problems in the financial domain which requires someone who thrives on challenges and is focused to deliver exceptional results. If this role sounds like a fit for you, we encourage you to apply even if you don't meet every requirement listed above.