Data Engineer - SQL/ Python (8-20 yrs)

Job Description :

- Over 8 years of IT experience in the Data warehouse area, Python and SQL.

- Expert level on SQL

- Expert level on AWS (S3, Lambda, Step Function, SQS, SNS, CloudWatch)

- Expert level on Python (Pandas and NumPy)

- Preferred Snowflake and Snowflake Stored Procedure experience

- Very strong hands-on experience in all the above technologies.

- Strong analytical and communication skills

- Should be able to manage a team of data engineers in terms of managing own delivery as well as manage team's delivery.

- Pitch-in and guide the team to ensure customer requirements are met.

- Review the work of the team and ensure quality deliverables are being delivered within the expected timeline.

Responsibilities :

- Function as integrators between business needs and technology solutions, helping to create technology solutions to meet clients' business needs

- Categorizing, cataloging, cleansing and normalizing of datasets

- Providing users access to datasets using REST and Python APIs

- Extraction, transformation, and loading of data from a wide variety of data sources using Python, SQL and AWS technologies

Communicating with business users and technology stakeholders

Create and maintain optimal data pipeline architecture.

Requirements :
- Self-starter and critical thinker, takes ownership of own projects and makes improvement suggestions for the entire infrastructure

- Broad understanding of fixed income, derivatives, futures, FX, or other financial-services instruments

- Can handle several projects with different priorities at the same time in a fast-paced environment

- Other : Elastic stack, RESTful APIs, Node.js

- Big data tools : Hadoop, Spark, Presto, Kafka, etc

- Object-oriented languages : Java, C++, etc

- Excellent self-management and problem-solving skills

- Can work independently and in a collaborative environment

- RDBMS : SQL Server and PostgreSQL

- Unix/Linux command-line experience

- Proactive, assertive and attentive to details

- Relational SQL database development

- Quick learner

- Excellent listening, and communication (both oral and written) skills

- Python and data analysis libraries (Pandas, NumPy, SciPy)