Data Engineer

• Work closely with Engineering and Analytics teams to assist in Schema Designing, Normalization of Databases, Query optimization, etc.
• Work with AWS cloud services: S3, EMR, Glue, RDS, Redshift.
• Create new and improve existing infrastructure for ETL workflows from a wide variety of data sources using SQL, NoSQL, and AWS big data technologies.
• Manage and monitor performance, capacity, and security of database systems and regularly perform server tuning and maintenance activities.
• Manage and process our Data Lake and ETL pipelines.
• Debug and troubleshoot database errors.
• Identify, design, and implement internal process improvements; optimizing data delivery, re-designing infrastructure for greater scalability, data archival.
• Lead the data projects backed by solid data points and facts.
• You'll be responsible for guiding the data team towards the company s goals.

Data Projects You'll be working on

• Slice Data-lake: S3 based data warehouse built using streaming data pipelines and AWS Glue ETL processes and AWS Athena
• Slice User Graph: A graph DB to be built and maintained on top of AWS Neptune Graph DB, for easy access to the info related to connected slice users
• Data Pipelines: Dumping data from various sources like SQL, NoSQL, Ledger DBs to Data Lake and/or Redshift for Analytics use cases
• Redshift Optimizations: Maintain and work towards keeping the Data warehouse healthy and suggest and implement optimizations

• 6- 8 years of experience in working as a Data Engineer.
• 2+ years of experience in handling ETL Jobs.
• 2+ years of experience with Spark and Hadoop technologies.
• Good knowledge of OLAP Database warehouse designing.
• Experience with a scripting language - Python preferably.
• Experience with SQL and NoSQL databases technologies like Redshift, MongoDB, Postgres/MySQL.
• Experience with AWS big data tools EMR, AWS Glue, S3 is a plus.
• Experience on Graph DB (Neo4j and/or OrientDB) AWS Neptune and Search DB (Elastic Search) is an added advantage.