Responsible for the execution and quality of the tools, services, datasets delivered by the Data Engineering team.
Build and deploy scalable data engineering solutions on the cloud for engineering, analytics data-science consumers.
Be at the forefront with the business engineering teams to learn, understand, identify and translate functional requirements into technical opportunities.
Own the low-level design and implementation while maintaining high-quality of design /programming standards
End-to-end module ownership of your functional area.
Sharp focus on performance optimizations continuous effort towards improving both read write latencies.
Researching on and integrating any big data tools and frameworks required to provide requested capabilities.
Time split - 80% hands-on programming/execution/research and 20% stakeholder management depending on experience exposure.
Dealing with ambiguity seeking to resolve those by collaborating with all necessary stakeholders.
Ensuring operational efficiency and actively participating in organizational initiatives with the objective of ensuring the highest customer value.
Must Have:
Experience in big data technologies(Apache Hadoop/Spark/Hive/Presto)
Experience with Message Queues(e.g. Apache Kafka/Kinesis/RabbitMQ)
Proficiency in at least one of the following programming languages - Python, Java or Scala.
Experience in building Highly Available, Fault Tolerant REST services preferably for data ingestion or data extraction.
Good understanding of data warehousing fundamentals.
Good exposure to SQL (T-SQL / PL-SQL / SPARK-SQL / HIVE-QL).
Experience with integrating data across multiple data sources.
Good understanding of distributed computing principles.
Strong analytical/quantitative skills and comfortable working with very large varied sets of data.
Good To Have:
Working knowledge of cloud-native (AWS preferred) services/tools.
Experience with MPP data warehouses (e.g.Snowflake/Redshift).
Proficiency in Apache Spark Apache Kafka.
Experience with any NoSQL storage(e.g.Redis / DynamoDB / Memcache).