Lead Data Engineer - Data Warehouse, Digital Business

Purpose: This position is required to provide technology leadership on medium to large scale products/platform with internal and direct customer impact.

Education: B.E./B.Tech, MCA or Equivalent Computer Degree

Purpose:

Big Data team is responsible for providing the cloud-native platform for distributed data processing at SonyLiv, working with Spark, Presto as well as other supporting technologies. This team is central to batch data processing in Data Platform at SonyLiv. It provides support for Spark, to ETL data into the Petabytes-scale data warehouse and access that data using Spark and Presto & different Columnar Db like Redshift. It also provides sub-second latency for a certain class of queries using Timeseries Db for monitoring. This team will collaborate with different stakeholder like Marketing, Analytics team, sales team, content team to organises, classify, filter and structure data to make them available for their queries.

Location: Bengaluru

Reporting Relationship: Engineering Manager, Digital Business

Business/Channel/Function: Digital

Experience:

• Overall 6+ years of IT experience with 5+ years in Data related technologies, and expertise of 3+ years in data-related AWS Cloud services
• Have 3+ year of experience in ClickHouse or any other OLAP system
• Mandatory to have knowledge of RedShift, Presto and BigQuery
• Good to have experience in Druid.
• Expert in SQL, Hive
• Worked on at least one distributed data processing framework: Spark (Core, Streaming, SQL), Storm or Flink, etc
• Experience is building data warehouse for Marketing, sales and content team will be added advantage.

Roles & Responsibility

• Establish and enforce Data Model, Data Warehouse, ETL, and BI standards
• Gather and review analytical needs of key groups such as Sales & Marketing, Finance and Manufacturing
• Train internal customers on available technologies
• Analyse, document, develop, integrate, and deploy analytics as per business needs
• Carry out monitoring and performance tuning in EDW, ETL and BI environments
• Create, Manage the BI reports and Alerts for Product and Business requirement.