Senior Data Engineer

Design, build, and maintain efficient and scalable data pipelines that collect,transform, and load data into Hive


Write ad

hoc spark jobs / presto / hive queries for various ad

hoc analysis
Develop python scripts to collect and ingest third party data into Uber Data Lake
Write various tests to detect various data quality issues like duplicates, nullcounts, data drifts etc




Proactively identify the opportunities to optimize the pipelines
Communicate with multiple stakeholders to gather requirements / understandthe scope of the work
Work with upstream and downstream teams to ensure end to end execution ofthe data pipelines
Maintain Various Dashboards of business critical metrics