Big Data - Senior Engineer

4-6 years of overall experience as Big Data Developer with goods hands on with PySpark and Hive

Strong hands on PySpark concepts like - RDDs , Transformations, Actions, Portioning, Data Frames, Datasets etc

Strong hands on with PySpark, specially Spark Sql

Expertise in estimation and design and development of large work items in Big Data (Hive and Spark)

Understanding Development experience in Big Data eco system i.e. but not limited to Hive, Pig, HDFS, Yarn/MR2, Hive, Sqoop Oozie

Experience delivering distributed and highly scalable applications on ----NoSQL/HDFS/Shared relational Databases/Map-Reduce

Experience in application development and deployment with focus on scalability, performance, availability, reliability, security

Hands on knowledge of Unix/Linux shell scripting job orchestration through Autosys.

Performance Tuning, optimization and best practices demonstration o the given technology

Well versed in Agile concepts- planning and executing sprints- preferably using JIRA

Excellent communication skills on both verbal and written for working with multiple cross functional teams like Business analysts, Program Managers etc.