Lead Consultant ,PySpark/Spark Developer

Undertaking end-to-end project delivery (from inception to post-implementation support), including review and finalization of business requirements, creation of functional specifications and/or system designs, and ensuring that end-solution meets business needs and expectations.

Analysis of existing designs and interfaces and applying design modifications or enhancements
Coding and documenting data processing scripts and stored procedures.

Providing business insights and analysis findings for ad-hoc data requests
Testing software components and complete solutions (including debugging and troubleshooting) and preparing migration documentation.

Providing reporting-line transparency through periodic updates on project or task status
Qualifications we seek in you
Minimum Qualifications
BE/B Tech/MCA
Excellent written and verbal communication skills

Preferred Qualifications/ Skills

Experience with the technical analysis and design, development and implementation of data warehousing / Data Lake solutions;
Well versed and Hand-on in Python, Spark, Pyspark,Spark SQL optimization.

Good understanding/experience in Hive/Impala/Spark
Strong SQL programming and stored procedure development skills
Hadoop experience and knowledge of SQL
Relational database experience (experience with Teradata is a big plus)
UNIX Shell scripting experience to support data warehousing solutions
Process oriented, focused on standardization, streamlining, and implementation of best practices delivery approach
Excellent problem solving and analytical skills
Excellent verbal and written communication skills
Experience in optimizing large data loads
Experience in OLAP/Relational databases would be good.

Exposure to an Agile Development environment would be a plus.

Knowledge about TWS Scheduler would an added advantage
Strong understanding of Data warehousing domain
Ability to architect an ETL solution and data conversion strategy
Good understanding of dimensional modelling