Data Specialist

Job Description:

The role requires data ingestion, processing and extraction of data to & from Hadoop Cluster using Informatica BDM. It requires experience in ETL, SQL, Scoop, Spark, Hive & Hadoop. Knowledge of Informatica EDC is desirable.

Shift Timings - 04:00PM TO 01:00AM

Experience:

• Extensive experience in delivering data pipelines using Cloudera distributions
• Hands on experience with extraction of unstructured and semi structured data using data processor transformation
• Working Knowledge of Informatica EDC tool is desirable

Education, Skillsets& Qualifications:

• Bachelor's degree in Computer Science or equivalent; Masters preferred Programming
• Solid programming skills in SQL
• Knowledge in data architecture, designing workflows, building ingestion framework
• Expertise of working on Enterprise Big data distribution like Cloudera. Exposure to big data components HDFS, HIVEQL, Spark, HBase, Impala is essential

Main Duties & Responsibilities:

• Building and optimizing data pipelines architectures and data sets; both batch-oriented and real time using Informatica BDM from both relational & non relational DBs
• Design, develop and customize mappings/workflows for different types of loads using Informatica BDM
• Leverage various transformations in Informatica BDM tool to work on data ingestion, data profiling & data quality
• Leverage Hadoop ecosystem knowledge to design, and develop capabilities to deliver our solutions
• Support and play a pivotal role as part of data engineering team to build the data lake on Cloudera for Enterprise analytics.
• Work hands on with Spark &KAFKA in addition to Informatica platform
• Design recommend best approach suited for data movement from different sources to HDFS
• Ensure optimum performance of the Informatica mappings and workflow
• Work with dynamic cross functional teams including business, Solution architects & Data engineers