Senior Technical Support Engineer

Job Description

Should have 3-6 years of Experience in Production Support, primarily managing Data pipelines.
Ability to support a mission-critical 24/7/365 environment.
Should have worked in Big Data Technologies for at least 2 years
Ability to read, understand and communicate complex technical information
Ability to express ideas in an organized, articulate and concise manner
Any certification in regards to Big Data services or Google cloud will be an added advantage

Experience in big data technologies like Hadoop, Hive, Spark, Oozie
Experience with working on large scale On-Premises ETL and Data servers.
Experience with ETL and Data pipeline workflow management tools: Autosys, TWS, Airflow, etc.
Advanced working SQL knowledge and experience working with relational databases and familiarity with a variety of databases (Teradata, Vertica, Oracle, Hive etc).
Understanding of data ingestion and hands on experience of running tools like Sqoop, Flume
Hands on experience in debugging Map reduce jobs, Hive queries, Spark jobs for failures and slowness
Ability to analyze logs for error and exceptions Ability to drill down errors to cluster issues, code issues etc.
Proficiency in at least one scripting language (Shell/Perl/Python etc.)
General operational exposure such as good troubleshooting skills, understanding of system’s capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
Good knowledge of Linux and debugging skills
Hands on experience on monitoring tools like Splunk, Dynatrace, Nagios, Ganglia etc. Ability to deploy, create plug-ins for custom monitoring stats
Strong verbal and written communication skills are mandatory
Excellent analytical and problem solving skills are mandatory
Solid troubleshooting abilities and able to work with a team to fix large production issues

Roles & Responsibilities

Should be capable of working independently & responsible for L1/L2/L3 Support.
Customer oriented and willing to work in 24/7 shift mode.
Should work with complete ownership and handle all issues without missing SLA’s
Should be able to debug the issues and report appropriately in the ticketing system
Handover the shift and circulate a daily report to project team.
Contribute to improve the efficiency of the assignment by quality improvements & innovative suggestions.
Create and maintain optimal ETL and Data pipelines on Bigdata framework.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Hive queries, etc.
Work with internal and external stakeholders with ETL and data-related technical issues and support their infrastructure needs.