Senior Data Engineer

Exp : 10-12 years

Notice : Immediate - 30 days

Job Description :

Key Responsibilities :

- Define what data is required to create a solution and work with connectivity engineers , users to collect the data

- Create and maintain optimal data pipeline architecture.

- Assemble large, complex data sets that meet functional / non-functional business requirements.

- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery for greater scalability

- Work on data preparation, data deep dive , help engineering, process and quality to understand the process/ machine behavior more closely using available data

- Deploy and monitor the solution

- Work with data and analytics experts to strive for greater functionality in our data systems.

- Work together with Data Architects and data modeling teams.

Skills /Competencies :

Data Wrangling Skills :

- Strong skill in data mining, data wrangling techniques for creating the required analytical dataset

- Experience building and optimizing 'big data' data pipelines, architectures and data sets

- Adaptive mindset to improvise on the data challenges and employ techniques to drive desired outcomes

Programming Skills :

- Experience with big data tools: Spark, Delta,CDC,NiFi, Kafka, etc

- Experience with relational SQL, NoSQL databases and query languages, including oracle, Hive, sparkQL.

- Experience with object-oriented languages: Scala, Java, C++ etc.

Visualization Skills :

- Know how of any visualization tools such as PowerBI, Tableau

- Good storytelling skills to present the data in simple and meaningful manner

Data Engineering Skills :

- Strong skill in data analysis techniques to generate finding and insights by means of exploratory data analysis

- Good understanding of how to transform and connect the data of various types and form

- Identify opportunities for data acquisition

- Explore ways to enhance data quality and reliability

- Build algorithms and prototypes

- Reformulating existing frameworks to optimize their functioning.

- Good understanding of optimization techniques to make the system performant for requirements

Location : Remote, Chennai, Hyderabad,Kolkata, Pune, Ahmedabad