Lead Data Engineer

Job Overview

We are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition, transformation, loading and processing of data from a multitude of disparate data sources, including structured and unstructured data for advanced analytics and machine learning in a big data environment.

Responsibilities:
-Engineer a modern data pipeline to collect, organize, and process data from disparate sources.
-Performs data management tasks, such as conduct data profiling, assess data quality, and write SQL queries to extract and integrate data
- Develop efficient data collection systems and sound strategies for getting quality data from different sources
-Consume and analyze data from the data pool to support inference, prediction and recommendation of actionable insights to support business growth.
-Design and develop ETL processes using tools and scripting. Troubleshoot and debug ETL processes. Performance tuning and opitimization of the ETL processes.
-Provide support to new of existing applications while recommending best practices and leading projects to implement new functionality.
- Collaborate in design reviews and code reviews to ensure standards are met. Recommend new standards for visualizations.
-Learn and develop new ETL techniques as required to keep up with the contemporary technologies.
-Reviews the solution requirements and architecture to ensure selection of appropriate technology, efficient use of resources and integration of multiple systems and technology.
-Support presentations to Customers and Partners
-Advising on new technology trends and possible adoption to maintain competitive advantage

Experience Needed:
-8+ years of related experience is required.
-A BS or Masters degree in Computer Science or related technical discipline is required
-ETL experience with data integration to support data marts, extracts and reporting
-Experience connecting to varied data sources
-Excellent SQL coding experience with performance optimization for data queries.
-Understands different data models like normalized, de-normalied, stars, and snowflake models. Worked with transactional, temporarl, time series, and structured and unstructured data.
-Experience on Azure Data Factory and Azure Synapse Analytics
-Worked in big data environments, cloud data stores, different RDBMS and OLAP solutions.
-Experience in cloud-based ETL development processes.
-Experience in deployment and maintenance of ETL Jobs.
-Is familiar with the principles and practices involved in development and maintenance of software solutions and architectures and in service delivery.
-Has strong technical background and remains evergreen with technology and industry developments.
At least 3 years of demonstrated success in software engineering, release engineering, and/or configuration management.
Highly skilled in scripting languages like PowerShell.
Substantial experience in the implementation and exectuion fo CI/CD processes.

Additional Requirements
-Demonstrated ability to have successfully completed multiple, complex technical projects
-Prior experience with application delivery using an Onshore/Offshore model
-Experience with business processes across multiple Master data domains in a services based company
-Demonstrates a rational and organized approach to the tasks undertaken and an awareness of the need to achieve quality.
-Demonstrates high standards of professional behavior in dealings with clients, colleagues and staff.
-Is able to make sound and far reaching decisions alone on major issues and to take full responsibility for them on a technical basis.
-Strong written communication skills. Is effective and persuasive in both written and oral communication.
- Experience with gathering end user requirements and writing technical documentation
- Time management and multitasking skills to effectively meet deadlines under time-to-market pressure