TransOrg Analytics - Lead Data Engineer - Hadoop/Spark (6-8 yrs)
TransOrg Analytics
posted 1d ago
Fixed timing
Key skills for the job
Job Description - Lead - Data Engineering
Why would you like to join us?
TransOrg Analytics is a Big Data and Machine Learning solutions and services company which offers advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, APAC and the Middle East. Our Automated Machine Learning product - 'Clonizo' has yielded significant incremental benefits to our clients. TransOrg has won accolades such as 'Top 50 Employers' by the Silicon Review magazine, 'Predictive Analytics Company of the Year' and 'Top 50 Big Data Companies' by the CIO Review magazine.
Overview :
You will have to work with data to help the organization make better business decisions. Using techniques from a range of disciplines, including computer programming, mathematics, and statistics to draw conclusions from data to describe, predict, and improve business performance.
Key Responsibilities [Functional] :
- Analyzing data using statistical techniques and providing reports.
- Developing and implementing databases and data collection systems.
- Acquiring data from primary and secondary sources and maintain data systems.
- Identifying, analyzing, and interpreting trends or patterns in complex data sets
- Lead the Data Engineering through its next phase of growth.
- Design/Implementation/Consulting experience of large-scale data solutions, including data
- Ingestion, Data Modelling, Design & Architecture. The projects may include development, deployment, and infrastructure with complexity around technology, cross-functional dependencies, and multiple stakeholders across various divisions and departments?
Identifying data sources
- Collecting data, Sourcing missing data, Organizing data in to usable formats, Analyzing data to find answers to specific questions, Setting up data infrastructure, Developing, implementing and maintaining databases
- Assessing quality of data and removing or cleaning data
- Generating information and insights from data sets and identifying trends and patterns
- Preparing reports for executive and project teams
- Creating visualizations of data
- A single point contact for all type of data and data related issues. Work closely with vendors, functional, operational, and technical teams across the business to understand their data needs and develop solutions that allow various functions to make data-informed decision.
- Will be responsible for data consistency and accuracy in timely manner pertaining to all data providers/vendors.
- Solid experience leading data teams in developing data fabric platforms.
- Have an eye for Data Quality and understanding of data completeness.
- Understanding different market data used in current business processes and the impact of making changes.
- Ability to build scale and performance as part of development.
- Ability to troubleshoot scale and performance issues, data validation issues
- Expert in implementing security best practices.
- Should know various tolls and techniques to ensure data is accurate and verified.
- Develop, document, maintain and support nimble data solutions that are able to both provide ready access to existing data features and incorporate new features in a timely manner.
- Develop the data roadmap using the most appropriate technologies and manage and expand in house data teams to support growth.
- Work cross-functionally across multiple locations so communication, collaboration, and organization are key to your success.
- Automation of multi-step repetitive tasks.
Skills, Qualifications and Experience required :
- Bachelor's degree in Computer Science, Information Systems (B.Tech/B.E./MCA)
- 6-8 years' experience in building analytics solutions with about 4+ years' experience in Hadoop, Spark and Data Analytics using cloud technologies
- Experience in delivering data pipelines using Cloudera or Hortonworks distributions
- Experience is delivering data pipelines on AWS or Google Cloud
- Good knowledge on frameworks like Spark, Beam, Flink, HDFS, HIVE, HBASE
- Must have knowledge of tools like Sqoop, Flume, Talend, and Kafka
- Candidates having knowledge on Machine Learning will be preferred
- Hands on experience on R or Python-Statistical Programming
- Extraction, Transformation and Loading (ETL) (Microsoft SSIS)
- Analytics / OLAP Cube Development (Microsoft SSAS and MDX), Report Development (Microsoft SSRS)
Good to have :
- Big Data, Hadoop and analytics knowledge
- Data Warehouse knowledge
- Problem-solving and troubleshooting.
- Strong knowledge & experience in Relational databases like Sybase, SQL Server and Oracle
- Experience working with Visualization tools: Power BI, Tableau, Looker, Qlickview, etc.
- Good analytical skills
- Excellent communication skills - both written and verbal.
Functional Areas: Software/Testing/Networking
Read full job descriptionPrepare for Lead Data Engineer roles with real interview advice
7-12 Yrs
Gurgaon / Gurugram, Bangalore / Bengaluru, Mumbai