Lead Data Engineer

We are looking for a Tech Lead expertise in Advanced Big Data Technology Stack. The person should be a hands-on technical expert in building and deploying applications using Big Data technologies. The person should have progressive experience in building highly scalable distributed systems. The person should have the ability to build a high-performance strong technical team that adheres to the strong quality standard of application development.

Roles and responsibilities

• With over 7 yrs. of hands-on experience with Data Technologies
• Requirement analysis and assess the technical feasibility of proposed solutions.
• Act as the technical specialist in designing and recommending architecture for application development/feature development
• Working on designing the application architecture and estimating effort in delivering features for new requirements
• Implement data ingestion and transform pipeline for analytics and Dashboard reporting for business
• Work in designing large-scale distributed computing applications using tools like Spark, Kafka, Hive, etc.
• Performing validations, reconciliation, and consolidations for the imported data, Data migration, and data generation.
• Effectively communicate with the Engineering Managers and Stakeholders to set the right expectations.

Technical Skills

• Minimum 5 years of progressive experience building solutions in Big Data environments
• Have a strong ability to build robust and resilient data pipelines that are scalable, fault-tolerant, and reliable in terms of data movement
• Hands-on experience of Apache Spark with Python for batch and stream processing
• Should know experience in batch and stream data processing
• Exposure to working on projects across multiple domains
• Hands-on experience in Apache Kafka
• Strong hands-on capabilities in SQL and NoSQL technologies
• Hands-on experience with AWS services like S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK, etc. is must have or similar services of either Azure/GCP
• Strong analytical/quantitative skills and comfortable working with very large sets of data.
• Excellent written and verbal communication skills