AmbitionBox
Jobs
- Reviews
- Salaries
- Interview Questions
- About Company
- Benefits
- Jobs
- Office Photos
- Community
Add office photos
All Filters
12 Derisk360 Jobs
Big Data Engineer
5-8 years
Chennai, Gurgaon / Gurugram, Bangalore / Bengaluru
1 vacancy
Big Data Engineer
Derisk360
posted 4d ago
Job Role Insights
Key skills for the job
Job Description
- Develop, optimize, and maintain streaming data pipelines using PySpark and Apache Kafka.
- Design and implement scalable ETL processes for real-time and batch data processing.
- Work with Amazon EMR, Apache Spark, Apache NIFI , or similar frameworks to build near real-time data pipelines.
- Develop data solutions and frameworks to handle high-volume, high-velocity data streams.
- Implement data storage solutions for structured and unstructured data, ensuring efficiency and reliability.
- Write clean, maintainable, and well-documented code using Python, Groovy, or Java .
- Optimize data structures and schemas for data processing and retrieval efficiency .
- Collaborate with cross-functional teams to define, design, and implement data-driven solutions.
- Troubleshoot and resolve performance bottlenecks and data quality issues .
- Stay updated with the latest technologies and best practices in big data and streaming architectures .
- Proficiency in PySpark and strong understanding of distributed computing principles .
- Hands-on experience with Apache Kafka (including kSQL, Mirror Maker, or similar tools) for real-time data streaming.
- Strong programming skills in at least one of the following: Python, Groovy, or Java .
- Good understanding of data structures, ETL design, and data storage solutions .
- Experience in working with Amazon EMR, Apache Spark, Apache NIFI , or similar streaming data frameworks.
- Ability to design and implement scalable and high-performance data pipelines .
- Experience with cloud-based big data solutions (AWS, GCP, or Azure) is a plus.
- Strong problem-solving and analytical skills.
- Excellent communication and collaboration abilities.
- Experience with SQL and NoSQL databases .
- Exposure to containerization technologies such as Docker and Kubernetes.
- Familiarity with CI/CD pipelines for data engineering workflows.
- Understanding of data governance, security, and compliance best practices.
- Work on cutting-edge real-time streaming and big data technologies .
- Collaborate with an expert team in a fast-paced, dynamic environment.
- Competitive compensation and opportunities for professional growth .
- Gain experience in modern cloud-based data architectures .
Employment Type: Full Time, Permanent
Read full job description What people at Derisk360 are saying
Derisk360 Benefits
View more benefits
Compare Derisk360 with
KPMG India
Deloitte
PwC
Ernst & Young
Protiviti
Control Risks
Marsh
Aon
BDO
Grant Thornton
TCS
Accenture
Wipro
Cognizant
Capgemini
HDFC Bank
Infosys
ICICI Bank
HCLTech
Tech Mahindra
Similar Jobs for you
Senior Data Analyst
8-13 Yrs
Chennai, Gurgaon / Gurugram, Bangalore / Bengaluru
GCCAI Architect
5-10 Yrs
Hyderabad / Secunderabad, Pune, Bangalore / Bengaluru
Application Security Engineer (DevSecOps & VAPT)
3-8 Yrs
Mumbai, New Delhi, Chennai +1 more
ServiceNow CMDB Implementation and support Specialist
5-10 Yrs
Kolkata, Mumbai, New Delhi +4 more
