Data Engineer 3

About the role

Must Have Skills - Data Engineering + Java or Scala + Pyspark

This role is for a Senior Big Data Engineer focusing on developing and maintaining data processing systems. The position requires hands-on experience with distributed computing frameworks like Apache Spark, Akka, and Hadoop, along with programming skills in Java or Scala. The ideal candidate will contribute to architectural discussions, help implement data pipelines, and support system optimization efforts. The role involves analyzing existing systems, identifying improvement areas, and working collaboratively with senior team members to resolve bottlenecks. Experience with ETL tools like Apache NiFi and Airflow is valuable, along with knowledge of version control systems and build tools. The position requires writing clean, maintainable code and participating in production support. Working in an Agile environment, the role emphasizes both technical skills and the ability to collaborate effectively with team members while handling client coordination under guidance from senior engineers.

Apply at : https://app.fabrichq.ai/jobs/d5cc82d5-69f6-475d-bb39-244bde23a66f

About the company

Fabric is an AI-powered technical screening platform that helps developers connect with their next opportunity. When you apply through Fabric, you'll participate in a thorough technical interview with our AI system, which you can complete at any time that works for you.

The interview focuses on understanding your actual technical experience and problem-solving abilities, not just keywords on your resume. If you pass the technical assessment, your profile can receive up to 15 interview invites from different companies. We work with some of India's leading tech companies, which means your single interview with Fabric could open doors to multiple promising opportunities.

Fabric streamlines the early stages of the hiring process so you can spend less time in preliminary screenings and more time having meaningful conversations with potential employers.

Expected Compensation

Upto 35 Lakhs Annually

Experience

6 to 10 years of data engineering experience

Key Responsibilities

• Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.
• Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.
• Design and implement systems to cleanse, process, and analyze large data sets using distributed processing tools like Akka and Spark.
• Understanding and critically reviewing existing data pipelines, and coming up with ideas in collaboration with Technical Leaders and Architects to improve upon current bottlenecks • Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.
• 7+ years of experience in developing highly scalable Big Data pipelines.
• In-depth understanding of the Big Data ecosystem including processing frameworks like Spark, Akka, Storm, and Hadoop, and the file types they deal with.
• Experience with ETL and Data pipeline tools like Apache NiFi, Airflow etc.
• Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design Patterns when required.
• Experience with Git and build tools like Gradle/Maven/SBT.
• Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization.
• Have elegant, readable, maintainable and extensible code style.
• Troubleshoot live production server issues.
• Handle client coordination and be able to work as a part of a team, be able to contribute independently and drive the team to exceptional contributions with minimal team supervision • Follow Agile methodology, JIRA for work planning, issue management/tracking