Home
Communities
Companies
- Companies
  
  Discover best places to work
- Compare Companies
  
  Compare & find best workplace
- Add Office Photos
  
  Bring your workplace to life
- Add Company Benefits
  
  Highlight your company's perks
Reviews
- Company reviews
  
  Read reviews for 6L+ companies
- Write a review
  
  Rate your former or current company
Salaries
- Browse salaries
  
  Discover salaries for 6L+ companies
- Salary calculator
  
  Calculate your take home salary
- Are you paid fairly?
  
  Check your market value
- Share your salary
  
  Help other jobseekers
- Gratuity calculator
  
  Check your gratuity amount
- HRA calculator
  
  Check how much of your HRA is tax-free
- Salary hike calculator
  
  Check your salary hike
Interviews
- Company interviews
  
  Read interviews for 40K+ companies
- Campus placements
  
  Interviews questions for 2K+ colleges
- Share interview questions
  
  Contribute your interview questions
Jobs
Awards

WINNERS AWAITED!
- ABECA 2025
  
  WINNERS AWAITED!
  
  AmbitionBox Employee Choice Awards - 4th Edition
- ABECA 2024
  
  AmbitionBox Employee Choice Awards - 3rd Edition
- AmbitionBox Best Places to Work 2022
  
  2nd Edition
- AmbitionBox Best Places to Work 2021
  
  1st Edition

For Employers

Add office photos

Employer? Claim Account for FREE

Luxoft

Compare

3.7

based on 641 Reviews

Video summary

2 Luxoft Big Data Engineer Jobs

BigData Engineer - Remote India

Luxoft

3.7

based on 641 Reviews

8-13 years

Bangalore / Bengaluru

1 vacancy

BigData Engineer - Remote India

Luxoft

posted 12d ago

Job Role Insights

Flexible timing

Key skills for the job

Python Java ETL Testing Hadoop Administration Pyspark Kafka

+ 1 more

Job Description

Responsibilities:• Highly capable in learning new technologies & frameworks and implementing them as per the project requirements by adhering to quality standards.
• Experience in all phases of data warehouse development lifecycle, from gathering requirements to testing, implementation, and support
• Adept in analyzing information system needs, evaluating end-user requirements, custom designing solutions and troubleshooting information systems.
• Develop and implement data pipelines that extracts, transforms, and loads data into an information product that helps to inform the organization in reaching strategic goals.
• Investigate and analyze alternative solutions for data storage, processing etc. to ensure most streamlined approaches are implemented.
• Ensure operational resiliency of existing data pipelines by monitoring and resolving any issues.
• Communicate, collaborate and work effectively in a global environment.
• Lead projects through design, implementation, automation, and maintenance for large scale ETL processes supporting multiple business units.
• Leverage industry best practices including proper use of source control, participation in code reviews, data validation and testing.
• Implement best practices in Data Governance to ensure the data is available, usable and secure according to internal policies.
• Mentor other Data Engineers on the team and ensure the efficient execution of their duties.
• Assist in leading the development team and serve as a technical resource for team members.
• Leverage new technologies and approaches to innovating with increasingly large data sets.
• Ability to write algorithms with different rules.
• Data warehousing principles & concepts and modification of existing data warehouse structures.
Mandatory Skills Description:• Must have experience deploying and working with big data technologies like Hadoop, Spark, and Sqoop.
• Experience with streaming frameworks like Kafka.
• Experience designing and building ETL pipeline using NiFi.
• Highly proficient in OO programming (Python, PySpark Java, and Scala).
• Experience with the Hadoop Ecosystem (HDFS, Yarn, MapReduce, Spark, Hive, Impala).
• Proficiency on Linux, Unix command line, Unix Shell Scripting, SQL and any Scripting language.
• Proficiency on Linux, Unix command line, Unix Shell Scripting, SQL and any Scripting language.
• Experience designing and implementing large, scalable distributed systems.
• Ability to debug production issues using standard command line tools.
• Create design documentation and maintain process documents.
• Ability to debug Hadoop / Hive job failures.
• Ability to use Cloudera in administering Hadoop.
Nice-to-Have Skills Description:• Cloud technologies like Databricks, AWS, Azure and GCP.