Home
Communities
Companies
- Companies
  
  Discover best places to work
- Compare Companies
  
  Compare & find best workplace
- Add Office Photos
  
  Bring your workplace to life
- Add Company Benefits
  
  Highlight your company's perks
Reviews
- Company reviews
  
  Read reviews for 6L+ companies
- Write a review
  
  Rate your former or current company
Salaries
- Browse salaries
  
  Discover salaries for 6L+ companies
- Salary calculator
  
  Calculate your take home salary
- Are you paid fairly?
  
  Check your market value
- Share your salary
  
  Help other jobseekers
Interviews
- Company interviews
  
  Read interviews for 40K+ companies
- Campus placements
  
  Interviews questions for 1K+ colleges
- Share interview questions
  
  Contribute your interview questions
Jobs
Awards

WINNERS AWAITED!
- ABECA 2025
  
  WINNERS AWAITED!
  
  AmbitionBox Employee Choice Awards - 4th Edition
- ABECA 2024
  
  AmbitionBox Employee Choice Awards - 3rd Edition
- AmbitionBox Best Places to Work 2022
  
  2nd Edition
- AmbitionBox Best Places to Work 2021
  
  1st Edition

Add office photos

Engaged Employer

IBM

Compare

4.1

based on 21.2k Reviews

Proud winner of ABECA 2024 - AmbitionBox Employee Choice Awards

Top Rated Mega Company - 2024

Top Rated Company for Women - 2024

Top Rated IT/ITES Company - 2024

2321 IBM Jobs

Data Engineer: Data Platforms-AWS

IBM India Pvt. Limited

4.1

based on 21.2k Reviews

2-6 years

₹ 4.5 - 28L/yr (AmbitionBox estimate)

Bangalore / Bengaluru

Data Engineer: Data Platforms-AWS

IBM

posted 8hr ago

Job Role Insights

Flexible timing

Key skills for the job

Data Analysis DevOps Data Engineering Python AWS SQL

+ 20 more

Job Description

Your Role and Responsibilities
As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs.

Your primary responsibilities include:

Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements.
Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization.
Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too.

Required Technical and Professional Expertise

Developed the Pysprk code for AWS Glue jobs and for EMR.. Worked on scalable distributed data system using Hadoop ecosystem in AWS EMR, MapR distribution..
Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).
Developed Hadoop streaming Jobs using python for integrating python API supported applications..
Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations..
Re - write some Hive queries to Spark SQL to reduce the overall batch time

Preferred Technical and Professional Expertise

Understanding of Devops.
Experience in building scalable end-to-end data ingestion and processing solutions
Experience with object-oriented and/or functional programming languages, such as Python, Java and Scala

Employment Type: Full Time, Permanent

Read full job description