filter salaries All Filters

31 Spark Streaming Jobs

Principal Software Engineer Architect

11-16 years

Bangalore / Bengaluru

1 vacancy

Principal Software Engineer Architect

Epsilon

posted 14d ago

Job Role Insights

Job Description

Why we are Looking for You?


At Epsilon, we run on our peoples ideas. Its how we solve problems and exceed expectations. Our team is now growing, and we are on the lookout for talented individuals who always raise the bar by constantly challenging themselves and are experts in building customized solutions in the digital marketing space.


What you will enjoy in this Role?


So, are you someone who wants to work with cutting-edge technology and enable marketers to create data-driven, omnichannel consumer experiences through data platforms? Then you could be exactly who we are looking for. Apply today and be part of a creative, innovative, and talented team thats not afraid to push boundaries or take risks.


What will you do?


As a Principal Software Engineer in the Epsilon Product Development team, you will design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop. Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure along with building, scheduling and maintaining workflows. Perform data integration and transformation, troubleshoot issues, Document processes, communicate technical concepts clearly, and continuously enhance our attribution engine/forecasting engine.

Strong written and verbal communication skills (in English) are required to facilitate work across multiple countries and time zones. Good understanding of Agile Methodologies – SCRUM.


Qualifications

  • Strong experience (12+ years) in Scala programming language and extensive experience with Apache Spark for Big Data processing for design, developing and maintaining scalable on-prem and cloud environments, especially on AWS EMR.
  • Proficiency in performance tuning of Spark jobs, optimizing resource usage, shuffling, partitioning, and caching for maximum efficiency in Big Data environments.
  • In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.
  • Expertise in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting.
  • Using Python to develop infrastructure modules. Hence, hands-on experience with Python.
  • Solid grasp of database systems and SQLs for writing efficient SQL’s (RDBMS/Warehouse) to handle TBS of data.
  • Familiarity with design patterns and best practices for efficient data modelling, partitioning strategies, and sharing for distributed systems and experience in building, scheduling and maintaining DAG workflows.
  • End-to-end ownership with definition, development, and documentation of software’s objectives, business requirements, deliverables, and specifications in collaboration with stakeholders.
  • Experience in working on GIT (or equivalent source control) and solid understanding of Unit and integration test frameworks.
  • Must have the ability to collaborate with stakeholders/teams to understand requirements and develop a working solution and the ability to work within tight deadlines and effectively prioritize and execute tasks in a high-pressure environment.
  • Must be able to mentor junior staff.

Advantageous to have experience on below:

  • Hands-on with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues.
  • Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualization.
  • Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making.
  • Experience with data visualization tools like Tableau, Power BI, or Grafana.
  • Familiarity with Docker for containerization and Kubernetes for orchestration.
  • Good to have AWS certification.

Employment Type: Full Time, Permanent

Read full job description

Epsilon Interview Questions & Tips

Prepare for Epsilon roles with real interview advice

Compare Epsilon with

Mu Sigma

2.6
Compare

Fractal Analytics

4.0
Compare

TCS

3.7
Compare

Wipro

3.7
Compare

Infosys

3.6
Compare

HCLTech

3.5
Compare

Tech Mahindra

3.5
Compare

Accenture

3.8
Compare

Capgemini

3.7
Compare

Cognizant

3.7
Compare

HGS

3.9
Compare

G4S

4.0
Compare

SGS

3.9
Compare

R.R. Donnelley

3.9
Compare

Iris Software

4.0
Compare

Intertek

3.8
Compare

Nielsen

3.6
Compare

BCD Travel

4.3
Compare

Williams Lea

3.6
Compare

Eicher Motors

4.1
Compare

Similar Jobs for you

Senior Principal Software Engineer at Epsilon

Bangalore / Bengaluru

10-18 Yrs

₹ 40-70 LPA

Data Architect at HashRoot

8-14 Yrs

₹ 35-45 LPA

Senior Staff Engineer at Nagarro

India

10-15 Yrs

₹ 22.5-37.5 LPA

Lead Data Engineer at Cynosure Corporate Solutions

8-12 Yrs

₹ 35-40 LPA

Big Data Engineer Lead at Racrosoft Technologies

10-20 Yrs

₹ 25-40 LPA

Big Data Engineer at Apptad

Delhi/Ncr

5-10 Yrs

₹ 35-40 LPA

Software Python Engineer at Yo HR Consultancy

5-10 Yrs

₹ 15-35 LPA

Principal Data Engineer at T3 Strategic Partners

10-15 Yrs

₹ 30-35 LPA

Data Architect at Virtusa Consulting Services Pvt Ltd

Hyderabad / Secunderabad

15-19 Yrs

₹ 40-50 LPA

Lead Architect at Gyansys Infotech Pvt ltd.

8-13 Yrs

₹ 18-45 LPA

11-16 Yrs

Bangalore / Bengaluru

14d ago·via naukri.com

6-10 Yrs

Pune

23d ago·via naukri.com

5-10 Yrs

New Delhi, Bangalore / Bengaluru, Mumbai

14d ago·via naukri.com

8-13 Yrs

Pune, Chennai

27d ago·via naukri.com

2-5 Yrs

Indore

27d ago·via naukri.com

4-9 Yrs

Pune, Chennai, Mumbai

15d ago·via naukri.com

4-9 Yrs

Chennai, Bangalore / Bengaluru, Hyderabad / Secunderabad

6mon ago·via naukri.com

Data Engineer

Apptware Solutions LLP

5-10 Yrs

Pune

11d ago·via naukri.com

4-9 Yrs

Hyderabad / Secunderabad

30d ago·via naukri.com

5-10 Yrs

Gurgaon / Gurugram

20d ago·via naukri.com
How was your last interview experience?
How was your last interview experience?
Rate your experience using AmbitionBox
Terrible
Terrible
Poor
Poor
Average
Average
Good
Good
Excellent
Excellent