Upload Button Icon Add office photos
filter salaries All Filters

900 Virtusa Consulting Services Jobs

Databricks + PySpark Professional

7-9 years

Bangalore / Bengaluru

1 vacancy

Databricks + PySpark Professional

Virtusa Consulting Services

posted 1d ago

Job Role Insights

Flexible timing

Job Description

Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines using PySpark and Databricks for ETL processing of large volumes of data.
Cloud Integration: Develop solutions leveraging Databricks on cloud platforms (AWS/Azure/GCP) to process and analyze data in a distributed computing environment.
Data Modeling: Build robust data models, ensuring high-quality data integration and consistency across multiple data sources.
Optimization: Optimize PySpark jobs for performance, ensuring the efficient use of resources and cost-effective execution.
Collaborative Development: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver actionable insights.
Automation & Monitoring: Implement monitoring solutions for data pipeline health, performance, and failure detection.
Documentation & Best Practices: Maintain comprehensive documentation of architecture, design, and code. Ensure adherence to best practices for data engineering, version control, and CI/CD processes.
Mentorship: Provide guidance to junior data engineers and help with the design and implementation of new features and components.
Required Skills & Qualifications:
Experience: 6+ years of experience in data engineering or software engineering roles, with a strong focus on PySpark and Databricks.
Technical Skills:
Proficient in PySpark for distributed data processing and ETL pipelines.
Experience working with Databricks for running Apache Spark workloads in a cloud environment.
Solid knowledge of SQL, data wrangling, and data manipulation.
Experience with cloud platforms (AWS, Azure, or GCP) and their respective data storage services (S3, ADLS, BigQuery, etc.).
Familiarity with data lakes, data warehouses, and NoSQL databases (e.g., MongoDB, Cassandra, HBase).
Experience with orchestration tools like Apache Airflow, Azure Data Factory, or DBT.
Familiarity with containerization (Docker, Kubernetes) and DevOps practices.
Problem Solving: Strong ability to troubleshoot and debug issues related to distributed computing, performance bottlenecks, and data quality.
Version Control: Proficient in Git based workflows and version control.
Communication Skills: Excellent written and verbal communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
Education: Bachelor or Masters degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

Employment Type: Full Time, Permanent

Read full job description

Prepare for Professional roles with real interview advice

Top Virtusa Consulting Services Professional Interview Questions

Q1. Reverse Stack with Recursion Reverse a given stack of integers using recursion. You must accomplish this without utilizing extra space beyo ... read more
Q2. Word Occurrence Counting Given a string 'S' of words, the goal is to determine the frequency of each word in the string. Consider a word as ... read more
Q3. Number and Digits Problem Statement You are provided with a positive integer N . Your task is to identify all numbers such that the sum of ... read more
View all 351 questions

What people at Virtusa Consulting Services are saying

What Virtusa Consulting Services employees are saying about work life

based on 4.8k employees
73%
88%
71%
93%
Flexible timing
Monday to Friday
No travel
Day Shift
View more insights

Virtusa Consulting Services Benefits

Submitted by Company
Paid Sick Leave
Group Insurance
Flexible Schedule
Education/Training/Tuition/Certification Reimbursement
Submitted by Employees
Work From Home
Health Insurance
Cafeteria
Soft Skill Training
Job Training
Gymnasium +6 more
View more benefits

Compare Virtusa Consulting Services with

Cognizant

3.7
Compare

TCS

3.7
Compare

Infosys

3.6
Compare

Accenture

3.8
Compare

Google

4.4
Compare

Wipro

3.7
Compare

HCLTech

3.5
Compare

Tech Mahindra

3.5
Compare

Capgemini

3.7
Compare

IBM

4.0
Compare

Genpact

3.8
Compare

DXC Technology

3.7
Compare

Sutherland Global Services

3.6
Compare

Optum Global Solutions

4.0
Compare

FIS

3.9
Compare

CGI Group

4.0
Compare

GlobalLogic

3.6
Compare

Bosch Global Software Technologies

3.9
Compare

UST

3.8
Compare

Atos

3.9
Compare

Similar Jobs for you

Azure Data Engineer at Glorious Insight

3-10 Yrs

₹ 8-24 LPA

Azure Data Engineer at Feuji Software Solutions

Hyderabad / Secunderabad

6-10 Yrs

₹ 7.5-12 LPA

Professional at Virtusa Consulting Services Pvt Ltd

Chennai

7-12 Yrs

₹ 9-14 LPA

Engineer at Leuwint Technologies

4-8 Yrs

₹ 8-20 LPA

Azure Data Engineer at Xanika Infotech

4-8 Yrs

₹ 8-15 LPA

Test Professional at Virtusa Consulting Services Pvt Ltd

Hyderabad / Secunderabad

8-13 Yrs

₹ 5-9 LPA

Senior Data Engineer at Resource Tree

4-8 Yrs

₹ 11-18 LPA

Senior Big Data Engineer at Info Services

3-6 Yrs

₹ 12-18 LPA

Pyspark Developer at Capgemini Technology Services India Limited

Pune

4-9 Yrs

₹ 6-14 LPA

Ipro at Wipro

Hyderabad / Secunderabad, Pune + 1

6-10 Yrs

₹ 12-22 LPA

Virtusa Consulting Services Bangalore / Bengaluru Office Location

View all
Bengaluru Office
Virtusa Consulting Services, RMZ Eco World, 3rd Floor, Sarjapur-Marathahalli Outer Ring Road, Deverabeesanahalli,Varthur Hobli Bengaluru
Karnataka 560103

Databricks + PySpark Professional

7-9 Yrs

Bangalore / Bengaluru

1d ago·via naukri.com

Senior Software Engineer

4-7 Yrs

₹ 15 - 15L/yr

Thane, Chennai

11hr ago·via naukri.com

ServiceNow Architect

13-20 Yrs

Hyderabad / Secunderabad, Chennai, Bangalore / Bengaluru

18hr ago·via naukri.com

Servicenow Architect

9-14 Yrs

Hyderabad / Secunderabad, Chennai, Bangalore / Bengaluru

1d ago·via naukri.com

Oracle DBA

6-11 Yrs

Bangalore / Bengaluru

1d ago·via naukri.com

ServiceNow Architect

12-16 Yrs

Bangalore / Bengaluru

1d ago·via naukri.com

Payments QA Professional

4-9 Yrs

Bangalore / Bengaluru

1d ago·via naukri.com

Python Robot QA Professional

6-8 Yrs

Chennai

1d ago·via naukri.com

Core Associate Architect

7-12 Yrs

Gurgaon / Gurugram

1d ago·via naukri.com

Automation Tester

8-13 Yrs

Bangalore / Bengaluru

1d ago·via naukri.com
write
Share an Interview