AmbitionBox
Search
Jobs
- Reviews
- Salaries
- Interview Questions
- About Company
- Benefits
- Jobs
- Office Photos
- Community
Add office photos
All Filters
82 Natobotics Jobs
Python + PySpark Developer
Natobotics
posted 3d ago
Job Role Insights
Key skills for the job
Job Description
About the Role:
We are looking for a skilled Python + PySpark Developer to join our team and help design, develop, and optimize data processing workflows. You will work on large-scale distributed data processing systems using Apache Spark and PySpark , processing vast amounts of data efficiently. This is an excellent opportunity for individuals passionate about big data and advanced analytics.
Key Responsibilities: - Design and implement scalable data processing pipelines using PySpark for batch and real-time data processing.
- Develop Python scripts and libraries to integrate, cleanse, and process data from various data sources (databases, APIs, etc.).
- Optimize PySpark jobs for performance and resource efficiency.
- Collaborate with data engineers and analysts to define data transformation and aggregation requirements.
- Analyze and troubleshoot issues in large-scale distributed data systems.
- Develop custom functions for data manipulation and aggregation using Spark DataFrame API and RDD API .
- Perform data analysis and create insightful visualizations for internal stakeholders.
- Write unit tests, document code, and maintain code quality through peer reviews and best practices.
- Monitor and optimize the performance of Apache Spark clusters.
- Work with cloud platforms such as AWS , Azure , or GCP to deploy Spark jobs and other data processing services.
- Strong experience with Python programming language and libraries such as Pandas , NumPy , Pytest , etc.
- Hands-on experience with Apache Spark and PySpark for distributed data processing.
- Familiarity with data formats such as CSV , JSON , Parquet , Avro , and other big data file formats.
- Solid understanding of Hadoop ecosystem components (HDFS, YARN, etc.).
- Proficiency with data wrangling, transformation, and aggregation.
- Experience with performance tuning, resource management, and debugging Spark applications.
- Understanding of cloud-based data storage solutions (S3, GCS, etc.) and distributed file systems.
- Familiarity with data warehousing concepts and tools.
- Good knowledge of SQL and relational databases (e.g., PostgreSQL, MySQL).
- Strong problem-solving skills and ability to work with large datasets.
- Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
- Bachelor s or Master s degree in Computer Science, Engineering, or a related field.
- Experience with additional big data frameworks like Flink , Kafka , Hive , etc.
- Knowledge of Docker and containerization for deployment.
- Experience with ETL tools like Airflow or Luigi .
- Familiarity with machine learning concepts and frameworks (e.g., MLlib , Scikit-learn ).
Employment Type: Full Time, Permanent
Read full job description What people at Natobotics are saying
What Natobotics employees are saying about work life
based on 5 employees
Day Shift
Natobotics Benefits
View more benefits
Compare Natobotics with
TCS
Compare
Accenture
Compare
Wipro
Compare
Cognizant
Compare
Capgemini
Compare
HDFC Bank
Compare
Infosys
Compare
ICICI Bank
Compare
HCLTech
Compare
Tech Mahindra
Compare
Genpact
Compare
Teleperformance
Compare
Concentrix Corporation
Compare
Axis Bank
Compare
Amazon
Compare
Jio
Compare
Reliance Retail
Compare
iEnergizer
Compare
IBM
Compare
LTIMindtree
Compare
Similar Jobs for you
Java developer with springboot and microservices
7-8 Yrs
Bangalore / Bengaluru
Share an Interview