Home
Communities
Companies
- Companies
  
  Discover best places to work
- Compare Companies
  
  Compare & find best workplace
- Add Office Photos
  
  Bring your workplace to life
- Add Company Benefits
  
  Highlight your company's perks
Reviews
- Company reviews
  
  Read reviews for 6L+ companies
- Write a review
  
  Rate your former or current company
Salaries
- Browse salaries
  
  Discover salaries for 6L+ companies
- Salary calculator
  
  Calculate your take home salary
- Are you paid fairly?
  
  Check your market value
- Share your salary
  
  Help other jobseekers
Interviews
- Company interviews
  
  Read interviews for 40K+ companies
- Campus placements
  
  Interviews questions for 1K+ colleges
- Share interview questions
  
  Contribute your interview questions
Jobs
Awards

WINNERS AWAITED!
- ABECA 2025
  
  WINNERS AWAITED!
  
  AmbitionBox Employee Choice Awards - 4th Edition
- ABECA 2024
  
  AmbitionBox Employee Choice Awards - 3rd Edition
- AmbitionBox Best Places to Work 2022
  
  2nd Edition
- AmbitionBox Best Places to Work 2021
  
  1st Edition

Add office photos

Employer? Claim Account for FREE

WN Infotech

Compare

No reviews yet

379 WN Infotech Jobs

Data Pipeline Developer - Cloud Infrastructure

WN Infotech

2-5 years

Noida, Pune, Gurgaon / Gurugram + 1 more

1 vacancy

WN Infotech

posted 5hr ago

Job Role Insights

Key skills for the job

Python SQL Automation Testing Information Technology Clinical Data Management Agile Coaching

+ 3 more

Job Description

Job Summary

We are seeking a skilled and motivated Data Pipeline Developer with expertise in Cloud Infrastructure to join our team. The ideal candidate will be responsible for designing, developing, and managing scalable data pipelines on cloud platforms. You will collaborate with data scientists, engineers, and analysts to ensure that data flows seamlessly across various systems and is efficiently processed for analysis and reporting. As part of a dynamic and innovative team, you will have the opportunity to leverage cutting-edge cloud technologies to drive business insights and data-driven decisions.

Your Role and Responsibilities

Data Pipeline Development: Design, build, and maintain robust and scalable data pipelines to integrate and process large volumes of data from multiple sources into cloud-based systems.

Cloud Platform Expertise: Utilize cloud services (AWS, Google Cloud, Azure) to architect and deploy solutions that are cost-effective, secure, and high-performing.

Automation: Automate repetitive tasks, optimize data workflows, and improve the efficiency of data pipelines.

Data Management: Ensure the integrity, consistency, and security of data as it is moved through various stages of the pipeline.

Collaborate: Work closely with data scientists, data analysts, and DevOps teams to understand data needs and ensure pipelines meet business requirements.

Performance Optimization: Monitor pipeline performance, identify bottlenecks, and implement optimization strategies.

Documentation: Provide detailed technical documentation for the data pipeline architecture and processes to ensure clear communication across teams.

Testing & Quality Assurance: Develop and execute test cases for data pipeline components and ensure high-quality standards in the final product.

Troubleshooting: Identify and resolve issues within the data pipeline, addressing performance problems and any data discrepancies promptly.

Required Technical and Professional Expertise

Cloud Technologies: Proficient in at least one cloud platform (AWS, Google Cloud, Azure) and experience with cloud-native tools and services like AWS Lambda, Google Cloud Functions, AWS S3, BigQuery, etc.

Data Pipeline Development: Strong hands-on experience with designing and building ETL/ELT pipelines, data integration, and batch processing.

Programming Languages: Expertise in programming languages such as Python, Java, or Scala for data pipeline development.

Data Storage Solutions: Knowledge of working with various data storage solutions like relational databases (SQL), NoSQL databases, and cloud storage systems (S3, GCS).

Data Orchestration Tools: Familiarity with data orchestration tools like Apache Airflow, AWS Step Functions, or Google Cloud Composer.

Version Control Systems: Proficient in using version control tools such as Git for collaborative code development.

Data Security & Privacy: Understanding of data security best practices and experience in ensuring data privacy compliance.

Problem-Solving: Excellent troubleshooting skills and ability to address issues in real-time in a fast-paced environment.

Collaboration: Strong communication and teamwork skills, with the ability to work cross-functionally in an agile environment.

Qualification & Experience Required

Educational Qualification:

Bachelor s or Master s degree in Computer Science, Information Technology, Engineering, or a related field.

Experience:

A minimum of 2 to 5 years of experience in developing and managing data pipelines within cloud environments.
Proven experience in handling large datasets, implementing data workflows, and working with cloud-based infrastructure.
Hands-on experience with data integration tools, APIs, and third-party integrations.

Certifications (Preferred):

Cloud certifications such as AWS Certified Solutions Architect, Google Cloud Professional Data Engineer, or Microsoft Azure Data Engineer are highly desirable.

Other Skills:

Strong understanding of data warehousing concepts and data modeling.
Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes is a plus.

Employment Type: Full Time, Permanent

Read full job description