Data Engineer

Rockwell Automation is a global technology leader focused on helping the world s manufacturers be more productive, sustainable, and agile. With more than 28,000 employees who make the world better every day, we know we have something special. Behind our customers - amazing companies that help feed the world, provide life-saving medicine on a global scale, and focus on clean water and green mobility - our people are energized problem solvers that take pride in how the work we do changes the world for the better.

We welcome all makers, forward thinkers, and problem solvers who are looking for a place to do their best work. And if that s you we would love to have you join us!

Job Description

Who we are!

We are a product and services technology company based in Bangalore, India. Our mission is to discover intelligent insights from enterprise data and deliver business value to our customers. We specialize in building innovative solutions in niche technology areas such as Big Data Analytics, Data Science, AI, IoT, Blockchain, and Cloud.

Visit our website for more details: www.knowledgelens.com

Job Description:

As a professional services provider, our people are our greatest asset. We are seeking the absolute best to join our world-class team. We value balanced individuals who are willing to go the extra mile to deliver exceptional results for our clients. At the same time, we look for individuals with strong character who complement their expertise and intelligence with personality and integrity.

If you believe you have the skills and qualities we seek, even if you don t meet every requirement listed in the job description, we encourage you to apply. We would rather have a conversation and determine if you are a good fit than never meet at all.

Consulting is an apprenticeship-based business, and many of the skills you need to master are acquired on the job.

Location: Bangalore, India

We are seeking exceptional candidates who are passionate about the exciting world of digital technologies, including the Internet of Things, Big Data, Advanced Analytics, Artificial Intelligence, and Machine Learning.

The ideal candidate will assume the role of a Data Engineer (Python, SQL, Databricks, Pandas, Docker, Kubernetes, Azure/AWS) in our Enterprise Technology Practice. They will oversee technical strategy and delivery for our IoT advisory and implementation projects throughout their entire lifecycle.

The candidate s responsibilities will include:

Data Pipeline Development:

• Build, deploy, and maintain scalable ETL pipelines on Databricks, ensuring high performance and reliability.
• Process and transform large datasets using Pandas, PySpark, or equivalent libraries.

Data Integration Transformation:

• Integrate data from multiple sources such as APIs, RDBMS, and file systems into data lakes or warehouses.
• Perform data cleaning, transformation, and enrichment to support analytics and business requirements.

Database Management:

• Write and optimize advanced SQL queries for complex data transformations and extractions.
• Design efficient database schemas and implement data partitioning for improved query performance.

Containerization Orchestration:

• Develop containerized workflows using Docker.
• Orchestrate applications using Kubernetes in production environments.

Cloud Platform Expertise:

• Leverage Azure (preferred) or AWS cloud services for data storage, processing, and orchestration.
• Utilize tools like Azure Data Factory, Databricks, or Glue for end-to-end data pipeline implementation.

CI/CD Implementation:

• Set up and maintain CI/CD pipelines for automated deployment and testing of data engineering solutions.
• Work with tools such as Azure DevOps, GitLab, or Jenkins for continuous integration.

Performance Optimization:

• Optimize workflows for scalability, speed, and cost-effectiveness.
• Monitor pipeline performance and troubleshoot production issues efficiently.

Code Quality Best Practices:

• Write clean, modular, and well-documented code adhering to best coding practices.
• Conduct code reviews to ensure high-quality deliverables.

Collaboration Communication:

• Work closely with data scientists, analysts, and business stakeholders to understand data requirements.
• Participate in agile workflows, sprint planning, and retrospectives.

Required Skills and Qualifications:

Technical Skills:

• Programming Languages: Advanced proficiency in Python, including libraries like Pandas.
• Big Data Tools: Hands-on experience with Databricks, PySpark, or other big data frameworks.
• Databases: Expertise in SQL, relational databases, and query optimization techniques.
• Containerization: Proficiency in Docker and Kubernetes for container orchestration.
• Cloud Platforms: Strong experience in Azure (preferred) or AWS, with knowledge of cloud-native data tools.
• CI/CD Pipelines: Experience with Azure DevOps, GitLab, or Jenkins for continuous integration and delivery.

Other Skills:

• Familiarity with data lake and data warehouse architectures.
• Knowledge of workflow orchestration tools like Airflow.
• Experience with version control systems like Git.

Experience:

• 2 to 4 years of experience in data engineering or a similar role.

Soft Skills:

• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Ability to prioritize and manage multiple tasks in a fast-paced environment.

Why Join Us

• Opportunity to work with the latest cloud and data engineering technologies like Databricks and Azure.
• Collaborative and inclusive work environment.
• Exposure to large-scale, real-time data processing.
• Emphasis on best coding practices and professional growth.
• Competitive compensation and learning opportunities.

Location - Electronic City, Bangalore

Work Mode - Hybrid (3 days in a week)

Rockwell Automation s hybrid policy aligns that employees are expected to work at a Rockwell location at least Mondays, Tuesdays, and Thursdays unless they have a business obligation out of the office.