Data Engineer

Role: Data Engineer

Location: Bangalore

Job Title: Data Engineer

Job Summary: As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure to support our data-driven initiatives. Leveraging your expertise in data engineering and software development, you will collaborate with cross-functional teams to build robust data solutions that enable efficient data processing, storage, and analysis. Your role will involve optimizing data workflows, ensuring data quality and integrity, and staying abreast of emerging technologies to continuously improve our data infrastructure.

Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines to ingest, process, and store large volumes of structured and unstructured data.
• Collaborate with data scientists, analysts, and other data engineers to understand data requirements and implement solutions that meet business needs.
• Optimize data storage solutions, including relational databases, data lakes, and data warehouses.
• Implement data governance practices to ensure data quality, consistency, and integrity across various data sources and pipelines.
• Develop monitoring and alerting systems to proactively identify and address issues related to data pipeline performance, reliability, and security.
• Automate repetitive tasks and streamline data workflows to improve efficiency and productivity.
• Stay updated on industry trends and best practices in data engineering and recommend innovative solutions to enhance our data infrastructure.
• Collaborate within the team to understand requirements and deliver high-quality solutions on time.
• Participate in code reviews, testing, and deployment activities to ensure the reliability, scalability, and maintainability of data systems.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years of experience in data engineering, with a proven track record of designing and implementing complex data solutions.
• Proficiency in programming languages such as Python, Java, or Scala, with experience in building and maintaining production-grade software applications.
• Strong expertise in data processing frameworks and technologies, such as Apache Spark, Apache Flink, Hadoop, or similar.
• Experience working with distributed computing systems and cloud-based data platforms, such as AWS, Google Cloud Platform, or Azure.
• Solid understanding of database systems, SQL, and data modeling principles.
• Familiarity with containerization technologies (e.g., Docker, Kubernetes) and microservices architecture.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Excellent problem-solving skills and attention to detail, with the ability to troubleshoot and debug complex data issues.
• Strong communication and interpersonal skills, with the ability to collaborate effectively in a fast-paced, dynamic environment.

Preferred Qualifications:

• Experience with real-time streaming data processing frameworks, such as Apache Kafka, Apache Flink, or similar.
• Certification in cloud computing (e.g., AWS Certified Solutions Architect, Google Cloud Professional Data Engineer).
• Experience with agile development methodologies and tools (e.g., Jira, Scrum).
• Contributions to open-source projects or participation in data engineering communities.

Behavioral Skills:

• Very good communication skills
• Should be very interactive and top on deliverables being working from the client environmentRole & responsibilities

Preferred candidate profile

Perks and benefits