Principal Software Engineer

Responsibilities:

• Lead the design, development, and maintenance of scalable and efficient data architectures that support the collection, storage, and analysis of large volumes of streaming data.
• Define and implement data modelling standards, ensuring consistency, integrity, and optimization of data structures using big data principles.
• Design and perform POC where needed and develop Data platform offering. 
• Collaborate with data engineers to build and optimize data pipelines for ingesting, processing, and transforming data from various sources.
• Identifies areas of improvements within the design and suggests solutions.
• Ensuring the efficient performance of data systems, including optimizing queries, improving data retrieval times, and ensuring minimal downtime.
• Evaluate and recommend new technologies, tools, and frameworks to enhance our data architecture and infrastructure in cloud native environment.
• Partner with business stakeholders to understand their data requirements and translate them into actionable solutions that drive business value.

Qualifications:

• Education qualification: B.E./B.Tech or equivalent degree in computers, IT or electronics from reputed collage.
• Should have 8 - 12 years of work experience in software development including developing distributed systems using big data technologies.
• 8 - 12 years of experience developing Applications using Core Java.
• 5 - 8 years of experience of working Big Data technologies including Realtime data processing applications along with ETL jobs for batch.
• Should have good experience and knowledge in the use and deployment of AWS services.
• Hands-on experience with designing and implementing real time-data pipelines, using integration tools like Apache Kafka using Kafka Streams and Spark Streaming applications.
• Hands-on experience with designing and implementing batch processing ETL/ELT jobs using Spark batch.
• Experience in NoSQL and SQL Databases including Cassandra and MySQL.
• Should have experience in performance optimization and products performance bench marking.
• Should have experience of deploying and maintaining applications on Kubernetes.
• Should have experience of monitoring tool kit like Prometheus and Grafana 
• Good to have experience of working with vault.
• Good to have experience of logging mechanism with EFK.  
• Proficiency in DevOps Tools and CICD tools (e.g. Azure DevOps, Chef, Puppet, Github)
• Also, he should be good in general programming/Algorithm/Data Structure

Required Skills:

Strong in Spark, Kafka, Kafka Streams, Spark streaming, MySQL, Cassandra, AWS, Kubernetes, + Java + Strong in Algorithm & Data Structures +  Role & responsibilities