Associate Staff Engineer (Big Data)

👋🏼We're Nagarro.

We are a Digital Product Engineering company that is scaling in a big way! We build products, services, and experiences that inspire, excite, and delight. We work at scale across all devices and digital mediums, and our people exist everywhere in the world (18000 experts across 36 countries, to be exact). Our work culture is dynamic and non-hierarchical. We are looking for great new colleagues. That is where you come in!

REQUIREMENTS:

• Proficient in building ETL pipelines using Apache Spark.
• Strong understanding of data modelling, data partitioning, and schema management.
• Strong coding skills in Python and SQL for data processing and transformation.
• Proficiency in using either AWS (Glue, S3, EMR, DMS) or Azure (ADF, ADLS, Databricks) for data engineering tasks.
• Strong problem-solving and communication skills.
• Develop and maintain a comprehensive understanding of our data pipeline and data architecture.
• Understanding of Databricks or Snowflake or Informatica
• Knowledge of data cataloging tools like Glue Data Catalog or Purview.
• Experience with CI/CD tools (e.g., Jenkins, GitHub Actions) for deploying and maintaining data workflows.
• Strong skills in debugging pipeline issues and familiarity with monitoring tools like CloudWatch, Datadog, or similar.
• Familiarity with workflow orchestration tools like Apache Airflow or Dagster or similar.
• Exposure to processing streaming data using tools like Spark Streaming, Kafka Streams, or AWS Kinesis.
• Familiarity with data quality tools or frameworks, such as Great Expectations or Deequ, to ensure data reliability.

RESPONSIBILITIES:

• Understanding the clients business use cases and technical requirements and be able to convert them into technical design which elegantly meets the requirements.
• Mapping decisions with requirements and be able to translate the same to developers.
• Identifying different solutions and being able to narrow down the best option that meets the client’s requirements.
• Defining guidelines and benchmarks for NFR considerations during project implementation
• Writing and reviewing design document explaining overall architecture, framework, and high-level design of the application for the developers
• Reviewing architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed.
• Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it
• Understanding and relating technology integration scenarios and applying these learnings in projects
• Resolving issues that are raised during code/review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken.
• Carrying out POCs to make sure that suggested design/technologies meet the requirements.