Associate Principal Engineer (Big Data)

👋🏼We're Nagarro.

We are a Digital Product Engineering company that is scaling in a big way! We build products, services, and experiences that inspire, excite, and delight. We work at scale across all devices and digital mediums, and our people exist everywhere in the world (18000 experts across 36 countries, to be exact). Our work culture is dynamic and non-hierarchical. We are looking for great new colleagues. That is where you come in!

REQUIREMENTS:

• Azure DataBricks Architect
• Proven experience in designing and implementing complex data solutions aligned with business objectives.
• Expertise in data modelling, integration, security, and governance
• Hands-on experience with guiding the virtual data model definition, defining Data Virtualisation architecture and deployment with focus on Azure, Databricks, PySpark technologies.
• Prior experience with establishing best practices for business optimisations.
• Experience with relational and non-relational data stores (Hadoop, SQL, Mongo DB), ETL or ELT tools (SSIS, Informatica, Matillion, DBT), DevOps, Data Lake and Data Fabric concepts
• In-depth experience with data governance, data integration and related technologies.
• Proficiency in a variety of database technologies, both relational and non-relational.
• Knowledge of cloud-based data solutions (e.g. Azure).

RESPONSIBILITIES:

• Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure Databricks.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Hadoop ecosystem components such as Hive.
• Implement real-time event-driven architecture using Kafka for streaming data ingestion and integration with ADF.
• Ensure high availability of the system by implementing monitoring tools like Prometheus/Grafana/Zabbix.