Azure Data Engineer

Azure Data Engineer

Work Location: Ahmedabad

Our Predictive Enterprise Capability is comprised of some of the most respected and well-known architects as well as brilliant new developers and designers. Together, our teams of professionals have delivered some of the most innovative and leading-edge implementations of Business Intelligence, Data Warehousing and Big Data solutions for the business-to-business as well as business-to-consumer space.As you will be joining a world-class team of consultants, excellent interpersonal and organization skills are essential. You need to have the ability to handle diverse situations, multiple projects and rapidly changing priorities. You need to be a self-starter who requires minimal supervision and one who is passionate about solving complex business problems. Since we are a Microsoft National Systems Integrator and have well established alliances with Tableau, Amazon (AWS), Hortonworks, SAP, Theobald; to name a few, in addition to working on projects close to home, you also have the ability to work with cross-functional/regional teams giving you flexibility to determine a work/life balance thats best suited for you. A Bachelors-Degree in Computer Science or related field is always preferred however those with equivalent industry experience are just as important to us.

Must Have Skills;

• Prior experience in ETL, data pipelines, data flow techniques using Azure Data Services
• Working experience in Python, Scala, Py Spark, Azure Data Factory, Azure Data Lake Gen2, Databricks, Azure Synapse and file formats like JSON & Parquet
• Experience in creating ADF Pipelines to source and process data sets.
• Experience in creating Databricks notebooks to cleanse, transform and enrich data sets.
• Good understanding about SQL, Databases, NO-SQL DBs, Data Warehouse, Hadoop and various data storage options on the cloud.
• Development experience in orchestration of pipelines
• Experience in deployment and monitoring techniques
• Working experience with Azure DevOps CI/CD pipelines to deploy Azure resources.
• Experience in handling operations/Integration with source repository
• Must have good knowledge on Datawarehouse concepts and Datawarehouse modelling

Good to have;

• Familiarity with DevOps, Agile Scrum methodologies and CI/CD
• Domain-driven development exposure
• Analytical / problem solving skills
• Strong communication skills
• Good experience with unit, integration and UAT support
• Able to design and code reusable components and functions
• Should able to review design, code & provide review comments with justification
• Zeal to learn new tool/technologies and adoption
• Power BI and Data Catalog experience

Responsibilities

• Analyze business problems and design technology solutions
• Ability to understand the data transformation requirements and implement the same in an optimum solution using Azure data services
• Capable to participate in customer discussion and suggest solutions as necessary
• Ability to carry out the data modeling activities and deliver the solution in agreement with baselined time & quality
• Stay focused on-task completion and delivery
• Supporting existing processes and implementing change requirements as part of a structured change control process
• Working in a project environment, communicating any identified risks and issues to the project manager accordingly and providing inputs to the change control process
• The consultant need to be hands-on and highly technical. Consultant will be in a customer facing role and accountable for the end-to-end technology delivery and deployment experience of data pipelines using Azure data services.
• Perform data cleansing, data validation, sanity checking, and unit testing associate with it
• Perform production cutover activities from a technical perspective