Azure Synapse + Spark + Azure cloud Professional

Years of experience

8+ years

Work Location

Any NTT location

Mandatory Skills

Azure Synapse + Spark + Azure cloud

Role - Azure Synapse Data Engineer
Responsibilities -
1. Understand business requirements and designing solutions that leverage the capabilities of the SAS.
2. Analysis, Develop and implement the components of the Azure Synapse. This includes data models, building designing, creating reports and documentation, connectivity and handling large data ingestions 3. Participating in technical and design decisions for Synapse applications. Provide technical insights and guidance to teams throughout the software development lifecycle
4. Design, develop, and maintain data pipelines to extract, transform, and load (ETL) data from various sources into our integrated data platform.
5. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver data-driven solutions.
6. Monitor and troubleshoot data processing issues, identifying and resolving root causes promptly.
7. Utilize GitHub for version control, continuous integration, and deployment of data-related artifacts.
8. Collaborate with the data architecture team to design and develop solutions for integration with other platforms (snowflake, informatica, etc)
9. Stay up-to-date with emerging data engineering trends, technologies, and best practices, and propose their integration when relevant.
10. Stay up-to-date with emerging data engineering trends, technologies, and best practices, and propose their integration when relevant.
11. Testing the solution to ensure that it meets the requirements and functions as expected. This includes unit testing, participation in integration testing user acceptance testing
12. Coordinating with stakeholders, documenting the deployment process, and ensuring that the solution is deployed in a secure and reliable manner
Requirement -
1. Bachelor's degree (Computer Science/Information Technology/Electronics Communication/ Information Science/Telecommunications)
2. 10+ years of work experience in IT Industry and strong knowledge in implementing/designing solutions using s/w application technologies 3. 5+ years of relevant experience in Reporting technologies
4. Design and implement data models within Azure Synapse efficiently, normalization and denormalization strategies depending on usage patterns
5. Understanding of data flows, business glossary/data definitions and business rules related to data management
6. Data ingestion and transformation pipelines using Azure Data Factory or native Synapse tools, includes data extraction, cleaning, filtering, and loading data
7. Data assessment reports based on analysis and presenting to management
8. T-SQL queries to analyze data within SQL pools, optimizing query performance by utilizing indexes, partitioning, and other optimization techniques
9. Knowledge of Data Quality processes, Master Data Management, and Data ownership
10. Experience with databases, data modeling analytics tools and methodologies, and ETL tools is a plus
11. Good communication skills and the flexibility to respond to different programmer demands
12. Good understanding of data and business analysis processes and the ability to explain what they mean for our clients
Skills -
Mandatory -
- Synapse - Scala
- Spark
- Azure Cloud
Desirable
- Azure DevOps, Python
- Insurance Domain Knowledge