Azure Data Engineer - Big Data (3-6 yrs)

About Polestar Solutions :

Polestar is a leading AI & analytics solutions company serving Fortune 1000 companies, startups, and governments across various industries, including CPG & retail, manufacturing, and pharmaceuticals.

Headquartered in Texas, we have branches across Noida, Kolkata, Mumbai, Bangalore, and the UK.

Our comprehensive analytics foundation, data science, and AI initiatives have earned us recognition in multiple reports from Forrester, Red Herring, Deloitte, ISG, Everest Group, AIM Research, and more.

Job Title : Azure Data Engineer

Experience : 3 to 7 years

Location : Bangalore, India

About the Role :

We are seeking a dynamic and experienced Azure Data Engineer to design, implement, and maintain scalable data solutions using the Microsoft Azure stack.

The ideal candidate will have a strong understanding of data engineering principles and hands-on experience building and optimizing data pipelines, deploying Big Data and BI solutions, and contributing to the growth of robust and scalable data platforms.

You will play a key role in shaping our data infrastructure and enabling data-driven decision-making for our clients.

Key Responsibilities :

- Data Estate Management : Implement and manage a scalable data estate on Microsoft Azure, ensuring data quality, availability, and security.

- Data Pipeline Development : Design, develop, and maintain efficient and reusable data pipelines using Azure Data Factory, Azure Functions, and other relevant Azure services. Focus on automation, performance optimization, and error handling.

- Big Data and BI Solutions : Design and deploy Big Data and Business Intelligence solutions leveraging Azure Databricks, Azure Synapse Analytics, and other relevant technologies.

- Data Platform Development : Lead the development and deployment of new data platforms, with a strong emphasis on scalability, performance, and maintainability.

- Reusable Components : Create and maintain reusable components and frameworks to accelerate data platform development and ensure consistency across projects.

- Architecture and Data Modeling : Define high-level architecture and data models for optimal performance, data integrity, and scalability.

- Mentorship and Guidance : Guide and mentor junior development team members, promoting best practices in data engineering and fostering a culture of continuous improvement.

- UAT and Go-Live : Oversee User Acceptance Testing (UAT) and Go-Live processes for data platform projects, ensuring smooth and successful deployments.

- Collaboration and Communication : Actively participate in team meetings, client workshops, and technical discussions, providing valuable technical insights and contributing to effective communication.

- Performance Tuning and Optimization : Proactively identify and address performance bottlenecks in data pipelines and data platforms, optimizing for efficiency and cost-effectiveness.

- Security Best Practices : Implement and adhere to security best practices for data storage, processing, and access, ensuring compliance with relevant regulations and policies.

Requirements :

- Experience : 3+ years of proven experience in Data Warehousing, Big Data, or Cloud environments.

- SQL Expertise : Strong proficiency in SQL, including complex queries, stored procedures, and performance tuning.

- Data Concepts : Solid understanding of OLAP/OLTP concepts, data modeling techniques, and data warehousing principles.

- Azure Cloud Skills : Hands-on experience with core Azure data services, including :

1. Azure Data Factory (ADF)

2. Azure Functions

3. Azure SQL Database

4. Azure Databricks

5. Azure Data Lake Storage Gen2

6. Azure Synapse Analytics

- Scripting : Experience with scripting languages like PowerShell for automation and infrastructure management.

- Big Data Technologies : Familiarity with Big Data technologies like Spark and Hadoop is a plus.

- Agile Development : Experience working in an Agile development environment.

- Cloud AI/ML : Knowledge of cloud-based AI/ML platforms (Azure, AWS, GCP) and related infrastructure concepts is an advantage.

- Communication Skills : Excellent communication, collaboration, and problem-solving skills.

- Education : Bachelor's degree in Computer Science, Information Technology, or a related field preferred.

Bonus Points :

- Experience with data visualization tools like Power BI.

- Knowledge of DevOps practices and tools.

- Certifications in Azure data engineering