Senior Data Engineer (Azure Stack)

As a Senior Data Engineer at smartSense, you will be a key person in designing, building, and optimizing data pipelines and solutions using the Azure stack. Your role will involve collaborating with cross-functional teams such as ML engineers, data analysts, other data engineers, the pre-sales team, and stakeholders to ensure our data infrastructure supports business goals and drives data-driven decision-making. You will also mentor team members, manage project timelines, and effectively communicate with stakeholders or customers, foster a collaborative environment by actively engaging with team members, promoting teamwork, and ensuring alignment across the team to achieve common goals.

Key Responsibilities:

• Design Development: Architect, design, and develop scalable and efficient data pipelines and ETL processes using Azure technologies.
• Data Streaming: Implement and manage data streaming solutions for real-time or batch data processing.
• Data Migration: Oversee data migration projects, ensuring seamless transitions of the data and maintaining data integrity.
• ETL ELT: Build and optimize ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes as needed.
• Data Processing: Process and analyze large volumes of data to extract valuable insights.
• Data Warehousing: Design and manage data warehousing solutions to support business intelligence and analytics.
• Mentorship: Mentor and guide junior team members, providing support and fostering their and your professional growth.
• Stakeholder Communication: Communicate effectively with stakeholders to gather requirements, provide updates, and ensure alignment with business objectives
• Time Estimation: Estimate time and resources required for project completion and manage timelines effectively
• Self-Starter: Proactively identify areas for improvement, take initiative on projects, and work independently with minimal supervision

Technical Skills:

• Programming Languages: Proficiency in Python, C# .NET, or Java.
• SQL Scripting: Advanced skills in SQL query writing, scripting, and PL/SQL or T-SQL.
• Databases: Experience with MS SQL Server, Azure SQL DB, and Azure Cosmos DB for NoSQL/MongoDB highly desirable.
• Azure Services: Expertise in Azure Data Factory, Azure Event Hub, Azure Databricks, Azure Storage, Other Azure services. Nice to have Azure Synapse Analytics and Azure Functions.
• Data Visualization: Will be added as an advantage if you have knowledge of Power BI for data visualization and reporting.
• Big Data Technologies: Experience with Apache Spark and Snowflake for big data processing and warehousing.

Qualifications:

• Bachelor s degree in Computer Science, Engineering, or a related field (Master s degree preferred).
• 5+ years of experience as a Data Engineer, with a focus on Azure technologies.
• Highly experienced in designing and implementing data pipelines, data migration, and data warehousing solutions.
• Have the ability to work effectively under pressure.
• Excellent communication and interpersonal skills.