Data Engineer

Role & responsibilities

Job Title: Tech Lead (MS Fabric, Data Pipelines, Data Design, SQL) & Developers
Location: Bangalore, Hyderabad
Experience: 4 12 Years

Why We Are Hiring:
Sonata Software is seeking an experienced Tech Lead to drive the design, development, and implementation of cutting-edge data solutions leveraging MS Fabric, data pipelines, and advanced data integration techniques. The ideal candidate will have expertise in data injection, notebook code (EDI), and data flow management. This role requires an individual who can optimize SQL queries, handle various data formats (including EDI, PDF, CSV, Excel, Web), and convert them into a Lakehouse architecture for seamless data conversion and integration. The semantic level of data design will also be a key focus, enabling enhanced analytics and decision-making. As a Tech Lead, you will collaborate with cross-functional teams to deliver high-quality data solutions.

Key Responsibilities:

• Lead the design and implementation of end-to-end data solutions using MS Fabric and data pipelines, ensuring scalability and performance.
• Optimize SQL queries and data models for better performance and resource management.
• Manage and mentor a team of data engineers, providing guidance on best practices in data design, data flow management, and lakehouse architecture.
• Design and implement data injection processes for various file types such as EDI, PDF, CSV, Excel, and Web, converting them into Lakehouse formats for further processing and analytics.
• Develop and optimize notebook code for EDI (Electronic Data Interchange) and other integration tasks, ensuring high efficiency in the process.
• Collaborate with business stakeholders to understand data requirements and build effective solutions that meet those needs.
• Oversee semantic-level data design, ensuring the accurate representation of business processes and enabling rich, actionable analytics.
• Design and implement data conversion pipelines to transform different file formats (e.g., EDI, PDF, CSV, Excel) into a consistent Lakehouse architecture.
• Work closely with cross-functional teams to design and optimize data flow and ensure efficient data integration across platforms.
• Provide guidance on best practices in data governance and data security, ensuring compliance with company policies and industry standards.

Mandatory Skills:

• Expertise in MS Fabric and hands-on experience with data pipelines for large-scale data processing and integration.
• Strong proficiency in SQL for query optimization and performance tuning.
• In-depth experience in data design and architecture, especially with lakehouse solutions.
• Experience with data injection, integrating diverse data formats (e.g., EDI, PDF, CSV, Excel, Web) into a unified data lakehouse architecture.
• Experience with notebook code (e.g., for EDI) and knowledge of data flow management.
• Solid understanding of semantic-level data design for enabling analytics and reporting.
• Familiarity with ETL tools for data processing and transformation.

Desirable Skills:

• Experience with cloud platforms (e.g., Azure, AWS) and their respective data solutions.
• Familiarity with data warehousing concepts and data management best practices.
• Strong analytical and problem-solving skills, with the ability to troubleshoot and resolve complex data integration issues.
• Experience in managing large datasets and optimizing data systems for high performance and scalability.

Why Join Us?
At Sonata Software, we are committed to fostering an innovative, collaborative work environment that empowers our team members to grow their skills while making a meaningful impact on the business. As a leader in data engineering and cloud technology, you'll be working on cutting-edge solutions to enable data-driven insights and strategies. Join us in shaping the future of data and analytics in an ever-evolving tech landscape.