Data Engineer - Lead

Role:
Primary Skill(s): Lead Data Engineer
Optimize data pipelines for performance and efficiency.
Work with databricks clusters and configuration management tools.
Use appropriate tools in the cloud data lake development and deployment. Developing/implementing cloud infrastructure to support current and future business needs.
Provide technical expertise and ownership in the diagnosis and resolution of issues.
Ensure all cloud solutions exhibit a higher level of cost efficiency, performance, security, scalability, and reliability.
Manage cloud data lake development and deployment on AWS /Databricks.
Manage and create workspaces, configure cloud resources, view usage data, and manage account identities, settings, and subscriptions in Databricks

Roles and Responsibility
Technical Leadership: Demonstrate leadership, and ability to guide business and technology teams in adoption of best practices and standards
Design Development: Design, develop, and maintain robust, scalable, and high-performance data estate
Architecture: Architect and design robust data solutions that meet business requirements include scalability, performance, and security.
Quality: Ensure the quality of deliverables through rigorous reviews, and adherence to standards.
Agile Methodologies: Actively participate in agile processes, including planning, stand-ups, retrospectives, and backlog refinement.
Collaboration: Work closely with system architects, data engineers, data scientists, data analysts, cloud engineers and other business stakeholders to determine optimal solution architecture that is future-proof too.
Innovation: Stay updated with the latest industry trends and technologies, and drive continuous improvement initiatives within the development team.
Documentation: Create and maintain technical documentation, including design documents, and architectural user guides.

Skills Qualification
Bachelor s degree in information technology or a related field. Equivalent experience may be considered.
Overall experience of 8-12 years in Data Engineering
Ability to understand and utilize execution plans effectively.
Excellent communication and client interfacing skills, both verbal and written.
Excellent problem-solving and analytical skills.
Ability to work effectively in a fast-paced, agile, and high intensity - dynamic environment.