Data Engineer - Lead

Lead Data Engineer (12+ Years of Experience)

Position Overview: We are seeking an exceptional Lead Data Engineer to drive and oversee our data engineering initiatives. The ideal candidate must have extensive experience in architecting enterprise-scale data solutions, with deep expertise in time series data, IoT data streams , and demonstrated success in leading complex data engineering projects. This role will focus on technical leadership, architectural decisions, and strategic planning while mentoring teams and establishing best practices.

Key Responsibilities:

1. Technical Leadership & Strategy:
   • Define and drive the technical vision for data platforms, including databases, storage layers, data warehouses, data lakes , and data mesh architectures
   • Establish architectural standards and best practices across the organization
   • Lead proof-of-concept initiatives for emerging technologies and drive innovation
   • Guide multiple teams in designing and implementing complex data solutions
2. Data Platform Architecture:
   • Architect enterprise-scale data platforms that support both batch and real-time processing
   • Design cross-platform data integration strategies
   • Establish patterns for building reusable components in IoT and industrial domains
   • Define scalability and performance standards for the organization
3. IoT Data Architecture:
   • Design enterprise architecture for IoT data streams processing using protocols like OPC UA , PI System, Modbus , and other industrial data sources
   • Architect solutions for complex time-series data processing at scale
   • Define standards for IoT data integration and processing
4. Enterprise Data Governance:
   • Develop and oversee organization-wide data governance frameworks
   • Establish data quality standards and monitoring strategies
   • Define security and compliance architectures for sensitive data
   • Lead data standardization initiatives across the enterprise
5. Team Leadership:
   • Build and mentor high-performing data engineering teams
   • Provide technical guidance and career development for team members
   • Collaborate with other technical leaders to align data strategies
   • Drive hiring decisions and team structure planning
6. Technology Strategy:
   • Lead cloud platform strategy across AWS and Azure
   • Evaluate and recommend new technologies and tools
   • Define standards for using tools like Apache Airflow , NiFi , Presto , and Apache Spark
   • Drive adoption of modern data architectures and practices

Required Skills and Experience:

1. Technical Leadership:
   • 12+ years of progressive experience in data engineering
   • Proven track record of leading large-scale data initiatives
   • Experience managing budgets and resource allocation
   • History of successful architectural decisions
2. Enterprise Architecture:
   • Expert knowledge of enterprise data architecture patterns
   • Deep expertise in data lakes, data warehouses, and data mesh implementations
   • Proven experience with large-scale time-series and IoT architectures
   • Track record of designing scalable, maintainable solutions
3. Technical Expertise:
   • Advanced knowledge of open-source tools and enterprise data platforms
   • Expert-level understanding of industrial IoT protocols
   • Deep expertise in cloud platforms ( AWS and Azure )
   • Advanced knowledge of data governance frameworks
4. Leadership Skills:
   • Demonstrated success in building and leading engineering teams
   • Strong executive communication abilities
   • Experience with strategic planning and roadmap development
   • Track record of mentoring and developing technical leaders

Preferred Qualifications:

1. Education: Masters degree in Computer Science, Engineering, or related field
2. Experience with enterprise architecture frameworks
3. Track record of publishing technical content or speaking at conferences
4. Experience with digital transformation initiatives
5. Professional certifications in cloud platforms or data technologies
6. Experience with enterprise-scale machine learning operations
7. Background in industrial automation systems architecture