Senior Data Engineer

Senior Data Engineer

(6+ Years of Experience)

Position Overview:

We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate should have a strong foundation in managing engineering data, with extensive expertise in time series data, IoT data streams , and a deep understanding of modern data engineering principle and tools. This role will focus on designing, building, and utilizing data platforms while ensuring scalability, governance, and high-quality data processing capabilities.

Key Responsibilities:

1. Data Platform Development:
   • Design, build, and maintain data platforms, including databases, storage layers, data warehouses, data lakes , and data mesh architectures .
   • Create robust pipelines for data ingestion, transformation, and storage.
   • Built reusable components for data integration and processing in IoT and industrial domains.
2. IoT Data Engineering:
   • Handle and process IoT data streams using protocols and technologies like OPC UA , PI System, Modbus, and other industrial data courses
   • Work with time-series data and ensure efficient ingestion, processing, and storage.
3. Data Governance and Standardization:
   • Implement and maintain data governance strategies to ensure quality, integrity, and security of data.
   • Standardize data definitions, metadata management, and processes across the organization.
4. ETL and Data Pipeline Expertise:
   • Develop end-to-end ETL pipelines for large-scale data processing.
   • Optimize pipelines for efficiency, reliability, and scalability.

1. Cloud Platform Expertise:
   • Build deploy and manage data solutions on AWS and Azure cloud platform.
   • Leverage cloud native tools like AWS Glue Azure Data Factory , L ambda etc. for data engineering tasks.
2. Open-Source Tool Utilization:
   • Implement and manage pipelines and workflows using tools like Apache Air Flow , NiFi , Presto and Apache spark .
3. Data Processing and Analytics:
   • Develop and deploy solutions for batch and real-time data processing.
   • Collaborate with analytics and data science teams to build end-to-end data flows.

Required Skills and Experience:

1. Technical Expertise:
   • 8+ years of experience in data engineering with a strong focus on time-series and IoT data.
   • Proven expertise in databases, data storage solutions, and modern architectures like data lakes, data warehouses, and data mesh.
   • Hands-on experience with open-source tools like Apache Airflow, NiFi, Presto, and Apache Spark.
2. IoT Domain Knowledge:
   • Experience with industrial IoT protocols such as OPC UA, Modbus, PI Systems , etc.
   • Strong understanding of time-series databases (e. g. Influx DB, Timescale DB ) and data handling techniques.
3. Cloud Expertise:
   • Solid experience with AWS (e. g. , Redshift , S3 , Athena ) and Azure (e. g. , Data Lake , Synapse , Event Hubs ).
4. ETL and Pipeline Development:
   • Deep expertise in designing, developing, and optimizing ETL processes for large-scale data environments.
   • Hands-on experience with data transformation tools and scripting languages, (e. g. , Python, SQL ).
5. Data Governance & Standardization:
   • Proven experience in implementing robust data governance frameworks and enforcing data standards.
6. Cross-Platform Integration:
   • Experience in integrating data from diverse systems, including real-time IoT and batch processing architectures.
7. Soft Skills:
   • Excellent problem-solving and communication skills.
   • Ability to work collaboratively with data scientists, analytics, and business stakeholders.
   • Strong leadership skill to guide junior engineers and influence architecture decisions.

Preferred Qualifications:

1. Education Background: Bachelor degree in engineering (Computer Science, Information Technology, or a related field).
2. Familiarity with data Lakehouse architectures (e. g. , Delta Lake).
3. Experience integrating machine learning pipelines into data workflows.
4. Knowledge of CI/CD pipelines for data solutions.
5. Exposure to industrial automation systems and their data integrations.