Data Platform Engineer

Job Summary :

We are seeking a skilled and motivated Data Platform Engineer to join our team. The role focuses on DB integration, data pre-processing, data pipeline creation, and supporting AI engineering efforts. As a Data Platform Engineer, you will work with diverse data sources and platforms, ensuring robust, secure, and scalable data infrastructure and pipelines for analytics, reporting, and AI solutions.

Key Responsibilities :

Database Integration :

- Integrate various database systems to support data collection, storage, and retrieval processes.

- Optimize database architecture for performance and scalability.

Data Pre-Processing & Pipeline Creation :

- Develop, maintain, and optimize ETL processes to extract, transform, and load data efficiently.

- Build and manage data pipelines for AI and analytics solutions.

Data Lake Management :

- Design, implement, and maintain data lakes on Azure, AWS, GCP, or other cloud/on-premise environments.

Cloud Services Management :

- Deploy and manage cloud-native services for seamless data integration and processing.

- Leverage cloud platforms to create scalable, reliable, and cost-effective data solutions.

Data Security and Monitoring :

- Ensure compliance with security standards and implement robust data protection measures.

- Monitor system performance and ensure high availability and reliability of data platforms.

Data Analytics Support :

- Enable data-driven decision-making by providing clean, processed, and well-structured data for analytics teams.

- Support advanced analytics and machine learning pipelines with appropriate data infrastructure.

API Development :

- Design and develop APIs for secure and efficient data access and exchange.

Required Skills & Qualifications :

1. Technical Expertise :

- Experience in Big Data Engineering with a focus on data lakes (Azure, AWS, GCP, cloud-native, and on-premise setups).

- Strong understanding of ETL processes and pipeline creation.

- Proficiency in API development for data integration and interaction.

2. Cloud and Data Engineering :

- Hands-on experience with cloud platforms and services.

- Expertise in designing and managing secure and scalable cloud-based solutions.

3. Data Analytics and Integration :

- Proficient in integrating structured and unstructured data from multiple sources.

- Familiarity with - data security- measures and system monitoring practices.

4. Problem Solving and Collaboration :

- Strong analytical skills and the ability to troubleshoot complex data engineering issues.

- Experience in collaborating with AI engineers, data scientists, and stakeholders.

5. Other Skills :

- Experience with system monitoring tools and ensuring high availability.

- Familiarity with modern programming languages and frameworks for data engineering.

Preferred Qualifications :

- Certifications in cloud platforms (e.g., AWS Certified Data Analytics, Azure Data Engineer Associate).

- Familiarity with modern AI/ML frameworks and data workflows.

- 5+ years of hands-on experience in data engineering roles.