Data Platform Engineer

Job Summary
We are seeking a skilled and motivated Data Platform Engineer to join our team. The role focuses on DB integration, data pre-processing, data pipeline creation, and supporting AI engineering efforts. As a Data Platform Engineer, you will work with diverse data sources and platforms, ensuring robust, secure, and scalable data infrastructure and pipelines for analytics, reporting, and AI solutions.

Key Responsibilities

- Database Integration:
- Integrate various database systems to support data collection, storage, and retrieval processes.
- Optimize database architecture for performance and scalability.

- Data Pre-Processing & Pipeline Creation:
- Develop, maintain, and optimize ETL processes to extract, transform, and load data efficiently.
- Build and manage data pipelines for AI and analytics solutions.

- Data Lake Management:
- Design, implement, and maintain data lakes on Azure, AWS, GCP, or other cloud/on-premise environments.

-Cloud Services Management:
- Deploy and manage cloud-native services for seamless data integration and processing.
- Leverage cloud platforms to create scalable, reliable, and cost-effective data solutions.

- Data Security and Monitoring:
- Ensure compliance with security standards and implement robust data protection measures.
- Monitor system performance and ensure high availability and reliability of data platforms.

- Data Analytics Support:
- Enable data-driven decision-making by providing clean, processed, and well-structured data for analytics teams.
- Support advanced analytics and machine learning pipelines with appropriate data infrastructure.

- API Development:
- Design and develop APIs for secure and efficient data access and exchange.

Required Skills & Qualifications

1. Technical Expertise:
- Experience in Big Data Engineering with a focus on data lakes (Azure, AWS, GCP, cloud-native, and on-premise setups).
- Strong understanding of ETL processes and pipeline creation.
- Proficiency in API development for data integration and interaction.

2. Cloud and Data Engineering:
- Hands-on experience with cloud platforms and services.
- Expertise in designing and managing secure and scalable cloud-based solutions.

3. Data Analytics and Integration:
- Proficient in integrating structured and unstructured data from multiple sources.
- Familiarity with data security measures and system monitoring practices.

4. Problem Solving and Collaboration:
- Strong analytical skills and the ability to troubleshoot complex data engineering issues.
- Experience in collaborating with AI engineers, data scientists, and stakeholders.

5. Other Skills:
- Experience with system monitoring tools and ensuring high availability.
- Familiarity with modern programming languages and frameworks for data engineering.

Preferred Qualifications
- Certifications in cloud platforms (e.g., AWS Certified Data Analytics, Azure Data Engineer Associate).
- Familiarity with modern AI/ML frameworks and data workflows.
- 5+ years of hands-on experience in data engineering roles.