Pluang - Senior Data Engineer (3-5 yrs)

Job Description :

As a Data Platform Engineer at Pluang, you will play a key role in designing, building, and scaling our data infrastructure to support real-time analytics, big data processing, and machine learning deployments. You'll be responsible for creating near-real-time ETL pipelines, ensuring data consistency across our Data Lake and Warehouse, and implementing advanced analytics solutions using cutting-edge tools like Apache Spark, Kafka Streams, and Apache Flink.

Working closely with our Data Science team, you'll contribute to building a Machine Learning Platform and ensure the scalability, reliability, and availability of Pluang's data ecosystem. This position is ideal for a skilled data engineer who's ready to innovate and drive Pluang's data strategy forward in the fintech space.

Responsibilities :

- Contributing to the Design and Architecture of Pluang's Data Platform.

- Designing and implementing near real-time ETL pipelines for syncing data to Data Lake and Data Warehouse.

- Building real-time analytics solutions using frameworks like Kafka Streams and Apache Flink.

- Leveraging Big data technologies like Apache Spark and HDFS for developing data processing jobs.

- Working with the Data Science team to develop a Machine Learning Platform for deploying ML models.

- Implementing monitoring and alerting framework for the Data Platform.

- Ensuring the scalability and availability of the Data Platform.

Requirements :

- Bachelor's degree in Computer Science or related technical field.

- Strong knowledge of working with Apache Spark using Python or Scala programming languages.

- Proficient with Apache Kafka or other data streaming technologies.

- Strong analytical and quantitative problem-solving ability.

- Excellent communication skills including the ability to communicate with peers and management.

Preferred Qualifications :

- Knowledge of Google Bigquery Data Warehouse.

- Hands-on experience working with serverless technologies like AWS Lambda or GCP cloud functions.

- Experience in developing ETL pipelines using Apache Airflow.

- Fintech experience would be preferred.