Data Engineer

PURPOSE OF THE POSITION

We are seeking a Data Engineer with a passion for building efficient, scalable, and reliable data pipelines. The ideal candidate will have strong expertise in SQL, ETL/ELT processes, and tools like dbt, Airflow, and Airbyte, along with experience in modern data warehousing solutions such as AWS Redshift and Google BigQuery. This role requires hands-on experience in designing and optimizing data pipelines to support business analytics and decision-making.

ORGANISATIONAL CONTEXT

CIMET currently provides end-to-end comparison and signup of Energy, Telco, Credit Cards, Personal loan, and residential solar plans through its online solution in a B2B and B2C environment. It presently caters to various Providers and has several White label partners and other co-branded partners. This portfolio will further expand over the next 18 months across financial services (loans and insurance products). The online comparison market is extremely crowded in Australia, with most comparators providing a light touch directory solution. CIMET saw this as an opportunity and developed a fully integrated online comparison and signup platform.

On 15 March 2022, CIMET decided to partner with iSelect who acquired a 49% stake in CIMET Holdings Going forward, CIMET will use this investment from iSelect to expand into new verticals including Home loans, Car loans, Life Insurance, Health Insurance, Pet Insurance among others. Additionally, CIMET has plans to expand into multiple geographies including India. CIMET today has around 130+ resources across Australia and India and will grow to over 200 over the next 2 years.

KEY ACCOUNTABILITIES

• Design, develop, and maintain ETL/ELT pipelines to support the ingestion, transformation, and loading of structured and unstructured data.
• Work with dbt (Data Build Tool) to transform data and implement data models in data warehouses.
• Orchestrate workflows and pipelines using Apache Airflow.
• Configure and manage Airbyte to facilitate seamless data extraction and integration across various sources.
• Optimize and maintain large-scale data warehouse solutions like AWS Redshift and Google BigQuery.
• Collaborate with data analysts, data scientists, and stakeholders to understand data requirements and deliver solutions.
• Develop e>icient and reusable code for data pipeline processes using SQL and Python.
• Monitor and troubleshoot data pipeline performance, ensuring data integrity and availability.
• Implement best practices for data security, governance, and compliance.
• Document data pipeline processes, data models, and workflows for easy collaboration and maintenance.

YOUR SKILLS AND EXPERIENCE

• Experience: 3+ years
• Proficiency in SQL for data transformation, analysis, and optimization.
• Hands-on experience with ETL/ELT tools like dbt, Airbyte, or similar.
• Strong knowledge of workflow orchestration tools like Apache Airflow.
• Experience in working with cloud-based data warehouses, such as AWS Redshift and Google BigQuery.
• Familiarity with data integration and migration concepts.
• Basic programming skills in Python for scripting and automation tasks.
• Understanding of data modeling techniques (e.g., star schema, snowflake schema).
• Strong problem-solving and debugging skills, with attention to detail.
• Knowledge of data governance and best practices for data pipeline performance optimization.

PREFERRED SKILLS

• Experience with additional data integration tools (e.g., Fivetran, Stitch).
• Familiarity with infrastructure-as-code (e.g., Terraform, CloudFormation).
• Exposure to real-time streaming tools like Kafka or Spark Streaming.
• Knowledge of data visualization tools (e.g., Tableau, Looker) for collaboration with BI teams.