Data Engineer

Job Purpose and Impact

As a Data Engineer at Cargill you work across the full stack to design, develop and operate high performance and data centric solutions using our comprehensive and modern data capabilities and platforms. You will play a critical role in enabling analytical insights and process efficiencies for Cargills diverse and complex business environments. You will work in a small team that shares your passion for building innovative, resilient, and high quality solutions while sharing, learning and growing together.

Key Accountabilities

Collaborate with business stakeholders, product owners and across your team on product or solution designs.

Develop robust, scalable and sustainable data products or solutions utilizing cloud based technologies.

Provide moderately complex technical support through all phases of product or solution life cycle.

Perform data analysis, handle data modeling and configure and develop data pipelines to move and optimize data assets.

Build moderately complex prototypes to test new concepts and provide ideas on reusable frameworks, components and data products or solutions and help promote adoption of new technologies.

Independently solve moderately complex issues with minimal supervision, while escalating more complex issues to appropriate staff.

Other duties as assigned

Qualifications

MINIMUM QUALIFICATIONS

Bachelors degree in a related field or equivalent experience

Minimum of two years of related work experience

Other minimum qualifications may apply

PREFERRED QUALIFCATIONS

Experience developing modern data architectures, including data warehouses, data lakes, data meshes, hubs and associated capabilities including ingestion, governance, modeling, observability and more.

Experience with data collection and ingestion capabilities, including AWS Glue, Kafka Connect and others.

Experience with data storage and management of large, heterogenous datasets, including formats, structures, and cataloging with such tools as Iceberg, Parquet, Avro, ORC, S3, HFDS, HIVE, Kudu or others.

Experience with transformation and modeling tools, including SQL based transformation frameworks,

orchestration and quality frameworks including dbt, Apache Nifi, Talend, AWS Glue, Airflow, Dagster, Great Expectations, Oozie and others

Experience working in Big Data environments including tools such as Hadoop and Spark

Experience working in Cloud Platforms including AWS, GCP or Azure

Experience of streaming and stream integration or middleware platforms, tools, and architectures such as Kafka, Flink, JMS, or Kinesis.

Strong programming knowledge of SQL, Python, R, Java, Scala or equivalent

Proficiency in engineering tooling including docker, git, and container orchestration services

Strong experience of working in devops models with demonstratable understanding of associated best practices for code management, continuous integration, and deployment strategies.

Experience and knowledge of data governance considerations including quality, privacy, security associated implications for data product development and consumption