As a Senior Data Engineer , you will be responsible for designing, developing, and maintaining the data infrastructure that powers our advertising technology platform. You will work closely with cross-functional teams to build scalable data pipelines, optimize data workflows, and ensure the reliability and performance of our data systems.
Responsibilities:
Design, develop, and maintain ETL pipelines to collect, process, and store data from various sources.
Work with APIs, databases, and other data sources to integrate data into the data warehouse.
Implement and maintain data validation and monitoring processes to ensure data accuracy and reliability.
Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions.
Transform raw data into a form that is easily usable for analysis and business intelligence.
Maintain comprehensive documentation of data pipelines, processes, and systems.
Optimize data systems and pipelines for performance and scalability.
Requirements:
Bachelor s degree in Computer Science, Information Systems, or a related field.
Minimum of 3 years of experience in data engineering or a related role.
Proficiency in SQL and experience with at least one programming language such as Python.
Experience with cloud-based data platforms (AWS, Azure, GCP) and services such as AWS Redshift or Google BigQuery.
Knowledge of big data technologies like Hadoop, Spark, or Kafka is a plus.
Experience with ETL tools such as Apache Airflow.
Familiarity with data warehousing concepts and relational databases.
Monitor and troubleshoot data pipeline issues, performance bottlenecks, and data quality issues in production environments.
Implement data governance policies and procedures to ensure data integrity, security, and compliance with regulatory requirements.
Understanding of data modeling and database design principles.
Strong problem-solving and analytical skills.
Excellent communication and collaboration skills.
Ability to work independently and manage multiple tasks and projects simultaneously.