Data Engineer

We are seeking a Data Engineer to join the Data Mining department. You will be responsible for designing and managing efficient data pipelines, optimizing databases, and enabling advanced analytics. This role is pivotal in ensuring data availability and reliability for lead generation, market research, and organizational decision-making.

Responsibilities

Data Pipeline Development:
Build and maintain scalable ETL pipelines to process and transform data from various sources.
Automate workflows for data extraction, validation, and delivery using Python and SQL.
Ensure pipelines are robust, efficient, and well-documented.
Database Management:
Manage and optimize PostgreSQL databases or similar systems.
Write and fine-tune complex SQL queries to support reporting and analytics needs.
Regularly monitor and enhance database performance and ensure data integrity.
Data Visualization and Reporting:
Design and maintain interactive dashboards in Power BI to support stakeholders.
Deliver actionable insights through advanced data visualizations and reporting frameworks.
Collaborate with teams to define metrics and align reporting with business objectives.
Data Quality and Governance:
Implement data quality checks to ensure accuracy, completeness, and consistency.
Maintain data compliance with industry standards and privacy regulations (e.g., GDPR, CCPA).
Develop processes to clean and standardize data before ingestion into systems.
Automation and Optimization:
Develop Python scripts to automate repetitive data management tasks.
Optimize data processing workflows to enhance operational efficiency.
Troubleshoot and resolve issues in existing automation tools and pipelines.
Collaboration and Support:
Work closely with data mining team, operations teams, and stakeholders to address data needs.
Integrate data from multiple platforms to enable seamless analytics.
Provide technical expertise on data architecture and engineering best practices.

Key Skills & Competencies
Expertise in Python for scripting and data automation.
Proficiency in SQL for database management and optimization.
Advanced Power BI skills for data visualization and dashboard creation.
Experience in designing and managing data pipelines and ETL workflows.
Knowledge of data governance, security, and compliance standards.
Strong problem-solving and critical-thinking skills.
Excellent communication and collaboration abilities for cross-functional work.