Commdel - Data Engineer - ETL (8-12 yrs)

Responsibilities :

- Data pipelines : Create data pipelines to pull data from multiple sources and prepare it for analytics.

- Data quality : Identify and resolve data quality issues, performance bottlenecks, and scalability concerns.

- Data modeling : Prepare data for predictive and prescriptive modeling.

- Data visualization : Create visualizations like charts and graphs to communicate insights.

- Data warehousing : Build and work with data warehouses to aggregate unstructured data.

- Data security : Ensure data security and privacy.

- Data architecture : Design and construct architectures, and define data flows.

- Data systems : Design, build, and maintain data systems.

- Data collection : Develop processes to collect data from various sources.

Requirements :

Required Experience : 7+ years of experience in data engineering.

Required Skills :

- Data Ingestion : Expertise in data ingestion tools and techniques (Apache Kafka, Apache NiFi, AWS Kinesis, Azure Event Hubs).

- Data Transformation : Proficiency in data transformation and ETL/ELT processes using tools like SQL, Python (Pandas, PySpark), and data integration platforms.

- Data Modeling : Strong understanding of data modeling concepts and techniques (dimensional modeling, relational modeling).

- Data Pipelines : Proven experience in designing and building scalable data pipelines.

- CI/CD Pipelines : Experience with building and maintaining CI/CD pipelines.

- Snowflake : Extensive experience with Snowflake data warehousing.

- Cloud Platforms : Proficiency in at least one major cloud platform (GCP, AWS, or Azure).

- Terraform : Experience with Infrastructure as Code (IaC) using Terraform.

- Jenkins : Experience with Jenkins for automation and CI/CD.

Preferred Qualifications :

- Experience with data governance and metadata management.

- Knowledge of machine learning and AI concepts.

- Experience with containerization and orchestration technologies (Docker, Kubernetes).

- Experience with Data lake technologies.

- Experience with data quality tools