Cloud Data Engineer

Cloud Data Engineer

Exp - 3 to 10 yrs

Location - Any

Responsibilities

• Design and implement cloud-based data solutions on Microsoft Azure/AWS or GCP.
• Develop, construct, test, and maintain scalable data pipelines.
• Optimize data storage and retrieval processes.
• Ensure data security and compliance with industry standards.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.
• Monitor and troubleshoot data pipeline issues.
• Implement data quality and validation processes.
• Stay up-to-date with the latest cloud technologies and best practices.
• Document data solutions and processes.
• Provide technical support and guidance to team members.
• Participate in code reviews and ensure coding standards are met.
• Automate data workflows and processes.
• Manage cloud resources and optimize costs.
• Develop and maintain ETL processes.
• Implement data governance policies.
• Ensure high availability and disaster recovery of data solutions.
• Perform data migration and integration tasks.
• Collaborate with cross-functional teams to deliver data solutions.
• Conduct performance tuning and optimization of data solutions.
• Provide training and support to end-users.

Qualifications

• Bachelors degree in Computer Science, Information Technology, or a related field.
• 3+ years of experience in cloud data engineering.
• Proficiency in Microsoft Azure or AWS or Google Cloud Platform (GCP)
• Strong programming skills in Python , PySpark and SQL
• Experience with big data technologies such as Hadoop, Spark, and Kafka.
• Knowledge of SQL and NoSQL databases.
• Familiarity with data warehousing concepts and technologies.
• Experience with ETL tools and processes.
• Strong understanding of data security and compliance.
• Excellent problem-solving and analytical skills.
• Ability to work in a fast-paced environment.
• Strong communication and collaboration skills.