Senior Data Engineer

We seek a highly skilled and experienced Big Data Architect to join our dynamic team for an esteemed overseas client. The ideal candidate will design, implement, and manage large-scale data architectures. This role requires a deep understanding of big data technologies, data lakes, data warehousing, ETL processes, and data pipeline orchestration. The candidate will play a critical role in handling large databases and data sets, ensuring data integrity, performance, and security.

Key Responsibilities:

• Architecture Design: Design and implement scalable and robust big data architectures to
  support data processing, storage, and analysis.
• Data Management: Develop and maintain data lake and data warehousing solutions for
  efficient data storage and retrieval.
• ETL Processes: Design, develop, and optimize ETL processes to ensure seamless data
  integration from various sources.
• Database Management: Manage and optimize Amazon Redshift clusters for high
  performance and cost efficiency.
• Big Data Technologies: Leverage big data technologies such as Hadoop, Spark, and other
  relevant tools to handle large-scale data processing.
• Scripting: Write and maintain efficient Python scripts for data processing and automation
  tasks.
• Pipeline Orchestration: Utilize Airflow or similar orchestration tools to automate and
  manage complex data pipelines.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to
  understand data requirements and deliver suitable solutions.
• Quality Assurance: Ensure data quality, integrity, and security across all data platforms.
• Monitoring & Troubleshooting: Monitor and troubleshoot data pipelines and architectures to
  ensure high availability and performance.
• Continuous Learning: Stay updated with the latest industry trends and best practices in big
  data technologies and architectures.
• Nice to have: Data Platform Engineering

Qualifications:

• Bachelors or Masters degree in Computer Science, Information Technology, or a related
  field.
• 5+ years of relevant experience in big data architecture, data engineering, or a related field.
• Strong understanding of data lake and data warehousing concepts and architectures.
• Proven experience with ETL technologies and tools.
• Hands-on experience with Amazon Redshift and managing large-scale data warehouses.
• Proficiency in big data technologies, including Hadoop, Spark, and related ecosystems.
• Strong Python scripting skills for data processing and automation.
• Experience with Airflow or similar orchestration tools.
• Experience in handling large databases and data sets, ensuring data quality and
  performance.
• Excellent problem-solving skills and ability to troubleshoot complex data issues.
• Strong communication skills and ability to work collaboratively with cross-functional teams.
• Knowledge of data governance, security, and compliance best practices.

Preferred Skills:

• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Familiarity with additional programming languages like SQL, Java, or Scala.
• Experience with data visualization tools and techniques.

Benefits Offered:

• Flexible Work Hours.
• Work on impactful, industry-changing projects.
• Continuous learning and career growth opportunities..
• Collaborative, innovative, and inclusive environment.
• Insurance Group Medical Coverage, Employee Personal Accident, Maternity and Paternity
  Leaves.
• Performance Feedback Sessions.
• Competitive Compensation.
• Rewards and Recognition Program, Employee Referral Program, Fun Events.