Senior Data Engineer

We are looking for an enthusiastic and adaptable Data Engineer to join our Data Engineering Team. You will be responsible for developing and managing Data Pipelines and assisting various data-driven requests across the company. The position involves a wide variety of technical skills, from developing high throughput Spark jobs to tuning performance of Big Data solutions. The ideal candidate will have experience working with large-scale data and building automated data ETL processes. Data warehousing experience would be ideal.
Key responsibilities include:
Build and maintain big data technologies, environments, and applications, seeking opportunities for improvements and efficiencies
Performing ETL (Extract, Transform, Load) processes based on business requirements using Apache Spark and data ingestion from Apache Kafka
Work with various data platforms including Apache Hadoop, Apache Ozone, AWS S3, Delta Lake, Apache Iceberg
Utilizing orchestration tools like Apache NiFi for managing and scheduling data flows efficiently
Writes performant SQL statement to analyze data with Hive/Impala/Oracle
Full application development lifecycle (SDLC) from design to deployment
Working with multiple stakeholders across teams to fulfill ad-hoc investigations, including large-scale data extraction, transformation, and analyses
About You
Excellent problem-solving skills with good understanding of data engineering concepts
Proficient in Apache Spark with Python and related technologies
Strong knowledge of SQL and performance tuning
Experience in Big Data technologies like Hadoop and Oracle Exadata
Solid knowledge of Linux environments and proficiency with bash scripting
Effective verbal and written communication skills
Nice to have knowledge or prior experience with any of the following
Apache Kafka, Apache Spark with Scala
Orchestration with Apache Nifi, Apache Airflow
Java development and microservices architecture
Build tools like Jenkins
Log analysis and monitoring using Splunk
Databricks, AWS
Working with large data sets with terabytes of data
Corporate Security Responsibility
Every person working for, or on behalf of, Mastercard is responsible for information security. All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and therefore, it is expected that the successful candidate for this position must:

Abide by Mastercard s security policies and practices;
Ensure the confidentiality and integrity of the information being accessed;
Report any suspected information security violation or breach, and
Complete all periodic mandatory security trainings in accordance with Mastercard s guidelines.

NOTE: Candidates go through a thorough screening and interview process. There is no need to include all preferred skills in the job description.

• Abide by Mastercard s security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercard s guidelines.