Big Data Engineer - Hadoop/Python/Spark

Job Type : Full-Time

Job Description :

About the Role :

As a Big Data Developer, you will be responsible for designing, developing, and maintaining applications that process and analyze large datasets. You will work closely with data analysts, data scientists, and other team members to extract valuable insights from data.

Key Responsibilities :

Big Data Technologies :

- Proficiency in Hadoop and Spark frameworks for distributed computing and data processing.

- Experience with HDFS, MapReduce, and Spark SQL.

- Knowledge of data ingestion and ETL processes.

Python :

- Strong programming skills in Python for data manipulation, analysis, and visualization.

- Experience with Python libraries like Pandas, NumPy, and Matplotlib.

Cloud Platforms :

- Familiarity with cloud platforms (GCP) and their big data offerings (e.g., Dataproc, Dataflow).

- Experience deploying and managing big data applications in the cloud.

Data Analysis and Visualization :

- Ability to analyze large datasets and extract meaningful insights.

- Experience with data visualization tools (e.g., Tableau, Power BI).

Data Pipelines :

- Design and implement data pipelines for data ingestion, transformation, and analysis.

- Optimize data pipelines for performance and scalability.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Data Science, or a related field.

- 3-7 years of experience in big data development.

- Strong programming skills in Python.

- Proficiency in Hadoop and Spark frameworks.

- Knowledge of cloud platforms (GCP) is a plus.

- Excellent problem-solving and analytical skills.

- Good communication and teamwork skills.

Preferred Skills :

- Experience with machine learning and data mining techniques.

- Knowledge of data warehousing and business intelligence tools.

- Certifications related to big data (e.g., Cloudera Certified Hadoop Administrator).

If you are passionate about big data and have a strong foundation in Hadoop, Spark, Python, and cloud platforms, we encourage you to apply for this exciting opportunity.