Big Data Engineer - Hadoop/Hive (5-14 yrs)

Job Description :

Must have skill :

- Develop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix scripting.

- Utilize Big Data technologies, including Hadoop, Hive, Hbase, and Spark to manage and manipulate large data sets.

- Write and optimize SQL and HQL for database management and data manipulation.

- Implement coding best practices to maintain software quality and readability.

- Optimize queries and tune performance for Hive and Spark applications. Collaborate with team members to design, develop, and implement robust data solutions. Stay abreast of the latest industry trends and technologies.

- A minimum of 5-7 years of industry experience in data engineering.

- Strong programming skills with at least 5 years of experience in Python, Scala, and SQL.

- Expertise in Big Data technologies such as Hadoop, Hive, Hbase, and Spark.

- Proficiency in Unix scripting with a minimum of 5 years of experience.

- Deep knowledge of SQL and HQL.

- Solid understanding of coding best practices.

- Experience in query optimization and performance tuning, particularly for Hive and Spark.

- Excellent problem-solving skills and attention to detail.

- Strong communication and teamwork skills.

Big Data Expertise :

- The candidate should have a strong understanding of Big Data technologies such as Hadoop, Hive, Hbase, and Spark.

- They should be capable of managing and manipulating large datasets using these technologies.

Spark Streaming Experience :

- The candidate should have recent and significant experience with Spark Streaming.

- This shows their ability to work with real-time data and perform stream processing.

Database Management :

- Proficiency in SQL and HQL is required.

- The candidate should be capable of creating, managing, and manipulating databases, as well as optimizing and tuning queries.