Jio Platforms Big Data Engineer Interview Questions and Answers

30 Questions in 20 Minutes

Q1. Baiscs of SQL,Python,AWS and spark in depth question
Add your answer

Q1. Explain pyspark architecture
Ans. 
PySpark architecture is based on the Apache Spark architecture, with additional components for Python integration.
• PySpark architecture includes Spark Core, Spark SQL, Spark Streaming, MLlib, and GraphX.

• It allows Python developers to interact with Spark using PySpark API.

• PySpark architecture enables distributed processing of large datasets using RDDs and DataFrames.

• It leverages the power of in-memory processing for fast...read more
Answered by AI
Add your answer

Q1. Basic Questions of Scala Functional Programming concepts.
Add your answer
Q2. Spark internal working and optimization techniques
Ans. 
Spark internal working and optimization techniques
• Spark uses Directed Acyclic Graph (DAG) for optimizing workflows

• Lazy evaluation helps in optimizing transformations by combining them into a single stage

• Caching and persistence of intermediate results can improve performance

• Partitioning data can help in parallel processing and reducing shuffle operations
Answered by AI
Add your answer

Q1. Project overview and spark architecture questions,scala coding questions
Add your answer

Q1. Project overview Scala and AWS services questions
Add your answer

Q1. Asked about multiple switches And experience
Add your answer

Q1. Aws sevices you have worked on
Ans. 
I have worked on various AWS services including S3, EC2, Lambda, Glue, and Redshift.
• S3 - Used for storing and retrieving data

• EC2 - Used for running virtual servers

• Lambda - Used for serverless computing

• Glue - Used for ETL (Extract, Transform, Load) processes

• Redshift - Used for data warehousing and analytics
Answered by AI
Add your answer
Q2. Question on python list and dictionary
Add your answer

Q1. Questions on project and skillset
Add your answer

Q1. Questions mostly revolved around Hadoop , Hive, Spark and SQL
Add your answer

Q1. Explain spark internal flow from end to end
Ans. 
Spark internal flow involves job submission, DAG creation, task scheduling, and execution.
• Job submission: User submits a Spark job to the SparkContext.

• DAG creation: SparkContext creates a Directed Acyclic Graph (DAG) of the job.

• Task scheduling: DAGScheduler breaks the DAG into stages and tasks, which are scheduled by TaskScheduler.

• Task execution: Executors execute the tasks and return results to the driver.

• Result fetch...read more
Answered by AI
Add your answer
Q2. Explain details of hive meta store
Ans. 
Hive meta store stores metadata about Hive tables, partitions, columns, and storage location.
• Hive meta store is a central repository that stores metadata information about Hive tables, partitions, columns, and storage location.

• It stores this metadata in a relational database like MySQL, Derby, or PostgreSQL.

• The metadata includes information such as table names, column names, data types, file formats, and storage locati...read more
Answered by AI
Add your answer
Q3. Write sql query for some simple tables
Ans. 
SQL query for simple tables
• Use SELECT statement to retrieve data

• Specify the columns you want to select

• Use FROM clause to specify the tables you are querying from

• Add WHERE clause to filter the results if needed
Answered by AI
Add your answer

Q1. Depth into data engineer skills
Add your answer
Q2. More questions on python programming
Add your answer

Basic logical and programming questions

Q1. Python and SQL basic questions
Add your answer

Q1. Salary and relocation discussion
Add your answer