GlobalLogic Senior Data Engineer Interview Questions and Answers for Experienced

Spark internal flow involves job submission, DAG creation, task scheduling, and execution.

• Job submission: User submits a Spark job to the SparkContext.

• DAG creation: SparkContext creates a Directed Acyclic Graph (DAG) of the job.

• Task scheduling: DAGScheduler breaks the DAG into stages and tasks, which are scheduled by TaskScheduler.

• Task execution: Executors execute the tasks and return results to the driver.

• Result fetch...read more

Hive meta store stores metadata about Hive tables, partitions, columns, and storage location.

• Hive meta store is a central repository that stores metadata information about Hive tables, partitions, columns, and storage location.

• It stores this metadata in a relational database like MySQL, Derby, or PostgreSQL.

• The metadata includes information such as table names, column names, data types, file formats, and storage locati...read more

SQL query for simple tables

• Use SELECT statement to retrieve data

• Specify the columns you want to select

• Use FROM clause to specify the tables you are querying from

• Add WHERE clause to filter the results if needed