Difference between logical plan and physical plan in pyspark?

Question

Accepted Answer

Logical plan represents the high-level abstract representation of the computation to be performed, while physical plan represents the actual execution plan with specific optimizations and details.

Logical plan is a high-level abstract representation of the computation to be performed.
Physical plan is the actual execution plan with specific optimizations and details.
Logical plan is created first and then optimized to generate the physical plan.
Physical plan includes details like data shuffling, partitioning, and join strategies.
Example: Logical plan may include operations like filter, groupBy, and join, while physical plan may include optimizations like broadcast join or partition pruning.

Difference between logical plan and physical plan in pyspark?

Top DELTA Data Engineer interview questions & answers

Popular interview questions of Data Engineer