What is the difference between lineage and directed acyclic graphs (DAG)?

Question

Accepted Answer

Lineage tracks the history of data transformations, while DAG is a graph structure with nodes representing tasks and edges representing dependencies.

Lineage focuses on the history of data transformations, showing how data has been derived or modified.
DAG is a graph structure where nodes represent tasks and edges represent dependencies between tasks.
Lineage helps in understanding the data flow and ensuring data quality and reliability.
DAG is commonly used in workflow management systems to represent the order of tasks and their dependencies.
An example of lineage would be tracking how raw data is transformed into a final report, showing all the intermediate steps.
An example of a DAG would be a workflow for processing data where each task depends on the completion of certain other tasks.

Top TCS Big Data Engineer interview questions & answers