Lowe's Senior Data Engineer Interview Questions and Answers

Spark is a distributed computing framework that processes big data in memory and is known for its speed and ease of use.

• Spark is an open-source, distributed computing system that provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

• It can run programs up to 100x faster than Hadoop MapReduce in memory, or 10x faster on disk.

• Spark provides high-level APIs in Java, Scala...read more

DAG implementation involves defining tasks and their dependencies in a directed acyclic graph.

• DAG is a graph where nodes represent tasks and edges represent dependencies between tasks.

• Tasks can only be executed once all their dependencies have been completed.

• Common tools for DAG implementation include Apache Airflow and Luigi.

• Example: Task A must be completed before Task B can start, forming a dependency between them.

Salting is a technique used in cryptography to add random data to passwords before hashing to prevent rainbow table attacks.

• Salting involves adding a random string of characters to a password before hashing it

• The salt value is unique for each password and is stored alongside the hashed password

• Salting prevents attackers from easily cracking passwords using precomputed rainbow tables

• Example: If a password is 'password12...read more