Encora Software Engineer Interview Questions and Answers

Creating RDDs in Spark involves loading data from external sources or parallelizing an existing collection.

• RDDs can be created by loading data from external sources like HDFS, local file system, or any other data source supported by Hadoop.

• RDDs can also be created by parallelizing an existing collection in the driver program.

• Transformations in Spark are operations that create a new RDD from an existing one.

• Examples of ...read more

Spark Streaming is implemented using DStreams which are a sequence of RDDs.

• DStreams are created by receiving input data streams from sources like Kafka, Flume, etc.

• The input data is then divided into small batches and processed using Spark's RDD operations.

• The processed data is then pushed to output sources like HDFS, databases, etc.

• Example: val lines = ssc.socketTextStream("localhost", 9999)

• Example: val words = lines.