DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Web TestEngine demo

Spark helps fast recovery of data in case of a worker fault by providing the MEMORY_AND_DISK storage level option.

If an executor on a worker node fails while calculating an RDD, that RDD can be recomputed by another executor using the lineage.

Spark builds a fault-tolerant layer on top of the legacy RDD data system, which by itself is not fault tolerant.

Due to the mutability of DataFrames after transformations, Spark reproduces them using observed lineage in case of worker node failure.

Spark is only fault-tolerant if this feature is specifically enabled via the spark.fault_recovery.enabled property.