Databricks DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Online Questions & Answers

• Question 111:

  The code block shown below should set the number of partitions that Spark uses when shuffling data for joins or aggregations to 100. Choose the answer that correctly fills the blanks in the code block to accomplish this.

  spark.sql.shuffle.partitions

  __1__.__2__.__3__(__4__, 100) A. 1. spark

  2.

  conf

  3.

  set

  4.

  "spark.sql.shuffle.partitions"

  B. 1. pyspark 2. config 3. set 4. spark.shuffle.partitions
  C. 1. spark 2. conf 3. get 4. "spark.sql.shuffle.partitions"
  D. 1. pyspark 2. config 3. set 4. "spark.sql.shuffle.partitions"
  E. 1. spark 2. conf 3. set 4. "spark.sql.aggregate.partitions"
  A

  ---

  Correct code block:

  spark.conf.set("spark.sql.shuffle.partitions", 100) The conf interface is part of the SparkSession, so you need to call it through spark and not pyspark. To configure spark, you need to use the set method, not the get method. get reads a

  property,

  but does not write it. The correct property to achieve what is outlined in the

• Question 112:

  Which of the following code blocks returns a DataFrame with an added column to DataFrame transactionsDf that shows the unix epoch timestamps in column transactionDate as strings in the format month/day/year in column transactionDateFormatted?

  Excerpt of DataFrame transactionsDf:

  A. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="dd/MM/yyyy"))
  B. transactionsDf.withColumnRenamed("transactionDate", "transactionDateFormatted", from_unixtime("transactionDateFormatted", format="MM/dd/yyyy"))
  C. transactionsDf.apply(from_unixtime(format="MM/dd/yyyy")).asColumn("transactionDateFor matted")
  D. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="MM/dd/yyyy"))
  E. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate"))
  D. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="MM/dd/yyyy"))

• Question 113:

  Which of the following code blocks reads in the two-partition parquet file stored at filePath, making sure all columns are included exactly once even though each partition has a different schema?

  Schema of first partition:

  1.root

  2.

  |-- transactionId: integer (nullable = true)

  3.

  |-- predError: integer (nullable = true)

  4.

  |-- value: integer (nullable = true)

  5.

  |-- storeId: integer (nullable = true)

  6.

  |-- productId: integer (nullable = true)

  7.

  |-- f: integer (nullable = true) Schema of second partition: 1.root

  2.

  |-- transactionId: integer (nullable = true)

  3.

  |-- predError: integer (nullable = true)

  4.

  |-- value: integer (nullable = true)

  5.

  |-- storeId: integer (nullable = true)

  6.

  |-- rollId: integer (nullable = true)

  7.

  |-- f: integer (nullable = true)

  8.

  |-- tax_id: integer (nullable = false)

  A. spark.read.parquet(filePath, mergeSchema='y')
  B. spark.read.option("mergeSchema", "true").parquet(filePath)
  C. spark.read.parquet(filePath)
  D. 1.nx = 0 2.for file in dbutils.fs.ls(filePath): 3. if not file.name.endswith(".parquet"): 4. continue 5. df_temp = spark.read.parquet(file.path) 6. if nx == 0: 7. df = df_temp 8. else: 9. df = df.union(df_temp) 10. nx = nx+1 11.df
  E. 1.nx = 0 2.for file in dbutils.fs.ls(filePath): 3. if not file.name.endswith(".parquet"): 4. continue 5. df_temp = spark.read.parquet(file.path) 6. if nx == 0: 7. df = df_temp 8. else: 9. df = df.join(df_temp, how="outer") 10. nx = nx+1 11.df
  B. spark.read.option("mergeSchema", "true").parquet(filePath)

• Question 114:

  Which of the following is a characteristic of the cluster manager?

  A. Each cluster manager works on a single partition of data.
  B. The cluster manager receives input from the driver through the SparkContext.
  C. The cluster manager does not exist in standalone mode.
  D. The cluster manager transforms jobs into DAGs.
  E. In client mode, the cluster manager runs on the edge node.
  B. The cluster manager receives input from the driver through the SparkContext.

• Question 115:

  Which of the following code blocks stores a part of the data in DataFrame itemsDf on executors?

  A. itemsDf.cache().count()
  B. itemsDf.cache(eager=True)
  C. cache(itemsDf)
  D. itemsDf.cache().filter()
  E. itemsDf.rdd.storeCopy()
  A. itemsDf.cache().count()

• Question 116:

  The code block displayed below contains an error. The code block is intended to perform an outer join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively.

  Find the error.

  Code block:

  transactionsDf.join(itemsDf, [itemsDf.itemId, transactionsDf.productId], "outer")

  A. The "outer" argument should be eliminated, since "outer" is the default join type.
  B. The join type needs to be appended to the join() operator, like join().outer() instead of listing it as the last argument inside the join() call.
  C. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.itemId == transactionsDf.productId.
  D. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.col("itemId") == transactionsDf.col("productId").
  E. The "outer" argument should be eliminated from the call and join should be replaced by joinOuter.
  C. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.itemId == transactionsDf.productId.

  ---

  Correct code block:

  transactionsDf.join(itemsDf, itemsDf.itemId == transactionsDf.productId, "outer") Static notebook | Dynamic notebook: See test 1, 33 (Databricks import instructions) (https://flrs.github.io/spark_practice_tests_code/#1/33.html , https://bit.ly/

  sparkpracticeexams_import_instructions)

• Question 117:

  Which of the following describes the characteristics of accumulators?

  A. Accumulators are used to pass around lookup tables across the cluster.
  B. All accumulators used in a Spark application are listed in the Spark UI.
  C. Accumulators can be instantiated directly via the accumulator(n) method of the pyspark.RDD module.
  D. Accumulators are immutable.
  E. If an action including an accumulator fails during execution and Spark manages to restart the action and complete it successfully, only the successful attempt will be counted in the accumulator.
  E. If an action including an accumulator fails during execution and Spark manages to restart the action and complete it successfully, only the successful attempt will be counted in the accumulator.

• Question 118:

  The code block displayed below contains an error. The code block is intended to join DataFrame itemsDf with the larger DataFrame transactionsDf on column itemId. Find the error.

  Code block:

  transactionsDf.join(itemsDf, "itemId", how="broadcast")

  A. The syntax is wrong, how= should be removed from the code block.
  B. The join method should be replaced by the broadcast method.
  C. Spark will only perform the broadcast operation if this behavior has been enabled on the Spark cluster.
  D. The larger DataFrame transactionsDf is being broadcasted, rather than the smaller DataFrame itemsDf.
  E. broadcast is not a valid join type.
  E. broadcast is not a valid join type.

• Question 119:

  Which of the following describes tasks?

  A. A task is a command sent from the driver to the executors in response to a transformation.
  B. Tasks transform jobs into DAGs.
  C. A task is a collection of slots.
  D. A task is a collection of rows.
  E. Tasks get assigned to the executors by the driver.
  E. Tasks get assigned to the executors by the driver.

• Question 120:

  Which of the following is one of the big performance advantages that Spark has over Hadoop?

  A. Spark achieves great performance by storing data in the DAG format, whereas Hadoop can only use parquet files.
  B. Spark achieves higher resiliency for queries since, different from Hadoop, it can be deployed on Kubernetes.
  C. Spark achieves great performance by storing data and performing computation in memory, whereas large jobs in Hadoop require a large amount of relatively slow disk I/O operations.
  D. Spark achieves great performance by storing data in the HDFS format, whereas Hadoop can only use parquet files.
  E. Spark achieves performance gains for developers by extending Hadoop's DataFrames with a user-friendly API.
  C. Spark achieves great performance by storing data and performing computation in memory, whereas large jobs in Hadoop require a large amount of relatively slow disk I/O operations.