Databricks Databricks Certifications DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Questions & Answers

• Question 151:

  Which of the following code blocks applies the boolean-returning Python function evaluateTestSuccess to column storeId of DataFrame transactionsDf as a user-defined function?

  A. 1.from pyspark.sql import types as T 2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.BooleanType()) 3.transactionsDf.withColumn("result", evaluateTestSuccessUDF(col("storeId")))

  B. 1.evaluateTestSuccessUDF = udf(evaluateTestSuccess) 2.transactionsDf.withColumn("result", evaluateTestSuccessUDF(storeId))

  C. 1.from pyspark.sql import types as T 2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.IntegerType()) 3.transactionsDf.withColumn("result", evaluateTestSuccess(col("storeId")))

  D. 1.evaluateTestSuccessUDF = udf(evaluateTestSuccess) 2.transactionsDf.withColumn("result", evaluateTestSuccessUDF(col("storeId")))

  E. 1.from pyspark.sql import types as T 2.evaluateTestSuccessUDF = udf(evaluateTestSuccess, T.BooleanType()) 3.transactionsDf.withColumn("result", evaluateTestSuccess(col("storeId")))

  Correct Answer: A

• Question 152:

  Which of the following code blocks performs an inner join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively, excluding columns value and storeId from DataFrame transactionsDf and column attributes from DataFrame itemsDf?

  A. transactionsDf.drop('value', 'storeId').join(itemsDf.select('attributes'), transactionsDf.productId==itemsDf.itemId)

  B. 1.transactionsDf.createOrReplaceTempView('transactionsDf') 2.itemsDf.createOrReplaceTempView('itemsDf') 3.spark.sql("SELECT -value, -storeId FROM transactionsDf INNER JOIN itemsDf ON productId==itemId").drop("attributes")

  C. transactionsDf.drop("value", "storeId").join(itemsDf.drop("attributes"), "transactionsDf.productId==itemsDf.itemId")

  D. 1.transactionsDf \

  2.

  .drop(col('value'), col('storeId')) \

  3.

  .join(itemsDf.drop(col('attributes')), col('productId')==col('itemId'))

  E. 1.transactionsDf.createOrReplaceTempView('transactionsDf') 2.itemsDf.createOrReplaceTempView('itemsDf') 3.statement = """ 4.SELECT * FROM transactionsDf 5.INNER JOIN itemsDf 6.ON transactionsDf.productId==itemsDf.itemId

  7."""

  8.spark.sql(statement).drop("value", "storeId", "attributes")

  Correct Answer: E

• Question 153:

  The code block displayed below contains an error. The code block is intended to write DataFrame transactionsDf to disk as a parquet file in location /FileStore/transactions_split, using column storeId as key for partitioning. Find the error.

  Code block:

  transactionsDf.write.format("parquet").partitionOn("storeId").save("/FileStore/transactions_s plit")A.

  A. The format("parquet") expression is inappropriate to use here, "parquet" should be passed as first argument to the save() operator and "/FileStore/transactions_split" as the second argument.

  B. Partitioning data by storeId is possible with the partitionBy expression, so partitionOn should be replaced by partitionBy.

  C. Partitioning data by storeId is possible with the bucketBy expression, so partitionOn should be replaced by bucketBy.

  D. partitionOn("storeId") should be called before the write operation.

  E. The format("parquet") expression should be removed and instead, the information should be added to the write expression like so: write("parquet").

  Correct Answer: B

• Question 154:

  Which of the following code blocks returns a DataFrame with an added column to DataFrame transactionsDf that shows the unix epoch timestamps in column transactionDate as strings in the format month/day/year in column transactionDateFormatted?

  Excerpt of DataFrame transactionsDf:

  A. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="dd/ MM/yyyy"))

  B. transactionsDf.withColumnRenamed("transactionDate", "transactionDateFormatted", from_unixtime ("transactionDateFormatted", format="MM/dd/yyyy"))

  C. transactionsDf.apply(from_unixtime(format="MM/dd/yyyy")).asColumn("transactionDateFor matted")

  D. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="MM/ dd/yyyy"))

  E. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate"))

  Correct Answer: D

• Question 155:

  Which of the following statements about executors is correct, assuming that one can consider each of the JVMs working as executors as a pool of task execution slots?

  A. Slot is another name for executor.

  B. There must be less executors than tasks.

  C. An executor runs on a single core.

  D. There must be more slots than tasks.

  E. Tasks run in parallel via slots.

  Correct Answer: E

• Question 156:

  Which of the following code blocks writes DataFrame itemsDf to disk at storage location filePath, making sure to substitute any existing data at that location?

  A. itemsDf.write.mode("overwrite").parquet(filePath)

  B. itemsDf.write.option("parquet").mode("overwrite").path(filePath)

  C. itemsDf.write(filePath, mode="overwrite")

  D. itemsDf.write.mode("overwrite").path(filePath)

  E. itemsDf.write().parquet(filePath, mode="overwrite")

  Correct Answer: A

• Question 157:

  Which of the following code blocks returns a copy of DataFrame transactionsDf where the column storeId has been converted to string type?

  A. transactionsDf.withColumn("storeId", convert("storeId", "string"))

  B. transactionsDf.withColumn("storeId", col("storeId", "string"))

  C. transactionsDf.withColumn("storeId", col("storeId").convert("string"))

  D. transactionsDf.withColumn("storeId", col("storeId").cast("string"))

  E. transactionsDf.withColumn("storeId", convert("storeId").as("string"))

  Correct Answer: D

• Question 158:

  In which order should the code blocks shown below be run in order to read a JSON file from location jsonPath into a DataFrame and return only the rows that do not have value 3 in column productId?

  1.

  importedDf.createOrReplaceTempView("importedDf")

  2.

  spark.sql("SELECT * FROM importedDf WHERE productId != 3")

  3.

  spark.sql("FILTER * FROM importedDf WHERE productId != 3")

  4.

  importedDf = spark.read.option("format", "json").path(jsonPath)

  5.

  importedDf = spark.read.json(jsonPath)

  A. 4, 1, 2

  B. 5, 1, 3

  C. 5, 2

  D. 4, 1, 3

  E. 5, 1, 2

  Correct Answer: E

• Question 159:

  Which of the following code blocks returns a new DataFrame in which column attributes of DataFrame itemsDf is renamed to feature0 and column supplier to feature1?

  A. itemsDf.withColumnRenamed(attributes, feature0).withColumnRenamed(supplier, feature1)

  B. 1.itemsDf.withColumnRenamed("attributes", "feature0") 2.itemsDf.withColumnRenamed("supplier", "feature1")

  C. itemsDf.withColumnRenamed(col("attributes"), col("feature0"), col("supplier"), col("feature1"))

  D. itemsDf.withColumnRenamed("attributes", "feature0").withColumnRenamed("supplier", "feature1")

  E. itemsDf.withColumn("attributes", "feature0").withColumn("supplier", "feature1")

  Correct Answer: D

• Question 160:

  The code block shown below should add column transactionDateForm to DataFrame transactionsDf. The column should express the unix-format timestamps in column transactionDate as string type like Apr 26 (Sunday). Choose the answer that correctly fills the blanks in the code block to accomplish this.

  transactionsDf.__1__(__2__, from_unixtime(__3__, __4__))

  A. 1. withColumn

  2.

  "transactionDateForm"

  3.

  "MMM d (EEEE)"

  4.

  "transactionDate"

  B. 1. select

  2.

  "transactionDate"

  3.

  "transactionDateForm"

  4.

  "MMM d (EEEE)"

  C. 1. withColumn

  2.

  "transactionDateForm"

  3.

  "transactionDate"

  4.

  "MMM d (EEEE)"

  D. 1. withColumn

  2.

  "transactionDateForm"

  3.

  "transactionDate"

  4.

  "MM d (EEE)"

  E. 1. withColumnRenamed

  2.

  "transactionDate"

  3.

  "transactionDateForm"

  4.

  "MM d (EEE)"

  Correct Answer: C