DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Real Exam

Question 1 Single choice

Which of the following code blocks writes DataFrame storesDF to file path filePath as text files?

A
storesDF.write(filePath)
B
storesDF.write.path(filePath)
C
storesDF.write().text(filePath)
D
storesDF.write.text(filePath)
E
storesDF.write.option("text").path(filePath)

Show answer and explanation

Correct answer: D

Explanation

storesDF.write returns a DataFrameWriter, and its text(filePath) method selects the text output format and destination in one call. Therefore storesDF.write.text(filePath) forms the required writer chain. Calling write as a function or using path without selecting a supported output method does not express the same text write.

Question 2 Single choice

Which of the following code blocks returns a DataFrame containing exactly one row when applied to transactionsDf?

Full DataFrame transactionsDf:

1.+-------------+---------+-----+-------+---------+----+

3.+-------------+---------+-----+-------+---------+----+

4.| 1| 3| 4| 25| 1|null|

5.| 2| 6| 7| 2| 2|null|

6.| 3| 3| null| 25| 3|null|

7.| 4| null| null| 3| 2|null|

8.| 5| null| null| null| 2|null|

9.| 6| 3| 2| 25| 2|null|

10.+-------------+---------+-----+-------+---------+----+

A
transactionsDf.where(col("storeId").between(3,25))
B
transactionsDf.filter((col("storeId")!=25) | (col("productId")==2))
C
transactionsDf.filter(col("storeId")==25).select("predError","storeId").distinct()
D
transactionsDf.select("productId", "storeId").where("storeId == 2 OR storeId != 25")
E
transactionsDf.where(col("value").isNull()).select("productId", "storeId").distinct()

Show answer and explanation

Correct answer: C

Explanation

Filtering for storeId == 25 keeps three rows. Selecting only predError and storeId makes each of those rows the identical pair (3, 25). The final distinct() removes the duplicate pairs, so transactionsDf yields a DataFrame containing exactly one row.

Question 3 Single choice

Which of the following code blocks shuffles DataFrame transactionsDf, which has 8 partitions, so that it has 10 partitions?

A
transactionsDf.repartition(transactionsDf.getNumPartitions()+2)
B
transactionsDf.repartition(transactionsDf.rdd.getNumPartitions()+2)
C
transactionsDf.coalesce(10)
D
transactionsDf.coalesce(transactionsDf.getNumPartitions()+2)
E
transactionsDf.repartition(transactionsDf._partitions+2)

Show answer and explanation

Correct answer: B

Explanation

transactionsDf.rdd.getNumPartitions() obtains the current partition count of 8 through the underlying RDD. Adding 2 computes the target count of 10, and repartition(...) performs the required shuffle to increase the DataFrame to that count. coalesce is primarily suitable for reducing partitions and does not meet the requested shuffle-based increase.

Question 4 Single choice

A developer notices that all the post-shuffle partitions in a dataset are smaller than the value set
forspark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold.

Which type of join will Adaptive Query Execution (AQE) choose in this case?

A
A Cartesian join
B
A shuffled hash join
C
A broadcast nested loop join
D
A sort-merge join

Show answer and explanation

Correct answer: B

Explanation

AQE compares the sizes of the post-shuffle partitions with spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold. Because every partition is below that threshold, each partition's hash map is small enough for the shuffled hash strategy. AQE therefore favors a shuffled hash join instead of retaining a sort-merge join.

Question 5 Single choice

Which of the following operations can be used to return a new DataFrame from DataFrame storesDF without inducing a shuffle?

A
storesDF.intersect()
B
storesDF.repartition(1)
C
storesDF.union()
D
storesDF.coalesce(1)
E
storesDF.rdd.getNumPartitions()

Show answer and explanation

Correct answer: D

Explanation

coalesce(1) reduces the number of partitions by combining existing partitions through a narrow transformation, so it can return a new DataFrame without a full shuffle. repartition(1) redistributes records and induces a shuffle. intersect also requires data movement, while getNumPartitions reports partition metadata rather than returning a DataFrame.

Question 6 Single choice

The code block shown below contains an error. The code block intended to return a DataFrame containing a column dayOfYear, an integer representation of the day of the year from column openDate from DataFrame storesDF. Identify the error.

Note that column openDate is of type integer and represents a date in the UNIX epoch format - the number of seconds since midnight on January 1st, 1970.

A sample of storesDF is displayed below:

Code block:

storesDF.withColumn("dayOfYear", dayofyear(col("openDate")))

A
The dayofyear() operation cannot extract the day of year from a column of type integer - column openDate must first be converted to type Timestamp.
B
The dayofyear() operation takes a quoted column name rather than a Column object as its first argument - the first argument should be "openDate".
C
The dayofyear() operation cannot extract the day of year from a column of type integer - column openDate must first be converted to type Date.
D
The dayofyear() operation is not applicable in a withColumn() call - the newColumn() operation must be used instead.
E
There is no dayofyear() operation - the day of year number must be extracted using substring utilities.

Show answer and explanation

Correct answer: A

Explanation

openDate holds raw UNIX epoch seconds as an integer, not a temporal value that dayofyear() can interpret directly. The column must first be cast to Timestamp, after which dayofyear() can extract its integer position within the year. withColumn remains appropriate for adding dayOfYear once that timestamp conversion is present.

Question 7 Single choice

A Data Analyst is working on the employees_df and needs to add a new column where a 10% tax is calculated on the salary. Additionally, the data frame contains the column age, which is not needed.

Which code fragment adds the tax column and removes the age column?

A
employees_df = employees_df.withColumn("tax", employees_df.salary * 10) .dropField(age)
B
employees_df = employees_df.withColumn("tax", employees_df.salary * 0.1) .drop("age")
C
employees_df = employees_df.withcolumn("tax", 1it(0.1) * col("salary")) dropField("age")
D
employees_df = employees_df.withColumn("tax", employees_df.salary * 10).drop("age")

Show answer and explanation

Correct answer: B

Question 8 Single choice

Which of the following code blocks reads a CSV at the file path filePath into a Data Frame with the specified schema schema?

A
spark.read().csv(filePath)
B
spark.read().schema("schema").csv(filePath)
C
spark.read.schema(schema).csv(filePath)
D
spark.read.schema("schema").csv(filePath)
E
spark.read().schema(schema).csv(filePath)

Show answer and explanation

Correct answer: C

Explanation

spark.read.schema(schema).csv(filePath) obtains a DataFrameReader, applies the supplied schema object, and reads the path as CSV. The variable schema must not be quoted, because quoting it would pass the literal word "schema" instead of the specified schema definition. In this API, read is accessed as a property.

Question 9 Multiple choice

A data scientist is working with a massive dataset that exceeds the memory capacity of a single machine. The data scientist is considering using Apache SparkTM instead of processing the data using traditional single-machine programming languages like standard Python scripts.

Which two advantages does Apache SparkTM offer over a normal single-machine language in this scenario? (Choose two.)

A
It eliminates the need to write any code, automatically handling all data processing.
B
It has built-in fault tolerance, allowing it to recover seamlessly from node failures during computation.
C
It processes data solely on disk storage, reducing the need for memory resources.
D
It can distribute data processing tasks across a cluster of machines, enabling horizontal scalability.
E
It requires specialized hardware to run, making it unsuitable for commodity hardware clusters.

Show answer and explanation

Correct answers: B, D

Explanation

Spark distributes data and processing tasks across a cluster of machines, providing horizontal scalability when the dataset exceeds one machine's capacity. It also has built-in fault tolerance: when a node fails during computation, Spark can recover affected work from its execution lineage. These capabilities address both the scale and resilience problems that a normal single-machine script cannot handle by itself.

Question 10 Single choice

Which of the following code blocks reads in parquet file /FileStore/imports.parquet as a DataFrame?

A
spark.mode("parquet").read("/FileStore/imports.parquet")
B
spark.read.path("/FileStore/imports.parquet", source="parquet")
C
spark.read().parquet("/FileStore/imports.parquet")
D
spark.read.parquet("/FileStore/imports.parquet")
E
spark.read().format('parquet').open("/FileStore/imports.parquet")

Show answer and explanation

Correct answer: D

Explanation

spark.read is the DataFrameReader associated with the Spark session, and its parquet method accepts the input path directly. Calling spark.read.parquet("/FileStore/imports.parquet") therefore loads the Parquet data and returns a DataFrame. The reader is an attribute rather than a callable, so forms using spark.read() are invalid.