Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 81:

  Which of the following is a benefit of the Databricks Lakehouse Platform embracing open source technologies?

  A. Cloud-specific integrations
  B. Simplified governance
  C. Ability to scale storage
  D. Ability to scale workloads
  E. Avoiding vendor lock-in
  E. Avoiding vendor lock-in

  ---

  Explanation

  One of the benefits of the Databricks Lakehouse Platform embracing open source technologies is that it avoids vendor lock-in. This means that customers can use the same open source tools and frameworks across different cloud providers, and migrate their data and workloads without being tied to a specific vendor. The Databricks Lakehouse Platform is built on open source projects such as Apache SparkTM, Delta Lake, MLflow, and Redash, which are widely used and trusted by millions of developers. By supporting these open source technologies, the Databricks Lakehouse Platform enables customers to leverage the innovation and community of the open source ecosystem, and avoid the risk of being locked into proprietary or closed solutions. The other options are either not related to open source technologies (A, B, C, D), or not benefits of the Databricks Lakehouse Platform (A, B).

  References: Databricks Documentation - Built on open source, Databricks Documentation - What is the Lakehouse Platform?, Databricks Blog - Introducing the Databricks Lakehouse Platform.

• Question 82:

  What is the role of a Silver table in the medallion architecture?

  A. Store machine learning results
  B. Hold raw sensor data
  C. Store cleaned and enriched data
  D. Store final dashboard-ready aggregates
  E. Store system logs
  C. Store cleaned and enriched data

  ---

  Explanation

  Silver tables contain transformed data that has been cleaned and enriched.

• Question 83:

  A data engineer has a Python notebook in Databricks, but they need to use SQL to accomplish a specific task within a cell. They still want all of the other cells to use Python without making any changes to those cells.

  Which of the following describes how the data engineer can use SQL within a cell of their Python notebook?

  A. It is not possible to use SQL in a Python notebook
  B. They can attach the cell to a SQL endpoint rather than a Databricks cluster
  C. They can simply write SQL syntax in the cell
  D. They can add %sql to the first line of the cell
  E. They can change the default language of the notebook to SQL
  D. They can add %sql to the first line of the cell

  ---

  Explanation

  In Databricks, you can use different languages within the same notebook by using magic commands. Magic commands are special commands that start with a percentage sign (%) and allow you to change the behavior of the cell. To use SQL within a cell of a Python notebook, you can add %sql to the first line of the cell. This will tell Databricks to interpret the rest of the cell as SQL code and execute it against the default database. You can also specify a different database by using the USE statement. The result of the SQL query will be displayed as a table or a chart, depending on the output mode. You can also assign the result to a Python variable by using the -o option. For example, %sql -o df SELECT * FROM my_table will run the SQL query and store the result as a pandas DataFrame in the Python variable df. Option A is incorrect, as it is possible to use SQL in a Python notebook using magic commands. Option B is incorrect, as attaching the cell to a SQL endpoint is not necessary and will not change the language of the cell. Option C is incorrect, as simply writing SQL syntax in the cell will result in a syntax error, as the cell will still be interpreted as Python code. Option E is incorrect, as changing the default language of the notebook to SQL will affect all the cells, not just one.

  References: Use SQL in Notebooks - Knowledge Base - Noteable, [SQL magic commands - Databricks], [Databricks SQL Guide - Databricks]

• Question 84:

  Which file format is used for storing Delta Lake Table?

  A. Parquet
  B. Delta
  C. SV
  D. JSON
  A. Parquet

  ---

  Explanation

  Delta Lake tables use the Parquet format as their underlying storage format. Delta Lake enhances Parquet by adding a transaction log that keeps track of all the operations performed on the table. This allows features like ACID transactions, scalable metadata handling, and schema enforcement, making it an ideal choice for big data processing and management in environments like Databricks.

  References:Databricks documentation on Delta Lake: Delta Lake Overview

• Question 85:

  Which query is performing a streaming hop from raw data to a Bronze table?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  E. Option E
  E. Option E

  ---

  Explanation

• Question 86:

  A data engineer needs to parse only png files in a directory that contains files with different suffixes. Which code should the data engineer use to achieve this task?

  A. df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \.append("/*.png")
  B. df = spark.readstream. format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \ .option("pathGlobfilter", "*.png") \ .load()
  C. df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \.option("pathGlobfilter", "*.png") \ .append()
  D. df = spark.readstream.format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \.load("/*.png")
  B. df = spark.readstream. format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \ .option("pathGlobfilter", "*.png") \ .load()

  ---

  Explanation

  The correct way to filter files by suffix in Auto Loader is to use the pathGlobFilter option. For parsing only .png files, the code should be:

  df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "binaryFile") \ .option("pathGlobfilter", "*.png") \ .load()

  This ensures only .png files are ingested from the directory.

• Question 87:

  A data engineer has been given a new record of data:

  id STRING = 'a1'

  rank INTEGER = 6

  rating FLOAT = 9.4

  Which of the following SQL commands can be used to append the new record to an existing Delta table my_table?

  A. INSERT INTO my_table VALUES ('a1', 6, 9.4)
  B. my_table UNION VALUES ('a1', 6, 9.4)
  C. INSERT VALUES ( 'a1' , 6, 9.4) INTO my_table
  D. UPDATE my_table VALUES ('a1', 6, 9.4)
  E. UPDATE VALUES ('a1', 6, 9.4) my_table
  A. INSERT INTO my_table VALUES ('a1', 6, 9.4)

  ---

  Explanation

  To append a new record to an existing Delta table, you can use the INSERT INTO statement with the VALUES clause. This statement will insert one or more rows into the table with the specified values. Option A is the only code block that follows this syntax correctly. Option B is incorrect, as it uses the UNION operator, which will return a new table that is the union of two tables, not append to an existing table. Option C is incorrect, as it uses the INSERT VALUES statement, which is not a valid SQL syntax. Option D is incorrect, as it uses the UPDATE statement, which will modify existing rows in the table, not append new rows. Option E is incorrect, as it uses the UPDATE VALUES statement, which is also not a valid SQL syntax.

  References: Insert data into a table using SQL | Databricks on AWS, Insert data into a table using SQL - Azure Databricks, Delta Lake Quickstart - Azure Databricks

• Question 88:

  Which compute option should be chosen in a scenario where small-scale ad-hoc Python scripts need to be run at high frequency and should wind down quickly after these queries have finished running?

  A. All-purpose Cluster
  B. Job Cluster
  C. Serverless Compute
  D. SQL Warehouse
  C. Serverless Compute

  ---

  Explanation

  Serverless Compute is ideal for small-scale, frequent, ad-hoc Python scripts because it provisions automatically, scales quickly, and terminates when queries finish, minimizing operational overhead and costs.

• Question 89:

  A data engineer is writing a script that is meant to ingest new data from cloud storage. In the event of the Schema change, the ingestion should fail. It should fail until the changes downstream source can be found and verified as intended changes.

  Which command will meet the requirements?

  A. failOnNewColumns
  B. none
  C. rescue
  D. addNewColumns
  A. failOnNewColumns

  ---

  Explanation

  The failOnNewColumns option enforces strict schema control during ingestion. If new columns appear in the source, the ingestion job fails immediately, ensuring downstream changes are reviewed and validated before processing continues.

• Question 90:

  A data engineer wants to create a new table containing the names of customers who live in France.

  They have written the following command:

  

  A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (Pll).

  Which line of code fills in the above blank to successfully complete the task?

  A. There is no way to indicate whether a table contains PII.
  B. "COMMENT PII"
  C. TBLPROPERTIES PII
  D. COMMENT "Contains PII"
  E. PII
  D. COMMENT "Contains PII"

  ---

  Explanation

  To include a property indicating that a table contains personally identifiable information (PII), the TBLPROPERTIES keyword is used in SQL to add metadata to a table. The correct syntax to define a table property for PII is as follows:

  CREATE TABLE customersInFrance

  USING DELTA

  TBLPROPERTIES ('PII' = 'true')

  AS

  SELECT id,

  firstName,

  lastName

  FROM customerLocations

  WHERE country = 'FRANCE';

  The TBLPROPERTIES ('PII' = 'true') line correctly sets a table property that tags the table as containing personally identifiable information. This is in accordance with organizational policies for handling sensitive information.

  References:Databricks documentation on Delta Lake: Delta Lake on Databricks