Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 21:

  Which of the following must be specified when creating a new Delta Live Tables pipeline?

  A. A key-value pair configuration
  B. The preferred DBU/hour cost
  C. A path to cloud storage location for the written data
  D. A location of a target database for the written data
  E. At least one notebook library to be executed
  E. At least one notebook library to be executed

  ---

  Explanation

  Option E is the correct answer because it is the only mandatory requirement when creating a new Delta Live Tables pipeline. A pipeline is a data processing workflow that contains materialized views and streaming tables declared in Python or SQL source files. Delta Live Tables infers the dependencies between these tables and ensures updates occur in the correct order. To create a pipeline, you need to specify at least one notebook library to be executed, which contains the Delta Live Tables syntax. You can also specify multiple libraries of different languages within your pipeline. The other options are optional or not applicable for creating a pipeline. Option A is not required, but you can optionally provide a key-value pair configuration to customize the pipeline settings, such as the storage location, the target schema, the notifications, and the pipeline mode. Option B is not applicable, as the DBU/hour cost is determined by the cluster configuration, not the pipeline creation. Option C is not required, but you can optionally specify a storage location for the output data from the pipeline. If you leave it empty, the system uses a default location. Option D is not required, but you can optionally specify a location of a target database for the written data, either in the Hive metastore or the Unity Catalog.

  References: Tutorial: Run your first Delta Live Tables pipeline, What is Delta Live Tables?, Create a pipeline, Pipeline configuration.

• Question 22:

  A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE.

  The table is configured to run in Production mode using the Continuous Pipeline Mode.

  Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

  A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
  B. All datasets will be updated once and the pipeline will persist without any processing.The compute resources will persist but go unused.
  C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
  D. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
  E. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
  C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.

  ---

  Explanation

• Question 23:

  Which of the following triggers processes all available data and stops?

  A. trigger(once=True)
  B. trigger(availableNow=True)
  C. trigger(continuous=True)
  D. trigger(batch=True)
  E. trigger(manual=True)
  B. trigger(availableNow=True)

  ---

  Explanation

  availableNow=True reads all available data once and then stops the stream.

• Question 24:

  A data engineer has a single-task Job that runs each morning before they begin working. After identifying an upstream data issue, they need to set up another task to run a new notebook prior to the original task. Which of the following approaches can the data engineer use to set up the new task?

  A. They can clone the existing task in the existing Job and update it to run the new notebook.
  B. They can create a new task in the existing Job and then add it as a dependency of the original task.
  C. They can create a new task in the existing Job and then add the original task as a dependency of the new task.
  D. They can create a new job from scratch and add both tasks to run concurrently.
  E. They can clone the existing task to a new Job and then edit it to run the new notebook.
  B. They can create a new task in the existing Job and then add it as a dependency of the original task.

  ---

  Explanation

  To set up the new task to run a new notebook prior to the original task in a single-task Job, the data engineer can use the following approach: In the existing Job, create a new task that corresponds to the new notebook that needs to be run. Set up the new task with the appropriate configuration, specifying the notebook to be executed and any necessary parameters or dependencies. Once the new task is created, designate it as a dependency of the original task in the Job configuration. This ensures that the new task is executed before the original task.

• Question 25:

  A data engineer streams customer orders into a Kafka topic (orders_topic) and is currently writing the ingestion script of a DLT pipeline. The data engineer needs to ingest the data from Kafka brokers to DLT using Databricks. What is the correct code for ingesting the data?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  C. Option C

  ---

  Explanation

  The correct way to ingest Kafka data into a Delta Live Tables pipeline is with SQL using CREATE LIVE TABLE ... AS SELECT ... FROM STREAM kafka.... This properly connects to the Kafka broker, subscribes to the topic, and casts the Kafka message value field for downstream processing in DLT.

• Question 26:

  A data analyst has developed a query that runs against Delta table. They want help from the data engineering team to implement a series of tests to ensure the data returned by the query is clean. However, the data engineering team uses

  Python for its tests rather than SQL.

  Which of the following operations could the data engineering team use to run the query and operate with the results in PySpark?

  A. SELECT * FROM sales
  B. spark.delta.table
  C. spark.sql
  D. There is no way to share data between PySpark and SQL.
  E. spark.table
  C. spark.sql

  ---

  Explanation

  The spark.sql operation allows the data engineering team to run a SQL query and return the result as a PySpark DataFrame. This way, the data engineering team can use the same query that the data analyst has developed and operate with the results in PySpark. For example, the data engineering team can use spark.sql("SELECT * FROM sales") to get a DataFrame of all the records from the sales Delta table, and then apply various tests or transformations using PySpark APIs. The other options are either not valid operations (A, D), not suitable for running a SQL query (B, E), or not returning a DataFrame (A).

  References: Databricks Documentation - Run SQL queries, Databricks Documentation - Spark SQL and DataFrames.

• Question 27:

  Where is Delta table data stored in Databricks?

  A. Databricks metastore
  B. Customer cloud storage
  C. Cluster memory
  D. Repos
  E. SQL endpoint
  B. Customer cloud storage

  ---

  Explanation

  Delta table data is stored in the customer's cloud environment, such as S3 or ADLS.

• Question 28:

  Which statement returns the physical location of a database?

  A. DESCRIBE DATABASE EXTENDED db_name
  B. SELECT LOCATION FROM DATABASE db_name
  C. SHOW DATABASE db_name
  D. DBFS PATH db_name
  E. PRINT DATABASE PATH db_name
  A. DESCRIBE DATABASE EXTENDED db_name

  ---

  Explanation

  DESCRIBE DATABASE EXTENDED includes the database's location in DBFS.

• Question 29:

  Which of the following code blocks will remove the rows where the value in column age is greater than 25 from the existing Delta table my_table and save the updated table?

  A. SELECT * FROM my_table WHERE age > 25;
  B. UPDATE my_table WHERE age > 25;
  C. DELETE FROM my_table WHERE age > 25;
  D. UPDATE my_table WHERE age
  E. DELETE FROM my_table WHERE age
  C. DELETE FROM my_table WHERE age > 25;

  ---

  Explanation

  The DELETE command in Delta Lake allows you to remove data that matches a predicate from a Delta table. This command will delete all the rows where the value in the column age is greater than 25 from the existing Delta table my_table and save the updated table. The other options are either incorrect or do not achieve the desired result. Option A will only select the rows that match the predicate, but not delete them. Option B will update the rows that match the predicate, but not delete them. Option D will update the rows that do not match the predicate, but not delete them. Option E will delete the rows that do not match the predicate, which is the opposite of what we want.

  References: Table deletes, updates, and merges -- Delta Lake Documentation

• Question 30:

  What is stored in a Databricks customer's cloud account?

  A. Data
  B. Cluster management metadata
  C. Databricks web application
  D. Notebooks
  A. Data

  ---

  Explanation

  In a Databricks customer's cloud account, the primary elements stored include: Data: This is the central type of content stored in the customer's cloud account. Data might include various datasets, tables, and files that are used and managed through Databricks platforms. Notebooks: These are also stored within a customer's cloud account. Notebooks include scripts, notes, and other information necessary for data analysis and processing tasks. Cluster management metadata is indeed managed through the cloud, but it's primarily handled by Databricks rather than stored directly in the customer's account. The Databricks web application itself is not stored within the customer's cloud account; rather, it's a service provided by Databricks.

  References:Databricks documentation: Data in Databricks