Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 141:

  Which Unity Catalog object controls fine-grained access permissions for tables and views?

  A. Workspace
  B. Schema
  C. Cluster
  D. Catalog
  E. Metastore
  E. Metastore

  ---

  Explanation

  In Unity Catalog, the metastore manages data governance and enforces access controls across catalogs, schemas, and objects.

• Question 142:

  A company uses Delta Sharing to collaborate with partners across different cloud providers and geographic regions. What will result in additional costs due to cross-region or egress fees?

  A. Sharing data within the same cloud provider and region
  B. Transferring data via Delta Sharing across clouds and across different geographic regions
  C. Accessing Delta Sharing data using a VPN within the same data center
  D. Utilizing Delta Sharing for internal data analytics within a single cloud environment
  B. Transferring data via Delta Sharing across clouds and across different geographic regions

  ---

  Explanation

  Transferring data via Delta Sharing across clouds and across different geographic regions incurs additional costs because cloud providers charge cross-region and egress fees when data leaves a region or moves between providers.

• Question 143:

  A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL. The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables.

  Which of the following changes will need to be made to the pipeline when migrating to Delta Live Tables?

  A. None of these changes will need to be made
  B. The pipeline will need to stop using the medallion-based multi-hop architecture
  C. The pipeline will need to be written entirely in SQL
  D. The pipeline will need to use a batch source in place of a streaming source
  E. The pipeline will need to be written entirely in Python
  A. None of these changes will need to be made

  ---

  Explanation

  Delta Live Tables is a declarative framework for building reliable, maintainable, and testable data processing pipelines. You define the transformations to perform on your data and Delta Live Tables manages task orchestration, cluster

  management, monitoring, data quality, and error handling. Delta Live Tables supports both SQL and Python as the languages for defining your datasets and expectations. Delta Live Tables also supports both streaming and batch sources,

  and can handle both append-only and upsert data patterns. Delta Live Tables follows the medallion lakehouse architecture, which consists of three layers of data: bronze, silver, and gold. Therefore, migrating to Delta Live Tables does not

  require any of the changes listed in the options B, C, D, or

  E. The data engineer and data analyst can use the same languages, sources, and architecture as before, and simply declare their datasets and expectations using Delta Live Tables

  syntax.

  References:

  What is Delta Live Tables?

  Transform data with Delta Live Tables

  What is the medallion lakehouse architecture?

• Question 144:

  Which of the following describes when to use the CREATE STREAMING LIVE TABLE (formerly CREATE INCREMENTAL LIVE TABLE) syntax over the CREATE LIVE TABLE syntax when creating Delta Live Tables (DLT) tables using SQL?

  A. CREATE STREAMING LIVE TABLE should be used when the subsequent step in the DLT pipeline is static.
  B. CREATE STREAMING LIVE TABLE should be used when data needs to be processed incrementally.
  C. CREATE STREAMING LIVE TABLE is redundant for DLT and it does not need to be used.
  D. CREATE STREAMING LIVE TABLE should be used when data needs to be processed through complicated aggregations.
  E. CREATE STREAMING LIVE TABLE should be used when the previous step in the DLT pipeline is static.
  B. CREATE STREAMING LIVE TABLE should be used when data needs to be processed incrementally.

  ---

  Explanation

  A streaming live table or view processes data that has been added only since the last pipeline update. Streaming tables and views are stateful; if the defining query changes, new data will be processed based on the new query and existing data is not recomputed. This is useful when data needs to be processed incrementally, such as when ingesting streaming data sources or performing incremental loads from batch data sources. A live table or view, on the other hand, may be entirely computed when possible to optimize computation resources and time. This is suitable when data needs to be processed in full, such as when performing complex transformations or aggregations that require scanning all the data.

  References: Difference between LIVE TABLE and STREAMING LIVE TABLE, CREATE STREAMING TABLE, Load data using streaming tables in Databricks SQL.

• Question 145:

  A data engineer wants to schedule their Databricks SQL dashboard to refresh every hour, but they only want the associated SQL endpoint to be running when it is necessary. The dashboard has multiple queries on multiple datasets associated with it. The data that feeds the dashboard is automatically processed using a Databricks Job.

  Which of the following approaches can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?

  A. They can turn on the Auto Stop feature for the SQL endpoint.
  B. They can ensure the dashboard's SQL endpoint is not one of the included query's SQL endpoint.
  C. They can reduce the cluster size of the SQL endpoint.
  D. They can ensure the dashboard's SQL endpoint matches each of the queries' SQL endpoints.
  E. They can set up the dashboard's SQL endpoint to be serverless.
  A. They can turn on the Auto Stop feature for the SQL endpoint.

  ---

  Explanation

  The Auto Stop feature allows the SQL endpoint to automatically stop after a specified period of inactivity. This can help reduce the cost and resource consumption of the SQL endpoint, as it will only run when it is needed to refresh the dashboard or execute queries. The data engineer can configure the Auto Stop setting for the SQL endpoint from the SQL Endpoints UI, by selecting the desired idle time from the Auto Stop dropdown menu. The default idle time is 120 minutes, but it can be set to as low as 15 minutes or as high as 240 minutes. Alternatively, the data engineer can also use the SQL Endpoints REST API to set the Auto Stop setting programmatically.

  References: SQL Endpoints UI, SQL Endpoints REST API, Refreshing SQL Dashboard

• Question 146:

  A data engineer needs to create a table in Databricks using data from their organization's existing SQLite database. They run the following command:

  CREATE TABLE jdbc_customer360

  USING

  OPTIONS (

  url "jdbc:sqlite:/customers.db", dbtable "customer360"

  )

  Which line of code fills in the above blank to successfully complete the task?

  A. autoloader
  B. org.apache.spark.sql.jdbc
  C. sqlite
  D. org.apache.spark.sql.sqlite
  B. org.apache.spark.sql.jdbc

  ---

  Explanation

  To create a table in Databricks using data from an SQLite database, the correct syntax involves specifying the format of the data source. The format in the case of using JDBC (Java Database Connectivity) with SQLite is specified by the

  org.apache.spark.sql.jdbc format. This format allows Spark to interface with various relational databases through JDBC. Here is how the command should be structured:

  CREATE TABLE jdbc_customer360

  USING org.apache.spark.sql.jdbc

  OPTIONS (

  url "jdbc:sqlite:/customers.db",

  dbtable "customer360"

  )

  The USING org.apache.spark.sql.jdbc line specifies that the JDBC data source is being used, enabling Spark to interact with the SQLite database via JDBC.

  References:Databricks documentation on JDBC: Connecting to SQL Databases

  using JDBC

• Question 147:

  A data engineer has a Python variable table_name that they would like to use in a SQL query. They want to construct a Python code block that will run the query using table_name. They have the following incomplete code block:

  ____(f"SELECT customer_id, spend FROM {table_name}")

  Which of the following can be used to fill in the blank to successfully complete the task?

  A. spark.delta.sql
  B. spark.delta.table
  C. spark.table
  D. dbutils.sql
  E. spark.sql
  E. spark.sql

  ---

  Explanation

  The spark.sql method can be used to execute SQL queries programmatically and return the result as a DataFrame. The spark.sql method accepts a string argument that contains a valid SQL statement. The data engineer can use a formatted string literal (f-string) to insert the Python variable table_name into the SQL query. The other methods are either invalid or not suitable for running SQL queries.

  References: Running SQL Queries Programmatically, Formatted string literals, spark.sql

• Question 148:

  Which of the following Structured Streaming queries is performing a hop from a Silver table to a Gold table?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  E. Option E
  E. Option E

  ---

  Explanation

  The best practice is to use "Complete" as output mode instead of "append" when working with aggregated tables. Since gold layer is work final aggregated tables, the only option with output mode as complete is option

  E.

• Question 149:

  A data engineer is developing an ETL process based on Spark SQL. The execution fails. The data engineer checks the Spark UI and can see the ERRORS as follows:

  "java.lang.OutofMemoryError: Java heap space"

  Which two corrective actions should the data engineer perform to resolve this issue? (Choose two.)

  A. Narrow the filters in order to collect less data in the query
  B. Upsize the worker nodes and activate autoshuffle partitions
  C. Upsize the driver node and deactivate autoshuffle partitions
  D. Cache the dataset in order to boost the query performance
  E. Fix the shuffle partitions to 50 to ensure the allocation
  A. Narrow the filters in order to collect less data in the query
  B. Upsize the worker nodes and activate autoshuffle partitions

  ---

  Explanation

  Reducing input with narrower filters lowers memory pressure, and increasing executor resources (upsize worker nodes) plus enabling adaptive/auto shuffle partitioning provides more memory and better partitioning to avoid Java heap OOM during Spark SQL execution.

• Question 150:

  Which component manages structured access control in Databricks?

  A. Delta Live Tables
  B. Unity Catalog
  C. MLflow
  D. Auto Loader
  E. Job Cluster
  B. Unity Catalog

  ---

  Explanation

  Unity Catalog centralizes and secures access control for data assets.