Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 181:

  Identify the impact of ON VIOLATION DROP ROW and ON VIOLATION FAIL UPDATE for a constraint violation.

  A data engineer has created an ETL pipeline using Delta Live table to manage their company travel reimbursement detail, they want to ensure that the if the location details has not been provided by the employee, the pipeline needs to be

  terminated.

  How can the scenario be implemented?

  A. CONSTRAINT valid_location EXPECT (location = NULL)
  B. CONSTRAINT valid_location EXPECT (location != NULL) ON VIOLATION FAIL UPDATE
  C. CONSTRAINT valid_location EXPECT (location != NULL) ON DROP ROW
  D. CONSTRAINT valid_location EXPECT (location != NULL) ON VIOLATION FAIL
  B. CONSTRAINT valid_location EXPECT (location != NULL) ON VIOLATION FAIL UPDATE

  ---

  Explanation

• Question 182:

  A data engineer needs to determine whether to use the built-in Databricks Notebooks versioning or version their project using Databricks Repos.

  Which of the following is an advantage of using Databricks Repos over the Databricks Notebooks versioning?

  A. Databricks Repos automatically saves development progress
  B. Databricks Repos supports the use of multiple branches
  C. Databricks Repos allows users to revert to previous versions of a notebook
  D. Databricks Repos provides the ability to comment on specific changes
  E. Databricks Repos is wholly housed within the Databricks Lakehouse Platform
  B. Databricks Repos supports the use of multiple branches

  ---

  Explanation

  Databricks Repos is a visual Git client and API in Databricks that supports common Git operations such as cloning, committing, pushing, pulling, and branch management. Databricks Notebooks versioning is a legacy feature that allows users to link notebooks to GitHub repositories and perform basic Git operations. However, Databricks Notebooks versioning does not support the use of multiple branches for development work, which is an advantage of using Databricks Repos. With Databricks Repos, users can create and manage branches for different features, experiments, or bug fixes, and merge, rebase, or resolve conflicts between them. Databricks recommends using a separate branch for each notebook and following data science and engineering code development best practices using Git for version control, collaboration, and CI/CD.

  References: Git integration with Databricks Repos - Azure Databricks | Microsoft Learn, Git version control for notebooks (legacy) | Databricks on AWS, Databricks Repos Is Now Generally Available - New `Files' Feature in ..., Databricks Repos - What it is and how we can use it | Adatis.

• Question 183:

  A data engineer has left the organization. The data team needs to transfer ownership of the data engineer's Delta tables to a new data engineer. The new data engineer is the lead engineer on the data team.

  Assuming the original data engineer no longer has access, which of the following individuals must be the one to transfer ownership of the Delta tables in Data Explorer?

  A. Databricks account representative
  B. This transfer is not possible
  C. Workspace administrator
  D. New lead data engineer
  E. Original data engineer
  C. Workspace administrator

  ---

  Explanation

  The workspace administrator is the only individual who can transfer ownership of the Delta tables in Data Explorer, assuming the original data engineer no longer has access. The workspace administrator has the highest level of permissions in the workspace and can manage all resources, users, and groups. The other options are either not possible or not sufficient to perform the ownership transfer. The Databricks account representative is not involved in the workspace management. The transfer is possible and not dependent on the original data engineer. The new lead data engineer may not have the necessary permissions to access or modify the Delta tables, unless granted by the workspace administrator or the original data engineer before leaving.

  References: Workspace access control, Manage Unity Catalog object ownership.

• Question 184:

  Which of the following is stored in the Databricks customer's cloud account?

  A. Databricks web application
  B. Cluster management metadata
  C. Repos
  D. Data
  E. Notebooks
  D. Data

  ---

  Explanation

  The only option that is stored in the Databricks customer's cloud account is data. Data is stored in the customer's cloud storage service, such as AWS S3 or Azure Data Lake Storage. The customer has full control and ownership of their data and can access it directly from their cloud account. Option A is not correct, as the Databricks web application is hosted and managed by Databricks on their own cloud infrastructure. The customer does not need to install or maintain the web application, but only needs to access it through a web browser. Option B is not correct, as the cluster management metadata is stored and managed by Databricks on their own cloud infrastructure. The cluster management metadata includes information such as cluster configuration, status, logs, and metrics. The customer can view and manage their clusters through the Databricks web application, but does not have direct access to the cluster management metadata. Option C is not correct, as the repos are stored and managed by Databricks on their own cloud infrastructure. Repos are version-controlled repositories that store code and data files for Databricks projects. The customer can create and manage their repos through the Databricks web application, but does not have direct access to the repos. Option E is not correct, as the notebooks are stored and managed by Databricks on their own cloud infrastructure. Notebooks are interactive documents that contain code, text, and visualizations for Databricks workflows. The customer can create and manage their notebooks through the Databricks web application, but does not have direct access to the notebooks.

  References: Databricks Architecture Databricks Data Sources Databricks Repos [Databricks Notebooks] [Databricks Data Engineer Professional Exam Guide]

• Question 185:

  Which of the following data lakehouse features results in improved data quality over a traditional data lake?

  A. A data lakehouse provides storage solutions for structured and unstructured data.
  B. A data lakehouse supports ACID-compliant transactions.
  C. A data lakehouse allows the use of SQL queries to examine data.
  D. A data lakehouse stores data in open formats.
  E. A data lakehouse enables machine learning and artificial Intelligence workloads.
  B. A data lakehouse supports ACID-compliant transactions.

  ---

  Explanation

  A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. One of the key features of a data lakehouse is that it supports ACID- compliant transactions, which means that it ensures data integrity, consistency, and isolation across concurrent read and write operations3. This feature results in improved data quality over a traditional data lake, which does not support transactions and may suffer from data corruption, duplication, or inconsistency due to concurrent or streaming data ingestion and processing.

  References: 1: What is a Data Lakehouse? - Databricks 2: What is a Data Lakehouse? Definition, features and benefits. - Qlik 3: ACID Transactions - Databricks : [Data Lake vs Data Warehouse: Key Differences] : [Data Lakehouse: The Future of Data Engineering]

• Question 186:

  In which of the following scenarios should a data engineer select a Task in the Depends On field of a new Databricks Job Task?

  A. When another task needs to be replaced by the new task
  B. When another task needs to fail before the new task begins
  C. When another task has the same dependency libraries as the new task
  D. When another task needs to use as little compute resources as possible
  E. When another task needs to successfully complete before the new task begins
  E. When another task needs to successfully complete before the new task begins

  ---

  Explanation

  A data engineer can create a multi-task job in Databricks that consists of multiple tasks that run in a specific order. Each task can have one or more dependencies, which are other tasks that must run before the current task. The Depends On field of a new Databricks Job Task allows the data engineer to specify the dependencies of the task. The data engineer should select a task in the Depends On field when they want the new task to run only after the selected task has successfully completed. This can help the data engineer to create a logical sequence of tasks that depend on each other's outputs or results. For example, a data engineer can create a multi-task job that consists of the following tasks: Task A: Ingest data from a source using Auto Loader Task B: Transform the data using Spark SQL Task C: Write the data to a Delta Lake table Task D: Analyze the data using Spark ML Task E: Visualize the data using Databricks SQL In this case, the data engineer can set the dependencies of each task as follows: Task A: No dependencies Task B: Depends on Task A Task C: Depends on Task B Task D: Depends on Task C Task E: Depends on Task D This way, the data engineer can ensure that each task runs only after the previous task has successfully completed, and the data flows smoothly from ingestion to visualization. The other options are incorrect because they do not describe valid scenarios for selecting a task in the Depends On field. The Depends On field does not affect the following aspects of a task: Whether the task needs to be replaced by another task Whether the task needs to fail before another task begins Whether the task has the same dependency libraries as another task Whether the task needs to use as little compute resources as possible

  References: Create a multi-task job, Run tasks conditionally in a Databricks job, Databricks Jobs.

• Question 187:

  Which of the following describes the type of workloads that are always compatible with Auto Loader?

  A. Dashboard workloads
  B. Streaming workloads
  C. Machine learning workloads
  D. Serverless workloads
  E. Batch workloads
  B. Streaming workloads

  ---

  Explanation

  Auto Loader is a Structured Streaming source that incrementally and efficiently processes new data files as they arrive in cloud storage. It supports both Python and SQL in Delta Live Tables, which are ideal for building streaming data pipelines. Auto Loader can handle near real-time ingestion of millions of files per hour and provide exactly- once guarantees when writing data into Delta Lake. Auto Loader is not designed for dashboard, machine learning, serverless, or batch workloads, which have different requirements and characteristics.

  References: What is Auto Loader?, Delta Live Tables

• Question 188:

  A data architect has determined that a table of the following format is necessary:

  

  Which of the following code blocks uses SQL DDL commands to create an empty Delta table in the above format regardless of whether a table already exists with this name?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  E. Option E
  E. Option E

  ---

  Explanation

  References: Create a table using SQL | Databricks on AWS, Create a table using SQL - Azure Databricks, Delta Lake Quickstart - Azure Databricks

• Question 189:

  Which of the following data workloads will utilize a Gold table as its source?

  A. A job that enriches data by parsing its timestamps into a human-readable format
  B. A job that aggregates uncleaned data to create standard summary statistics
  C. A job that cleans data by removing malformatted records
  D. A job that queries aggregated data designed to feed into a dashboard
  E. A job that ingests raw data from a streaming source into the Lakehouse
  D. A job that queries aggregated data designed to feed into a dashboard

  ---

  Explanation

  A Gold table is a table that contains highly refined and aggregated data that powers analytics, machine learning, and production applications. It represents data that has been transformed into knowledge, rather than just information. A Gold table is typically the final output of a medallion lakehouse architecture, where data flows from Bronze to Silver to Gold tables, with each layer improving the structure and quality of data. A job that queries aggregated data designed to feed into a dashboard is an example of a data workload that will utilize a Gold table as its source, as it requires data that is ready for consumption and analysis. The other options are either data workloads that will use a Bronze or Silver table as their source, or data workloads that will produce a Gold table as their output.

  References: Databricks Documentation - What is the medallion lakehouse architecture?, Databricks Documentation - What is a Medallion Architecture?, K21Academy - Delta Lake Architecture and Azure Databricks Workspace.

• Question 190:

  A data engineer is using the following code block as part of a batch ingestion pipeline to read from a composable table:

  

  Which of the following changes needs to be made so this code block will work when the transactions table is a stream source?

  A. Replace predict with a stream-friendly prediction function
  B. Replace schema(schema) with option ("maxFilesPerTrigger", 1)
  C. Replace "transactions" with the path to the location of the Delta table
  D. Replace format("delta") with format("stream")
  E. Replace spark.read with spark.readStream
  E. Replace spark.read with spark.readStream

  ---

  Explanation

  To read from a stream source, the data engineer needs to use the spark.readStream method instead of the spark.read method. The spark.readStream method returns a DataStreamReader object that can be used to specify the details of the input source, such as the format, the schema, the path, and the options. The spark.read method is only suitable for batch processing, not streaming processing. The other changes are not necessary or correct for reading from a stream source.

  References: Structured Streaming Programming Guide, Read a stream, Databricks Data Sources