Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 71:

  A Delta Live Table pipeline includes two datasets defined using streaming live table. Three datasets are defined against Delta Lake table sources using live table.

  The table is configured to run in Production mode using the Continuous Pipeline Mode.

  What is the expected outcome after clicking Start to update the pipeline assuming previously unprocessed data exists and all definitions are valid?

  A. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.
  B. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.
  C. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.
  D. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.
  D. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.

  ---

  Explanation

  In Delta Live Tables (DLT), when configured to run in Continuous Pipeline Mode, particularly in a production environment, the system is designed to continuously process and update data as it becomes available. This mode keeps the compute resources active to handle ongoing data processing and automatically updates all datasets defined in the pipeline at predefined intervals. Once the pipeline is manually stopped, the compute resources are terminated to conserve resources and reduce costs. This mode is suitable for production environments where datasets need to be kept up-to-date with the latest data.

  References:Databricks documentation on Delta Live Tables: Delta Live Tables Guide

• Question 72:

  A data engineer needs to apply custom logic to identify employees with more than 5 years of experience in array column employees in table stores. The custom logic should create a new column exp_employees that is an array of all of the employees with more than 5 years of experience for each row. In order to apply this custom logic at scale, the data engineer wants to use the FILTER higher-order function.

  Which of the following code blocks successfully completes this task?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  E. Option E
  A. Option A

  ---

  Explanation

  Option A is the correct answer because it uses the FILTER higher-order function correctly to filter out employees with more than 5 years of experience from the array column "employees". It applies a lambda function i -> i.years_exp > 5 that checks if the years of experience of each employee in the array is greater than 5. If this condition is met, the employee is included in the new array column "exp_employees".

  References: The use of higher-order functions like FILTER can be referenced from Databricks documentation on Higher-Order Functions.

• Question 73:

  Which tool is used by Auto Loader to process data incrementally?

  A. Spark Structured Streaming
  B. Unity Catalog
  C. Checkpointing
  D. Databricks SQL
  A. Spark Structured Streaming

  ---

  Explanation

  Auto Loader in Databricks utilizes Spark Structured Streaming for processing data incrementally. This allows Auto Loader to efficiently ingest streaming or batch data at scale and to recognize new data as it arrives in cloud storage. Spark Structured Streaming provides the underlying engine that supports various incremental data loading capabilities like schema inference and file notification mode, which are crucial for the dynamic nature of data lakes.

  References:Databricks documentation on Auto Loader: Auto Loader Overview

• Question 74:

  An organization has implemented a data pipeline in Databricks and needs to ensure it can scale automatically based on varying workloads without manual cluster management. The goal is to meet the company's Service Level Agreements (SLAs), which require high availability and minimal downtime, while Databricks automatically handles resource allocation and optimization.

  Which approach fulfills these requirements?

  A. Deploy Job Clusters with fixed configurations, dedicated to specific tasks, without automatic scaling.
  B. Use Spot Instances to allocate resources dynamically while minimizing costs, with potential interruptions.
  C. Use Interactive Clusters in Databricks, adjusting cluster sizes manually based on workload demands.
  D. Use Serverless compute in Databricks to automatically scale and provision resources with minimal manual intervention.
  D. Use Serverless compute in Databricks to automatically scale and provision resources with minimal manual intervention.

  ---

  Explanation

  Serverless compute in Databricks automatically provisions and scales resources to meet workload demands, ensuring high availability and minimal downtime while reducing the need for manual cluster management, which aligns with SLA requirements.

• Question 75:

  A new data engineering team team has been assigned to an ELT project. The new data engineering team will need full privileges on the table sales to fully manage the project.

  Which of the following commands can be used to grant full permissions on the database to the new data engineering team?

  A. GRANT ALL PRIVILEGES ON TABLE sales TO team;
  B. GRANT SELECT CREATE MODIFY ON TABLE sales TO team;
  C. GRANT SELECT ON TABLE sales TO team;
  D. GRANT USAGE ON TABLE sales TO team;
  E. GRANT ALL PRIVILEGES ON TABLE team TO sales;
  A. GRANT ALL PRIVILEGES ON TABLE sales TO team;

  ---

  Explanation

  To grant full permissions on a table to a user or a group, you can use the GRANT ALL PRIVILEGES ON TABLE statement. This statement will grant all the possible privileges on the table, such as SELECT, CREATE, MODIFY, DROP, ALTER, etc. Option A is the only code block that follows this syntax correctly. Option B is incorrect, as it does not grant all the possible privileges on the table, but only a subset of them. Option C is incorrect, as it only grants the SELECT privilege on the table, which is not enough to fully manage the project. Option D is incorrect, as it grants the USAGE privilege on the table, which is not a valid privilege for tables. Option E is incorrect, as it grants all the privileges on the table team to the user or group sales, which is the opposite of what the question asks.

  References: Grant privileges on a table using SQL | Databricks on AWS, Grant privileges on a table using SQL - Azure Databricks, SQL Privileges - Databricks

• Question 76:

  In a healthcare provider organization using Delta Lake to store electronic health records (EHRs), a data analyst needs to analyze a snapshot of the patient_records table from two weeks ago before some recent data corrections were applied. What approach should the Data Engineer take to allow the analyst to query that specific prior version?

  A. Truncate the table to remove all data, then reload the data from two weeks ago into the truncated table for the analyst to query.
  B. Identify the version number corresponding to two weeks ago from the Delta transaction log, share that version number with the analyst to query using VERSION AS OF syntax, or export that version to a new Delta table for the analyst to query.
  C. Restore the table to the version from two weeks ago using the RESTORE command, and have the analyst query the restored table.
  D. Use the VACUUM command to remove all versions of the table older than two weeks, then the analyst can query the remaining version.
  B. Identify the version number corresponding to two weeks ago from the Delta transaction log, share that version number with the analyst to query using VERSION AS OF syntax, or export that version to a new Delta table for the analyst to query.

  ---

  Explanation

• Question 77:

  A data engineer wants to schedule their Databricks SQL dashboard to refresh once per day, but they only want the associated SQL endpoint to be running when it is necessary.

  Which of the following approaches can the data engineer use to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard?

  A. They can ensure the dashboard's SQL endpoint matches each of the queries' SQL endpoints.
  B. They can set up the dashboard's SQL endpoint to be serverless.
  C. They can turn on the Auto Stop feature for the SQL endpoint.
  D. They can reduce the cluster size of the SQL endpoint.
  E. They can ensure the dashboard's SQL endpoint is not one of the included query's SQL endpoint.
  C. They can turn on the Auto Stop feature for the SQL endpoint.

  ---

  Explanation

  The data engineer can use the Auto Stop feature to minimize the total running time of the SQL endpoint used in the refresh schedule of their dashboard. The Auto Stop feature allows the SQL endpoint to automatically shut down when there are no active connections, which will minimize the total running time of the SQL endpoint. By scheduling the dashboard to refresh once per day, the SQL endpoint will only be running for a short period of time each day, which will minimize the total running time and reduce costs.

• Question 78:

  A data engineer needs to create a table in Databricks using data from a CSV file at location /path/to/csv.

  They run the following command:

  

  Which of the following lines of code fills in the above blank to successfully complete the task?

  A. None of these lines of code are needed to successfully complete the task
  B. USING CSV
  C. FROM CSV
  D. USING DELTA
  E. FROM "path/to/csv"
  B. USING CSV

  ---

  Explanation

• Question 79:

  A Python file is ready to go into production and the client wants to use the cheapest but most efficient type of cluster possible. The workload is quite small, only processing 10GBs of data with only simple joins and no complex aggregations or wide transformations.

  Which cluster meets the requirement?

  A. Interactive cluster
  B. Job cluster with spot instances enabled
  C. Job cluster with spot instances disabled
  D. Job cluster with Photon enabled
  B. Job cluster with spot instances enabled

  ---

  Explanation

  A job cluster with spot instances enabled is the cheapest and efficient option for small, production workloads. Spot instances reduce costs, and a job cluster ensures the cluster only runs during the workload execution.

• Question 80:

  A data engineer and data analyst are working together on a data pipeline. The data engineer is working on the raw, bronze, and silver layers of the pipeline using Python, and the data analyst is working on the gold layer of the pipeline using SQL The raw source of the pipeline is a streaming input. They now want to migrate their pipeline to use Delta Live Tables.

  Which change will need to be made to the pipeline when migrating to Delta Live Tables?

  A. The pipeline can have different notebook sources in SQL and Python.
  B. The pipeline will need to be written entirely in SQL.
  C. The pipeline will need to be written entirely in Python.
  D. The pipeline will need to use a batch source in place of a streaming source.
  A. The pipeline can have different notebook sources in SQL and Python.

  ---

  Explanation

  When migrating to Delta Live Tables (DLT) with a data pipeline that involves different programming languages across various data layers, the migration does not require unifying the pipeline into a single language. Delta Live Tables support multi-language pipelines, allowing data engineers and data analysts to work in their preferred languages, such as Python for data engineering tasks (raw, bronze, and silver layers) and SQL for data analytics tasks (gold layer). This capability is particularly beneficial in collaborative settings and leverages the strengths of each language for different stages of data processing.

  References:Databricks documentation on Delta Live Tables: Delta Live Tables Guide