Databricks Databricks Certifications DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Questions & Answers

• Question 31:

  In order for Structured Streaming to reliably track the exact progress of the processing so that it can handle any kind of failure by restarting and/or reprocessing, which of the following two approaches is used by Spark to record the offset range of the data being processed in each trigger?

  A. Checkpointing and Write-ahead Logs

  B. Structured Streaming cannot record the offset range of the data being processed in each trigger.

  C. Replayable Sources and Idempotent Sinks

  D. Write-ahead Logs and Idempotent Sinks

  E. Checkpointing and Idempotent Sinks

  Correct Answer: A

  Explanation: The engine uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. -- in the link search for "The engine uses " youll find the answer. https://spark.apache.org/docs/latest/structured-streaming- programming-guide.html#:~:text=The%20engine%20uses%20checkpointing%20and,being%20processe d%20in%20each%20trigger.

• Question 32:

  A data engineer has been using a Databricks SQL dashboard to monitor the cleanliness of the input data to a data analytics dashboard for a retail use case. The job has a Databricks SQL query that returns the number of store-level records where sales is equal to zero. The data engineer wants their entire team to be notified via a messaging webhook whenever this value is greater than 0.

  Which of the following approaches can the data engineer use to notify their entire team via a messaging webhook whenever the number of stores with $0 in sales is greater than zero?

  A. They can set up an Alert with a custom template.

  B. They can set up an Alert with a new email alert destination.

  C. They can set up an Alert with one-time notifications.

  D. They can set up an Alert with a new webhook alert destination.

  E. They can set up an Alert without notifications.

  Correct Answer: D

• Question 33:

  A data engineer wants to create a new table containing the names of customers that live in France. They have written the following command:

  

  A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (PII).

  Which of the following lines of code fills in the above blank to successfully complete the task?

  A. There is no way to indicate whether a table contains PII.

  B. "COMMENT PII"

  C. TBLPROPERTIES PII

  D. COMMENT "Contains PII"

  E. PII

  Correct Answer: D

  Explanation: Ref:https://www.databricks.com/discover/pages/data-quality-management CREATE TABLE my_table (id INT COMMENT 'Unique Identification Number', name STRING COMMENT 'PII', age INT COMMENT 'PII') TBLPROPERTIES ('contains_pii'=True) COMMENT 'Contains PII';

• Question 34:

  Which of the following is hosted completely in the control plane of the classic Databricks architecture?

  A. Worker node

  B. JDBC data source

  C. Databricks web application

  D. Databricks Filesystem

  E. Driver node

  Correct Answer: C

  Explanation: In the classic Databricks architecture, the control plane includes components like the Databricks web application, the Databricks REST API, and the Databricks Workspace. These components are responsible for managing and controlling the Databricks environment, including cluster provisioning, notebook management, access control, and job scheduling. The other options, such as worker nodes, JDBC data sources, Databricks Filesystem (DBFS), and driver nodes, are typically part of the data plane or the execution environment, which is separate from the control plane. Worker nodes are responsible for executing tasks and computations, JDBC data sources are used to connect to external databases, DBFS is a distributed file system for data storage, and driver nodes are responsible for coordinating the execution of Spark jobs.

• Question 35:

  A new data engineering team team. has been assigned to an ELT project. The new data engineering team will need full privileges on the database customers to fully manage the project.

  Which of the following commands can be used to grant full permissions on the database to the new data engineering team?

  A. GRANT USAGE ON DATABASE customers TO team;

  B. GRANT ALL PRIVILEGES ON DATABASE team TO customers;

  C. GRANT SELECT PRIVILEGES ON DATABASE customers TO teams;

  D. GRANT SELECT CREATE MODIFY USAGE PRIVILEGES ON DATABASE customers TO team;

  E. GRANT ALL PRIVILEGES ON DATABASE customers TO team;

  Correct Answer: E

  Explanation: To grant full privileges on the database "customers" to the new data engineering team, you can use the GRANT ALL PRIVILEGES command as shown in option E. This command provides the team with all possible privileges on the specified database, allowing them to fully manage it.

• Question 36:

  A data engineer has three tables in a Delta Live Tables (DLT) pipeline. They have configured the pipeline to drop invalid records at each table. They notice that some data is being dropped due to quality concerns at some point in the DLT pipeline. They would like to determine at which table in their pipeline the data is being dropped.

  Which of the following approaches can the data engineer take to identify the table that is dropping the records?

  A. They can set up separate expectations for each table when developing their DLT pipeline.

  B. They cannot determine which table is dropping the records.

  C. They can set up DLT to notify them via email when records are dropped.

  D. They can navigate to the DLT pipeline page, click on each table, and view the data quality statistics.

  E. They can navigate to the DLT pipeline page, click on the "Error" button, and review the present errors.

  Correct Answer: D

  Explanation: To identify the table in a Delta Live Tables (DLT) pipeline where data is being dropped due to quality concerns, the data engineer can navigate to the DLT pipeline page, click on each table in the pipeline, and view the data quality statistics. These statistics often include information about records dropped, violations of expectations, and other data quality metrics. By examining the data quality statistics for each table in the pipeline, the data engineer can determine at which table the data is being dropped.

• Question 37:

  A data engineer needs to use a Delta table as part of a data pipeline, but they do not know if they have the appropriate permissions. In which of the following locations can the data engineer review their permissions on the table?

  A. Databricks Filesystem

  B. Jobs

  C. Dashboards

  D. Repos

  E. Data Explorer

  Correct Answer: E

• Question 38:

  A data engineer is running code in a Databricks Repo that is cloned from a central Git repository. A colleague of the data engineer informs them that changes have been made and synced to the central Git repository. The data engineer now needs to sync their Databricks Repo to get the changes from the central Git repository.

  Which of the following Git operations does the data engineer need to run to accomplish this task?

  A. Merge

  B. Push

  C. Pull

  D. Commit

  E. Clone

  Correct Answer: C

  Explanation: From the docs:

  In Databricks Repos, you can use Git functionality to:

  Clone, push to, and pull from a remote Git repository. Create and manage branches for development work, including merging, rebasing, and resolving conflicts.

  Create notebooksandmdash;including IPYNB notebooksandmdash;and edit them and other files.

  Visually compare differences upon commit and resolve merge conflicts.

  Source: https://docs.databricks.com/en/repos/index.html

• Question 39:

  A data analyst has a series of queries in a SQL program. The data analyst wants this program to run every day. They only want the final query in the program to run on Sundays. They ask for help from the data engineering team to complete this task.

  Which of the following approaches could be used by the data engineering team to complete this task?

  A. They could submit a feature request with Databricks to add this functionality.

  B. They could wrap the queries using PySpark and use Python's control flow system to determine when to run the final query.

  C. They could only run the entire program on Sundays.

  D. They could automatically restrict access to the source table in the final query so that it is only accessible on Sundays.

  E. They could redesign the data model to separate the data used in the final query into a new table.

  Correct Answer: B

• Question 40:

  Which of the following benefits of using the Databricks Lakehouse Platform is provided by Delta Lake?

  A. The ability to manipulate the same data using a variety of languages

  B. The ability to collaborate in real time on a single notebook

  C. The ability to set up alerts for query failures

  D. The ability to support batch and streaming workloads

  E. The ability to distribute complex data operations

  Correct Answer: D

  Explanation: Delta Lake is a key component of the Databricks Lakehouse Platform that provides several benefits, and one of the most significant benefits is its ability to support both batch and streaming workloads seamlessly. Delta Lake allows you to process and analyze data in real-time (streaming) as well as in batch, making it a versatile choice for various data processing needs. While the other options may be benefits or capabilities of Databricks or the Lakehouse

  Platform in general, they are not specifically associated with Delta Lake.