Databricks Databricks Certifications DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Questions & Answers

• Question 51:

  Which of the following describes a scenario in which a data engineer will want to use a single-node cluster?

  A. When they are working interactively with a small amount of data

  B. When they are running automated reports to be refreshed as quickly as possible

  C. When they are working with SQL within Databricks SQL

  D. When they are concerned about the ability to automatically scale with larger data

  E. When they are manually running reports with a large amount of data

  Correct Answer: A

  Explanation: A Single Node cluster is a cluster consisting of an Apache Spark driver and no Spark workers. A Single Node cluster supports Spark jobs and all Spark data sources, including Delta Lake. A Standard cluster requires a minimum of one Spark worker to run Spark jobs.

• Question 52:

  A data engineer needs to determine whether to use the built-in Databricks Notebooks versioning or version their project using Databricks Repos.

  Which of the following is an advantage of using Databricks Repos over the Databricks Notebooks versioning?

  A. Databricks Repos automatically saves development progress

  B. Databricks Repos supports the use of multiple branches

  C. Databricks Repos allows users to revert to previous versions of a notebook

  D. Databricks Repos provides the ability to comment on specific changes

  E. Databricks Repos is wholly housed within the Databricks Lakehouse Platform

  Correct Answer: B

  Explanation: An advantage of using Databricks Repos over the built-in Databricks Notebooks versioning is the ability to work with multiple branches. Branching is a fundamental feature ofversion control systems like Git, which Databricks Repos is built upon. It allows you to create separate branches for different tasks, features, or experiments within your project. This separation helps in parallel development and experimentation without affecting the main branch or the work of other team members. Branching provides a more organized and collaborative development environment, making it easier to merge changes and manage different development efforts. While Databricks Notebooks versioning also allows you to track versions of notebooks, it may not provide the same level of flexibility and collaboration as branching in Databricks Repos.

• Question 53:

  A data engineer needs to create a table in Databricks using data from a CSV file at location /path/to/csv.

  They run the following command:

  

  Which of the following lines of code fills in the above blank to successfully complete the task?

  A. None of these lines of code are needed to successfully complete the task

  B. USING CSV

  C. FROM CSV

  D. USING DELTA

  E. FROM "path/to/csv"

  Correct Answer: B

• Question 54:

  A data engineer has realized that they made a mistake when making a daily update to a table. They need to use Delta time travel to restore the table to a version that is 3 days old. However, when the data engineer attempts to time travel to the older version, they are unable to restore the data because the data files have been deleted.

  Which of the following explains why the data files are no longer present?

  A. The VACUUM command was run on the table

  B. The TIME TRAVEL command was run on the table

  C. The DELETE HISTORY command was run on the table

  D. The OPTIMIZE command was nun on the table

  E. The HISTORY command was run on the table

  Correct Answer: A

  Explanation: The VACUUM command in Delta Lake is used to clean up and remove unnecessary data files that are no longer needed for time travel or query purposes. When you run VACUUMwith certain retention settings, it can delete older data files, which might include versions of data that are older than the specified retention period. If the data engineer is unable to restore the table to a version that is 3 days old because the data files have been deleted, it's likely because the VACUUM command was run on the table, removing the older data files as part of data cleanup.

• Question 55:

  Which of the following describes a scenario in which a data team will want to utilize cluster pools?

  A. An automated report needs to be refreshed as quickly as possible.

  B. An automated report needs to be made reproducible.

  C. An automated report needs to be tested to identify errors.

  D. An automated report needs to be version-controlled across multiple collaborators.

  E. An automated report needs to be runnable by all stakeholders.

  Correct Answer: A

  Explanation: Cluster pools are typically used in distributed computing environments, such as cloud-based data platforms like Databricks. They allow you to pre-allocate a set of compute resources (a cluster) for specific tasks or workloads. In this case, if an automated report needs to be refreshed as quickly as possible, you can allocate a cluster pool with sufficient resources to ensure fast data processing and report generation. This helps ensure that the report is generated with minimal latency and can be delivered to stakeholders in a timely manner. Cluster pools allow you to optimize resource allocation for high-demand, time-sensitive tasks like real-time report generation.

• Question 56:

  A data engineer has a Python variable table_name that they would like to use in a SQL query. They want to construct a Python code block that will run the query using table_name.

  They have the following incomplete code block:

  ____(f"SELECT customer_id, spend FROM {table_name}")

  Which of the following can be used to fill in the blank to successfully complete the task?

  A. spark.delta.sql

  B. spark.delta.table

  C. spark.table

  D. dbutils.sql

  E. spark.sql

  Correct Answer: E

• Question 57:

  An engineering manager wants to monitor the performance of a recent project using a Databricks SQL query. For the first week following the project's release, the manager wants the query results to be updated every minute. However, the manager is concerned that the compute resources used for the query will be left running and cost the organization a lot of money beyond the first week of the project's release.

  Which of the following approaches can the engineering team use to ensure the query does not cost the organization any money beyond the first week of the project's release?

  A. They can set a limit to the number of DBUs that are consumed by the SQL Endpoint.

  B. They can set the query's refresh schedule to end after a certain number of refreshes.

  C. They cannot ensure the query does not cost the organization money beyond the first week of the project's release.

  D. They can set a limit to the number of individuals that are able to manage the query's refresh schedule.

  E. They can set the query's refresh schedule to end on a certain date in the query scheduler.

  Correct Answer: E

  If a dashboard is configured for automatic updates, it has a Scheduled button at the top, rather than a Schedule button. To stop automatically updating the dashboard and remove its subscriptions:

  Click Scheduled.

  In the Refresh every drop-down, select Never.

  Click Save. The Scheduled button label changes to Schedule. Source:https://learn.microsoft.com/en-us/azure/databricks/sql/user/dashboards/

• Question 58:

  A data engineer has a Job that has a complex run schedule, and they want to transfer that schedule to other Jobs.

  Rather than manually selecting each value in the scheduling form in Databricks, which of the following tools can the data engineer use to represent and submit the schedule programmatically?

  A. pyspark.sql.types.DateType

  B. datetime

  C. pyspark.sql.types.TimestampType

  D. Cron syntax

  E. There is no way to represent and submit this information programmatically

  Correct Answer: D

• Question 59:

  A data engineer needs to apply custom logic to string column city in table stores for a specific use case. In order to apply this custom logic at scale, the data engineer wants to create a SQL user-defined function (UDF). Which of the following code blocks creates this SQL UDF?

  

  A. Option A

  B. Option B

  C. Option C

  D. Option D

  E. Option E

  Correct Answer: A

  Explanation: https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined- functions.html

• Question 60:

  A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE. Three datasets are defined against Delta Lake table sources using LIVE TABLE.

  The table is configured to run in Production mode using the Continuous Pipeline Mode.

  Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after clicking Start to update the pipeline?

  A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist to allow for additional testing.

  B. All datasets will be updated once and the pipeline will persist without any processing. The compute resources will persist but go unused.

  C. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped.

  D. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated.

  E. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing.

  Correct Answer: C

  Explanation: In a Delta Live Table pipeline running in Continuous Pipeline Mode, when you click Start to update the pipeline, the following outcome is expected: All datasets defined using STREAMING LIVE TABLE and LIVE TABLE against Delta Lake table sources will be updated at set intervals. The compute resources will be deployed for the update process and will be active during the execution of the pipeline. The compute resources will be terminated when the pipeline is stopped or shut down. This mode allows for continuous and periodic updates to the datasets as new data arrives or changes in the underlying Delta Lake tables occur. The compute resources are provisioned and utilized during the update intervals to process the data and perform the necessary operations.