Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Questions & Answers

• Question 91:

  When scheduling Structured Streaming jobs for production, which configuration automatically recovers from query failures and keeps costs low?

  A. Cluster: New Job Cluster; Retries: Unlimited; Maximum Concurrent Runs: Unlimited
  B. Cluster: New Job Cluster; Retries: None; Maximum Concurrent Runs: 1
  C. Cluster: Existing All-Purpose Cluster; Retries: Unlimited; Maximum Concurrent Runs: 1
  D. Cluster: New Job Cluster; Retries: Unlimited; Maximum Concurrent Runs: 1
  E. Cluster: Existing All-Purpose Cluster; Retries: None; Maximum Concurrent Runs: 1
  D. Cluster: New Job Cluster; Retries: Unlimited; Maximum Concurrent Runs: 1

• Question 92:

  Which statement regarding stream-static joins and static Delta tables is correct?

  A. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of each microbatch.
  B. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of the job's initialization.
  C. The checkpoint directory will be used to track state information for the unique keys present in the join.
  D. Stream-static joins cannot use static Delta tables because of consistency issues.
  E. The checkpoint directory will be used to track updates to the static Delta table.
  A. Each microbatch of a stream-static join will use the most recent version of the static Delta table as of each microbatch.

• Question 93:

  A data architect has designed a system in which two Structured Streaming jobs will concurrently write to a single bronze Delta table. Each job is subscribing to a different topic from an Apache Kafka source, but they will write data with the same schema. To keep the directory structure simple, a data engineer has decided to nest a checkpoint directory to be shared by both streams.

  The proposed directory structure is displayed below:

  

  Which statement describes whether this checkpoint directory structure is valid for the given scenario and why?

  A. No; Delta Lake manages streaming checkpoints in the transaction log.
  B. Yes; both of the streams can share a single checkpoint directory.
  C. No; only one stream can write to a Delta Lake table.
  D. Yes; Delta Lake supports infinite concurrent writers.
  E. No; each of the streams needs to have its own checkpoint directory.
  E. No; each of the streams needs to have its own checkpoint directory.

• Question 94:

  Assuming that the Databricks CLI has been installed and configured correctly, which Databricks CLI command can be used to upload a custom Python Wheel to object storage mounted with the DBFS for use with a production job?

  A. configure
  B. fs
  C. jobs
  D. libraries
  E. workspace
  B. fs

• Question 95:

  In order to prevent accidental commits to production data, a senior data engineer has instituted a policy that all development work will reference clones of Delta Lake tables. After testing both deep and shallow clone, development tables are

  created using shallow clone.

  A few weeks after initial table creation, the cloned versions of several tables implemented as Type 1 Slowly Changing Dimension (SCD) stop working. The transaction logs for the source tables show that vacuum was run the day before.

  Why are the cloned tables no longer working?

  A. The data files compacted by vacuum are not tracked by the cloned metadata; running refresh on the cloned table will pull in recent changes.
  B. Because Type 1 changes overwrite existing records, Delta Lake cannot guarantee data consistency for cloned tables.
  C. The metadata created by the clone operation is referencing data files that were purged as invalid by the vacuum command
  D. Running vacuum automatically invalidates any shallow clones of a table; deep clone should always be used when a cloned table will be repeatedly queried.
  C. The metadata created by the clone operation is referencing data files that were purged as invalid by the vacuum command

• Question 96:

  Two of the most common data locations on Databricks are the DBFS root storage and external object storage mounted with dbutils.fs.mount(). Which of the following statements is correct?

  A. DBFS is a file system protocol that allows users to interact with files stored in object storage using syntax and guarantees similar to Unix file systems.
  B. By default, both the DBFS root and mounted data sources are only accessible to workspace administrators.
  C. The DBFS root is the most secure location to store data, because mounted storage volumes must have full public read and write permissions.
  D. Neither the DBFS root nor mounted storage can be accessed when using %sh in a Databricks notebook.
  E. The DBFS root stores files in ephemeral block volumes attached to the driver, while mounted directories will always persist saved data to external storage between sessions.
  A. DBFS is a file system protocol that allows users to interact with files stored in object storage using syntax and guarantees similar to Unix file systems.

• Question 97:

  The data science team has created and logged a production model using MLflow. The following code correctly imports and applies the production model to output the predictions as a new DataFrame namedpredswith the schema "customer_id LONG, predictions DOUBLE, date DATE".

  

  The data science team would like predictions saved to a Delta Lake table with the ability to compare all predictions across time. Churn predictions will be made at most once per day. Which code block accomplishes this task while minimizing potential compute costs?

  

  A. preds.write.mode("append").saveAsTable("churn_preds")
  B. preds.write.format("delta").save("/preds/churn_preds")
  C. Option C
  D. Option D
  E. Option E
  A. preds.write.mode("append").saveAsTable("churn_preds")

• Question 98:

  A Structured Streaming job deployed to production has been experiencing delays during peak hours of the day. At present, during normal execution, each microbatch of data is processed in less than 3 seconds. During peak hours of the day, execution time for each microbatch becomes very inconsistent, sometimes exceeding 30 seconds. The streaming write is currently configured with a trigger interval of 10 seconds.

  Holding all other variables constant and assuming records need to be processed in less than 10 seconds, which adjustment will meet the requirement?

  A. Decrease the trigger interval to 5 seconds; triggering batches more frequently allows idle executors to begin processing the next batch while longer running tasks from previous batches finish.
  B. Increase the trigger interval to 30 seconds; setting the trigger interval near the maximum execution time observed for each batch is always best practice to ensure no records are dropped.
  C. The trigger interval cannot be modified without modifying the checkpoint directory; to maintain the current stream state, increase the number of shuffle partitions to maximize parallelism.
  D. Use the trigger once option and configure a Databricks job to execute the query every 10 seconds; this ensures all backlogged records are processed with each batch.
  E. Decrease the trigger interval to 5 seconds; triggering batches more frequently may prevent records from backing up and large batches from causing spill.
  D. Use the trigger once option and configure a Databricks job to execute the query every 10 seconds; this ensures all backlogged records are processed with each batch.

• Question 99:

  A data engineer, User A, has promoted a new pipeline to production by using the REST API to programmatically create several jobs. A DevOps engineer, User B, has configured an external orchestration tool to trigger job runs through the REST API. Both users authorized the REST API calls using their personal access tokens.

  Which statement describes the contents of the workspace audit logs concerning these events?

  A. Because the REST API was used for job creation and triggering runs, a Service Principal will be automatically used to identity these events.
  B. Because User B last configured the jobs, their identity will be associated with both the job creation events and the job run events.
  C. Because these events are managed separately, User A will have their identity associated with the job creation events and User B will have their identity associated with the job run events.
  D. Because the REST API was used for job creation and triggering runs, user identity will not be captured in the audit logs.
  E. Because User A created the jobs, their identity will be associated with both the job creation events and the job run events.
  C. Because these events are managed separately, User A will have their identity associated with the job creation events and User B will have their identity associated with the job run events.

• Question 100:

  To reduce storage and compute costs, the data engineering team has been tasked with curating a series of aggregate tables leveraged by business intelligence dashboards, customer-facing applications, production machine learning models, and ad hoc analytical queries.

  The data engineering team has been made aware of new requirements from a customer- facing application, which is the only downstream workload they manage entirely. As a result, an aggregate tableused by numerous teams across the organization will need to have a number of fields renamed, and additional fields will also be added.

  Which of the solutions addresses the situation while minimally interrupting other teams in the organization without increasing the number of tables that need to be managed?

  A. Send all users notice that the schema for the table will be changing; include in the communication the logic necessary to revert the new table schema to match historic queries.
  B. Configure a new table with all the requisite fields and new names and use this as the source for the customer-facing application; create a view that maintains the original data schema and table name by aliasing select fields from the new table.
  C. Create a new table with the required schema and new fields and use Delta Lake's deep clone functionality to sync up changes committed to one table to the corresponding table.
  D. Replace the current table definition with a logical view defined with the query logic currently writing the aggregate table; create a new table to power the customer-facing application.
  E. Add a table comment warning all users that the table schema and field names will be changing on a given date; overwrite the table in place to the specifications of the customer- facing application.
  B. Configure a new table with all the requisite fields and new names and use this as the source for the customer-facing application; create a view that maintains the original data schema and table name by aliasing select fields from the new table.