Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Questions & Answers

• Question 111:

  A member of the data engineering team has submitted a short notebook that they wish to schedule as part of a larger data pipeline. Assume that the commands provided below produce the logically correct results when run as presented.

  

  Which command should be removed from the notebook before scheduling it as a job?

  A. Cmd 2
  B. Cmd 3
  C. Cmd 4
  D. Cmd 5
  E. Cmd 6
  E. Cmd 6

• Question 112:

  The business intelligence team has a dashboard configured to track various summary metrics for retail stories. This includes total sales for the previous day alongside totals and averages for a variety of time periods. The fields required to populate this dashboard have the following schema:

  

  For Demand forecasting, the Lakehouse contains a validated table of all itemized sales updated incrementally in near real-time. This table named products_per_order, includes the following fields:

  

  Because reporting on long-term sales trends is less volatile, analysts using the new dashboard only require data to be refreshed once daily. Because the dashboard will be queried interactively by many users throughout a normal business day, it should return results quickly and reduce total compute associated with each materialization.

  Which solution meets the expectations of the end users while controlling and limiting possible costs?

  A. Use the Delta Cache to persists the products_per_order table in memory to quickly the dashboard with each query.
  B. Populate the dashboard by configuring a nightly batch job to save the required to quickly update the dashboard with each query.
  C. Use Structure Streaming to configure a live dashboard against the products_per_order table within a Databricks notebook.
  D. Define a view against the products_per_order table and define the dashboard against this view.
  B. Populate the dashboard by configuring a nightly batch job to save the required to quickly update the dashboard with each query.

• Question 113:

  A Delta Lake table representing metadata about content posts from users has the following schema:

  user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE

  This table is partitioned by the date column. A query is run with the following filter:

  longitude < 20 and longitude > -20

  Which statement describes how data will be filtered?

  A. Statistics in the Delta Log will be used to identify partitions that might Include files in the filtered range.
  B. No file skipping will occur because the optimizer does not know the relationship between the partition column and the longitude.
  C. The Delta Engine will use row-level statistics in the transaction log to identify the flies that meet the filter criteria.
  D. Statistics in the Delta Log will be used to identify data files that might include records in the filtered range.
  E. The Delta Engine will scan the parquet file footers to identify each row that meets the filter criteria.
  D. Statistics in the Delta Log will be used to identify data files that might include records in the filtered range.

• Question 114:

  The data engineering team maintains the following code:

  

  Assuming that this code produces logically correct results and the data in the source tables has been de-duplicated and validated, which statement describes what will occur when this code is executed?

  A. A batch job will update the enriched_itemized_orders_by_account table, replacing only those rows that have different values than the current version of the table, using accountID as the primary key.
  B. The enriched_itemized_orders_by_account table will be overwritten using the current valid version of data in each of the three tables referenced in the join logic.
  C. An incremental job will leverage information in the state store to identify unjoined rows in the source tables and write these rows to the enriched_iteinized_orders_by_account table.
  D. An incremental job will detect if new rows have been written to any of the source tables; if new rows are detected, all results will be recalculated and used to overwrite the enriched_itemized_orders_by_account table.
  E. No computation will occur until enriched_itemized_orders_by_account is queried; upon query materialization, results will be calculated using the current valid version of data in each of the three tables referenced in the join logic.
  B. The enriched_itemized_orders_by_account table will be overwritten using the current valid version of data in each of the three tables referenced in the join logic.

• Question 115:

  A production workload incrementally applies updates from an external Change Data Capture feed to a Delta Lake table as an always-on Structured Stream job. When data was initially migrated for this table, OPTIMIZE was executed and most data files were resized to 1 GB. Auto Optimize and Auto Compaction were both turned on for the streaming production job. Recent review of data files shows that most data files are under 64 MB, although each partition in the table contains at least 1 GB of data and the total table size is over 10 TB.

  Which of the following likely explains these smaller file sizes?

  A. Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations
  B. Z-order indices calculated on the table are preventing file compaction C Bloom filler indices calculated on the table are preventing file compaction
  C. Databricks has autotuned to a smaller target file size based on the overall size of data in the table
  D. Databricks has autotuned to a smaller target file size based on the amount of data in each partition
  A. Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations

• Question 116:

  A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source. That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months.

  Which describes how Delta Lake can help to avoid data loss of this nature in the future?

  A. The Delta log and Structured Streaming checkpoints record the full history of the Kafka producer.
  B. Delta Lake schema evolution can retroactively calculate the correct value for newly added fields, as long as the data was in the original source.
  C. Delta Lake automatically checks that all fields present in the source data are included in the ingestion layer.
  D. Data can never be permanently dropped or deleted from Delta Lake, so data loss is not possible under any circumstance.
  E. Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state.
  E. Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state.

• Question 117:

  An hourly batch job is configured to ingest data files from a cloud object storage container where each batch represent all records produced by the source system in a given hour. The batch job to process these records into the Lakehouse is sufficiently delayed to ensure no late-arriving data is missed. Theuser_idfield represents a unique key for the data, which has the following schema:

  user_id BIGINT, username STRING, user_utc STRING, user_region STRING, last_login BIGINT, auto_pay BOOLEAN, last_updated BIGINT New records are all ingested into a table namedaccount_historywhich maintains a full record of all data in the same schema as the source. The next table in the system is namedaccount_currentand is implemented as a Type 1 table representing the most recent value for each uniqueuser_id.

  Assuming there are millions of user accounts and tens of thousands of records processed hourly, which implementation can be used to efficiently update the describedaccount_currenttable as part of each hourly batch job?

  A. Use Auto Loader to subscribe to new files in the account history directory; configure a Structured Streaminq trigger once job to batch update newly detected files into the account current table.
  B. Overwrite the account current table with each batch using the results of a query against the account history table grouping by user id and filtering for the max value of last updated.
  C. Filter records in account history using the last updated field and the most recent hour processed, as well as the max last iogin by user id write a merge statement to update or insert the most recent value for each user id.
  D. Use Delta Lake version history to get the difference between the latest version of account history and one version prior, then write these records to account current.
  E. Filter records in account history using the last updated field and the most recent hour processed, making sure to deduplicate on username; write a merge statement to update or insert the most recent value for each username.
  C. Filter records in account history using the last updated field and the most recent hour processed, as well as the max last iogin by user id write a merge statement to update or insert the most recent value for each user id.

• Question 118:

  A junior data engineer has configured a workload that posts the following JSON to the Databricks REST API endpoint2.0/jobs/create.

  

  Assuming that all configurations and referenced resources are available, which statement describes the result of executing this workload three times?

  A. Three new jobs named "Ingest new data" will be defined in the workspace, and they will each run once daily.
  B. The logic defined in the referenced notebook will be executed three times on new clusters with the configurations of the provided cluster ID.
  C. Three new jobs named "Ingest new data" will be defined in the workspace, but no jobs will be executed.
  D. One new job named "Ingest new data" will be defined in the workspace, but it will not be executed.
  E. The logic defined in the referenced notebook will be executed three times on the referenced existing all purpose cluster.
  C. Three new jobs named "Ingest new data" will be defined in the workspace, but no jobs will be executed.

• Question 119:

  Which statement describes the correct use of pyspark.sql.functions.broadcast?

  A. It marks a column as having low enough cardinality to properly map distinct values to available partitions, allowing a broadcast join.
  B. It marks a column as small enough to store in memory on all executors, allowing a broadcast join.
  C. It caches a copy of the indicated table on attached storage volumes for all active clusters within a Databricks workspace.
  D. It marks a DataFrame as small enough to store in memory on all executors, allowing a broadcast join.
  E. It caches a copy of the indicated table on all nodes in the cluster for use in all future queries during the cluster lifetime.
  D. It marks a DataFrame as small enough to store in memory on all executors, allowing a broadcast join.

• Question 120:

  A data engineer needs to capture pipeline settings from an existing in the workspace, and use them to create and version a JSON file to create a new pipeline. Which command should the data engineer enter in a web terminal configured with the Databricks CLI?

  A. Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command
  B. Stop the existing pipeline; use the returned settings in a reset command
  C. Use the alone command to create a copy of an existing pipeline; use the get JSON command to get the pipeline definition; save this to git
  D. Use list pipelines to get the specs for all pipelines; get the pipeline spec from the return results parse and use this to create a pipeline
  A. Use the get command to capture the settings for the existing pipeline; remove the pipeline_id and rename the pipeline; use this in a create command