Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Questions & Answers

• Question 41:

  Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push-down?

  A. In the Executor's log file, by gripping for "predicate push-down"
  B. In the Stage's Detail screen, in the Completed Stages table, by noting the size of data read from the Input column
  C. In the Storage Detail screen, by noting which RDDs are not stored on disk
  D. In the Delta Lake transaction log. by noting the column statistics
  E. In the Query Detail screen, by interpreting the Physical Plan
  E. In the Query Detail screen, by interpreting the Physical Plan

• Question 42:

  An external object storage container has been mounted to the location/mnt/finance_eda_bucket. The following logic was executed to create a database for the finance team:

  

  After the database was successfully created and permissions configured, a member of the finance team runs the following code:

  

  If all users on the finance team are members of thefinancegroup, which statement describes how thetx_salestable will be created?

  A. A logical table will persist the query plan to the Hive Metastore in the Databricks control plane.
  B. An external table will be created in the storage container mounted to /mnt/finance eda bucket.
  C. A logical table will persist the physical plan to the Hive Metastore in the Databricks control plane.
  D. An managed table will be created in the storage container mounted to /mnt/finance eda bucket.
  E. A managed table will be created in the DBFS root storage container.
  D. An managed table will be created in the storage container mounted to /mnt/finance eda bucket.

• Question 43:

  A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFramedf. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute

  interval. Events are recorded once per minute per device.

  Streaming DataFramedfhas the following schema:

  "device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"

  Code block:

  

  Choose the response that correctly fills in the blank within the code block to complete this task.

  A. to_interval("event_time", "5 minutes").alias("time")
  B. window("event_time", "5 minutes").alias("time")
  C. "event_time"
  D. window("event_time", "10 minutes").alias("time")
  E. lag("event_time", "10 minutes").alias("time")
  B. window("event_time", "5 minutes").alias("time")

• Question 44:

  The Databricks CLI is use to trigger a run of an existing job by passing the job_id parameter. The response that the job run request has been submitted successfully includes a filed run_id. Which statement describes what the number alongside this field represents?

  A. The job_id is returned in this field.
  B. The job_id and number of times the job has been are concatenated and returned.
  C. The number of times the job definition has been run in the workspace.
  D. The globally unique ID of the newly triggered run.
  D. The globally unique ID of the newly triggered run.

• Question 45:

  The data engineer team is configuring environment for development testing, and production before beginning migration on a new data pipeline. The team requires extensive testing on both the code and data resulting from code execution, and

  the team want to develop and test against similar production data as possible.

  A junior data engineer suggests that production data can be mounted to the development testing environments, allowing pre production code to execute against production data.

  Because all users have

  Admin privileges in the development environment, the junior data engineer has offered to configure permissions and mount this data for the team. Which statement captures best practices for this situation?

  A. Because access to production data will always be verified using passthrough credentials it is safe to mount data to any Databricks development environment.
  B. All developer, testing and production code and data should exist in a single unified workspace; creating separate environments for testing and development further reduces risks.
  C. In environments where interactive code will be executed, production data should only be accessible with read permissions; creating isolated databases for each environment further reduces risks.
  D. Because delta Lake versions all data and supports time travel, it is not possible for user error or malicious actors to permanently delete production data, as such it is generally safe to mount production data anywhere.
  C. In environments where interactive code will be executed, production data should only be accessible with read permissions; creating isolated databases for each environment further reduces risks.

• Question 46:

  The data engineer team has been tasked with configured connections to an external database that does not have a supported native connector with Databricks. The external database already has data security configured by group membership. These groups map directly to user group already created in Databricks that represent various teams within the company.

  A new login credential has been created for each group in the external database. The Databricks Utilities Secrets module will be used to make these credentials available to Databricks users.

  Assuming that all the credentials are configured correctly on the external database and group membership is properly configured on Databricks, which statement describes how teams can be granted the minimum necessary access to using these credentials?

  A. `'Read'' permissions should be set on a secret key mapped to those credentials that will be used by a given team.
  B. No additional configuration is necessary as long as all users are configured as administrators in the workspace where secrets have been added.
  C. "Read" permissions should be set on a secret scope containing only those credentials that will be used by a given team.
  D. "Manage" permission should be set on a secret scope containing only those credentials that will be used by a given team.
  C. "Read" permissions should be set on a secret scope containing only those credentials that will be used by a given team.

• Question 47:

  A Delta Lake table in the Lakehouse named customer_parsams is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data

  engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.

  Immediately after each update succeeds, the data engineer team would like to determine the difference between the new version and the previous of the table.

  Given the current implementation, which method can be used?

  A. Parse the Delta Lake transaction log to identify all newly written data files.
  B. Execute DESCRIBE HISTORY customer_churn_params to obtain the full operation metrics for the update, including a log of all records that have been added or modified.
  C. Execute a query to calculate the difference between the new version and the previous version using Delta Lake's built-in versioning and time travel functionality.
  D. Parse the Spark event logs to identify those rows that were updated, inserted, or deleted.
  C. Execute a query to calculate the difference between the new version and the previous version using Delta Lake's built-in versioning and time travel functionality.

• Question 48:

  The data architect has mandated that all tables in the Lakehouse should be configured as external (also known as "unmanaged") Delta Lake tables. Which approach will ensure that this requirement is met?

  A. When a database is being created, make sure that the LOCATION keyword is used.
  B. When configuring an external data warehouse for all table storage, leverage Databricks for all ELT.
  C. When data is saved to a table, make sure that a full file path is specified alongside the Delta format.
  D. When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.
  E. When the workspace is being configured, make sure that external cloud object storage has been mounted.
  D. When tables are created, make sure that the EXTERNAL keyword is used in the CREATE TABLE statement.

• Question 49:

  A table in the Lakehouse namedcustomer_churn_paramsis used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering

  team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.

  The churn prediction model used by the ML team is fairly stable in production. The team is only interested in making predictions on records that have changed in the past 24 hours.

  Which approach would simplify the identification of these changed records?

  A. Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.
  B. Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.
  C. Calculate the difference between the previous model predictions and the current customer_churn_params on a key identifying unique customers before making new predictions; only make predictions on those customers not in the previous predictions.
  D. Modify the overwrite logic to include a field populated by calling spark.sql.functions.current_timestamp() as data are being written; use this field to identify records written on a particular date.
  E. Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.
  E. Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.

• Question 50:

  Which statement describes the default execution mode for Databricks Auto Loader?

  A. New files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table.
  B. Cloud vendor-specific queue storage and notification services are configured to track newly arriving files; new files are incrementally and impotently into the target Delta Lake table.
  C. Webhook trigger Databricks job to run anytime new data arrives in a source directory; new data automatically merged into target tables using rules inferred from the data.
  D. New files are identified by listing the input directory; the target table is materialized by directory querying all valid files in the source directory.
  A. New files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table.