Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-PROFESSIONAL-DATA-ENGINEER Online Questions & Answers

• Question 61:

  A nightly job ingests data into a Delta Lake table using the following code:

  

  The next step in the pipeline requires a function that returns an object that can be used to manipulate new records that have not yet been processed to the next table in the pipeline.

  Which code snippet completes this function definition?

  def new_records():

  A. return spark.readStream.table("bronze")
  B. return spark.readStream.load("bronze")
  C. return (spark.read .table("bronze") .filter(col("ingest_time") == current_timestamp()) ) D. return spark.read.option("readChangeFeed", "true").table ("bronze")
  E. return (spark.read .table("bronze") .filter(col("source_file") == f"/mnt/daily_batch/{year}/{month}/{day}") )
  E. return (spark.read .table("bronze") .filter(col("source_file") == f"/mnt/daily_batch/{year}/{month}/{day}") )

• Question 62:

  What is a method of installing a Python package scoped at the notebook level to all nodes in the currently active cluster?

  A. Use andPip install in a notebook cell
  B. Run source env/bin/activate in a notebook setup script
  C. Install libraries from PyPi using the cluster UI
  D. Use andsh install in a notebook cell
  A. Use andPip install in a notebook cell

• Question 63:

  A data engineer wants to reflector the following DLT code, which includes multiple definition with very similar code:

  

  In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.

  

  The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for tables. How can the data engineer fix this?

  A. Convert the list of configuration values to a dictionary of table settings, using table names as keys.
  B. Convert the list of configuration values to a dictionary of table settings, using different input the for loop.
  C. Load the configuration values for these tables from a separate file, located at a path provided by a pipeline parameter.
  D. Wrap the loop inside another table definition, using generalized names and properties to replace with those from the inner table
  A. Convert the list of configuration values to a dictionary of table settings, using table names as keys.

• Question 64:

  Spill occurs as a result of executing various wide transformations. However, diagnosing spill requires one to proactively look for key indicators. Where in the Spark UI are two of the primary indicators that a partition is spilling to disk?

  A. Stage's detail screen and Executor's files
  B. Stage's detail screen and Query's detail screen
  C. Driver's and Executor's log files
  D. Executor's detail screen and Executor's log files
  D. Executor's detail screen and Executor's log files

• Question 65:

  A Databricks job has been configured with 3 tasks, each of which is a Databricks notebook. Task A does not depend on other tasks. Tasks B and C run in parallel, with each having a serial dependency on Task A.

  If task A fails during a scheduled run, which statement describes the results of this run?

  A. Because all tasks are managed as a dependency graph, no changes will be committed to the Lakehouse until all tasks have successfully been completed.
  B. Tasks B and C will attempt to run as configured; any changes made in task A will be rolled back due to task failure.
  C. Unless all tasks complete successfully, no changes will be committed to the Lakehouse; because task A failed, all commits will be rolled back automatically.
  D. Tasks B and C will be skipped; some logic expressed in task A may have been committed before task failure.
  E. Tasks B and C will be skipped; task A will not commit any changes because of stage failure.
  D. Tasks B and C will be skipped; some logic expressed in task A may have been committed before task failure.

• Question 66:

  A Delta Lake table representing metadata about content from user has the following schema:

  user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE

  Based on the above schema, which column is a good candidate for partitioning the Delta Table?

  A. Date
  B. Post_id
  C. User_id
  D. Post_time
  A. Date

• Question 67:

  Which statement regarding spark configuration on the Databricks platform is true?

  A. Spark configuration properties set for an interactive cluster with the Clusters UI will impact all notebooks attached to that cluster.
  B. When the same spar configuration property is set for an interactive to the same interactive cluster.
  C. Spark configuration set within an notebook will affect all SparkSession attached to the same interactive cluster
  D. The Databricks REST API can be used to modify the Spark configuration properties for an interactive cluster without interrupting jobs.
  A. Spark configuration properties set for an interactive cluster with the Clusters UI will impact all notebooks attached to that cluster.

• Question 68:

  The data engineering team is migrating an enterprise system with thousands of tables and views into the Lakehouse. They plan to implement the target architecture using a series of bronze, silver, and gold tables. Bronze tables will almost exclusively be used by production data engineering workloads, while silver tables will be used to support both data engineering and machine learning workloads. Gold tables will largely serve business intelligence and reporting purposes. While personal identifying information (PII) exists in all tiers of data, pseudonymization and anonymization rules are in place for all data at the silver and gold levels.

  The organization is interested in reducing security concerns while maximizing the ability to collaborate across diverse teams.

  Which statement exemplifies best practices for implementing this system?

  A. Isolating tables in separate databases based on data quality tiers allows for easy permissions management through database ACLs and allows physical separation of default storage locations for managed tables.
  B. Because databases on Databricks are merely a logical construct, choices around database organization do not impact security or discoverability in the Lakehouse.
  C. Storinq all production tables in a single database provides a unified view of all data assets available throughout the Lakehouse, simplifying discoverability by granting all users view privileges on this database.
  D. Working in the default Databricks database provides the greatest security when working with managed tables, as these will be created in the DBFS root.
  E. Because all tables must live in the same storage containers used for the database they're created in, organizations should be prepared to create between dozens and thousands of databases depending on their data isolation requirements.
  A. Isolating tables in separate databases based on data quality tiers allows for easy permissions management through database ACLs and allows physical separation of default storage locations for managed tables.

• Question 69:

  A transactions table has been liquid clustered on the columns product_id, user_id, and event_date. Which operation lacks support for cluster on write?

  A. spark.writestream.format('delta').mode('append')
  B. CTAS and RTAS statements
  C. INSERT INTO operations
  D. spark.write.format('delta').mode('append')
  A. spark.writestream.format('delta').mode('append')

• Question 70:

  A junior data engineer is working to implement logic for a Lakehouse table namedsilver_device_recordings. The source data contains 100 unique fields in a highly nested JSON structure.

  Thesilver_device_recordingstable will be used downstream to power several production monitoring dashboards and a production model. At present, 45 of the 100 fields are being used in at least one of these applications.

  The data engineer is trying to determine the best approach for dealing with schema declaration given the highly-nested structure of the data and the numerous fields.

  Which of the following accurately presents information about Delta Lake and Databricks that may impact their decision-making process?

  A. The Tungsten encoding used by Databricks is optimized for storing string data; newly- added native support for querying JSON strings means that string types are always most efficient.
  B. Because Delta Lake uses Parquet for data storage, data types can be easily evolved by just modifying file footer information in place.
  C. Human labor in writing code is the largest cost associated with data engineering workloads; as such, automating table declaration logic should be a priority in all migration workloads.
  D. Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.
  E. Schema inference and evolution on .Databricks ensure that inferred types will always accurately match the data types used by downstream systems.
  D. Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.