Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 121:

  A data engineer wants to run SQL and Python code within the same Databricks notebook. How can SQL be executed in a Python notebook cell?

  A. Use spark.sql() inside the Python cell.
  B. Use %sql magic command at the start of the cell.
  C. Use sql.run() command.
  D. Attach notebook to a SQL Warehouse only.
  E. It's not possible.
  B. Use %sql magic command at the start of the cell.

  ---

  Explanation

  In Databricks notebooks, %sql enables inline execution of SQL commands inside notebooks of other languages such as Python.

• Question 122:

  A data engineer is designing an ETL pipeline to process both streaming and batch data from multiple sources. The pipeline must ensure data quality, handle schema evolution, and provide easy maintenance. The team is considering using Delta Live Tables (DLT) in Databricks to achieve these goals. They want to understand the key features and benefits of DLT that make it suitable for this use case.

  Why is Delta Live Tables (DLT) an appropriate choice?

  A. Automatic data quality checks, built-in support for schema evolution, and declarative pipeline development
  B. Manual schema enforcement, high operational overhead, and limited scalability
  C. Requires custom code for data quality checks, no support for streaming data, and complex pipeline maintenance
  D. Supports only batch processing, no data versioning, and high infrastructure costs
  A. Automatic data quality checks, built-in support for schema evolution, and declarative pipeline development

  ---

  Explanation

  Delta Live Tables (DLT) is appropriate because it provides automatic data quality checks through expectations, built-in support for schema evolution, and a declarative approach to pipeline development, making it easier to build and maintain both streaming and batch ETL pipelines.

• Question 123:

  A data engineer is working on a Databricks project that utilizes cloud storage. The data engineer wants to load several JSON files from containers on a storage account as soon as the file arrives within the storage account. Which syntax should the data engineer follow to first load the files into a dataframe and check that it is working as expected using Python?

  A. df = spark.read.json("input/path")
  B. df = spark.readStream.format("cloud").option("json").load("/input/path")
  C. df = spark.readStream.format("json".load("input/path")
  D. df = spark.readStream.format("cloudFiles").option("cloudFiles.format", "json").load("/input/path")
  D. df = spark.readStream.format("cloudFiles").option("cloudFiles.format", "json").load("/input/path")

  ---

  Explanation

  To automatically ingest JSON files as soon as they arrive in cloud storage, the correct syntax is to use Auto Loader with spark.readStream.format("cloudFiles").option("cloudFiles.format", "json").load("/input/ path"). This enables continuous ingestion of new files into a streaming dataframe.

• Question 124:

  A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The code block used by the data engineer is below:

  

  If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?

  A. processingTime(1)
  B. trigger(availableNow=True)
  C. trigger(parallelBatch=True)
  D. trigger(processingTime="once")
  E. trigger(continuous="once")
  B. trigger(availableNow=True)

  ---

  Explanation

  https://spark.apache.org/docs/latest/api/python/reference/pyspark.ss/api/pyspark.sql.strea ming.DataStreamWriter.trigger.html

• Question 125:

  A data engineer needs to use a Delta table as part of a data pipeline, but they do not know if they have the appropriate permissions.

  In which of the following locations can the data engineer review their permissions on the table?

  A. Databricks Filesystem
  B. Jobs
  C. Dashboards
  D. Repos
  E. Data Explorer
  E. Data Explorer

  ---

  Explanation

  Data Explorer is a graphical interface that allows you to browse, create, and manage data objects such as databases, tables, and views in your workspace. You can also review and modify the permissions on these data objects using Data Explorer. To access Data Explorer, you can click on the Data icon in the sidebar, or use the %sql magic command in a notebook. You can then select a database and a table, and click on the Permissions tab to view and edit the access control lists (ACLs) for the table. You can also use SQL commands such as SHOW GRANT and GRANT to query and modify the permissions on a Delta table.

  References: Data Explorer Access control for Delta tables SHOW GRANT [GRANT]

• Question 126:

  A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The cade block used by the data engineer is below:

  

  If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?

  A. trigger("5 seconds")
  B. trigger()
  C. trigger(once="5 seconds")
  D. trigger(processingTime="5 seconds")
  E. trigger(continuous="5 seconds")
  D. trigger(processingTime="5 seconds")

  ---

  Explanation

  The processingTime option specifies a time-based trigger interval for fixed interval micro-batches. This means that the query will execute a micro-batch to process data every 5 seconds, regardless of how much data is available. This option is suitable for near-real time processing workloads that require low latency and consistent processing frequency. The other options are either invalid syntax (A, C), default behavior (B), or experimental feature (E).

  References: Databricks Documentation - Configure Structured Streaming trigger intervals, Databricks Documentation - Trigger.

• Question 127:

  A data organization leader is upset about the data analysis team's reports being different from the data engineering team's reports. The leader believes the siloed nature of their organization's data engineering and data analysis architectures is to blame.

  Which of the following describes how a data lakehouse could alleviate this issue?

  A. Both teams would autoscale their work as data size evolves
  B. Both teams would use the same source of truth for their work
  C. Both teams would reorganize to report to the same department
  D. Both teams would be able to collaborate on projects in real-time
  E. Both teams would respond more quickly to ad-hoc requests
  B. Both teams would use the same source of truth for their work

  ---

  Explanation

  A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. By using a data lakehouse, both the data analysis and data engineering teams can access the same data sources and formats, ensuring data consistency and quality across their reports. A data lakehouse also supports schema enforcement and evolution, data validation, and time travel to old table versions, which can help resolve data conflicts and errors1.

  References: 1: What is a Data Lakehouse? - Databricks 2: What is a data lakehouse? | IBM

• Question 128:

  An engineering manager uses a Databricks SQL query to monitor ingestion latency for each data source. The manager checks the results of the query every day, but they are manually rerunning the query each day and waiting for the results. Which of the following approaches can the manager use to ensure the results of the query are updated each day?

  A. They can schedule the query to refresh every 1 day from the SQL endpoint's page in Databricks SQL.
  B. They can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL.
  C. They can schedule the query to refresh every 1 day from the query's page in Databricks SQL.
  D. They can schedule the query to run every 1 day from the Jobs UI.
  E. They can schedule the query to run every 12 hours from the Jobs UI.
  C. They can schedule the query to refresh every 1 day from the query's page in Databricks SQL.

  ---

  Explanation

  Databricks SQL allows users to schedule queries to run automatically at a specified frequency and time zone. This can help users to keep their dashboards or alerts updated with the latest data. To schedule a query, users need to do the

  following steps:

  In the Query Editor, click Schedule > Add schedule to open a menu with schedule settings.

  Choose when to run the query. Use the dropdown pickers to specify the frequency, period, starting time, and time zone. Optionally, select the Show cron syntax checkbox to edit the schedule in Quartz Cron Syntax. Choose More options to

  show optional settings. Users can also choose a name for the schedule, and a SQL warehouse to power the query. Click Create. The query will run automatically according to the schedule. The other options are incorrect because they do not

  refer to the correct location or frequency to schedule the query. The query's page in Databricks SQL is the place where users can edit, run, or schedule the query. The SQL endpoint's page in Databricks SQL is the place where users can

  manage the SQL warehouses and SQL endpoints. The Jobs UI is the place where users can create, run, or schedule jobs that execute notebooks, JARs, or Python scripts.

  References: Schedule a query, What are Databricks SQL alerts?,

  Jobs.

• Question 129:

  A data engineer wants to create a relational object by pulling data from two tables. The relational object does not need to be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to avoid copying and storing physical data.

  Which of the following relational objects should the data engineer create?

  A. Spark SQL Table
  B. View
  C. Database
  D. Temporary view
  E. Delta Table
  D. Temporary view

  ---

  Explanation

  A temporary view is a relational object that is defined in the metastore and points to an existing DataFrame. It does not copy or store any physical data, but only saves the query that defines the view. The lifetime of a temporary view is tied to the SparkSession that was used to create it, so it does not persist across different sessions or applications. A temporary view is useful for accessing the same data multiple times within the same notebook or session, without incurring additional storage costs. The other options are either materialized (A, E), persistent (B, C), or not relational objects ?

  References: Databricks Documentation - Temporary View, Databricks Community - How do temp views actually work?, Databricks Community - What's the difference between a Global view and a Temp view?, Big Data Programmers - Temporary View in Databricks.

• Question 130:

  A data engineer is managing a data pipeline in Databricks, where multiple Delta tables are used for various transformations. The team wants to track how data flows through the pipeline, including identifying dependencies between Delta tables, notebooks, jobs, and dashboards. The data engineer is utilizing the Unity Catalog lineage feature to monitor this process.

  How does Unity Catalog's data lineage feature support the visualization of relationships between Delta tables, notebooks, jobs, and dashboards?

  A. Unity Catalog lineage visualizes dependencies between Delta tables, notebooks, and jobs, but does not provide column-level tracing or relationships with dashboards.
  B. Unity Catalog lineage only supports visualizing relationships at the table level and does not extend to notebooks, jobs, or dashboards.
  C. Unity Catalog lineage provides an interactive graph that tracks dependencies between tables and notebooks but excludes any job-related dependencies or dashboard visualizations.
  D. Unity Catalog provides an interactive graph that visualizes the dependencies between Delta tables, notebooks, jobs, and dashboards, while also supporting column-level tracking of data transformations.
  B. Unity Catalog lineage only supports visualizing relationships at the table level and does not extend to notebooks, jobs, or dashboards.

  ---

  Explanation

  Unity Catalog lineage provides an interactive graph that shows how data flows across Delta tables, notebooks, jobs, and dashboards, with column-level lineage to trace transformations precisely for governance and auditing.