Free DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Web TestEngine Demo

Q15 Single choice

A data engineer is building a PySpark ingestion pipeline in a local IDE and must execute heavy DataFrame transformations on a remote Databricks cluster for scale while stepping through business logic locally. The workspace uses Unity Catalog and a cluster running Databricks Runtime 15.4 LTS. The engineer also plans to register UDFs.

What should the data engineer do to avoid runtime issues during development with Databricks Connect?

A
Match the local Python minor version to the cluster's Python minor version when using UDF's, and use a Databricks Connect package compatible with the cluster's Runtime version
B
Use any local Python version as long as the PySpark minor version matches, because the cluster's Python version is isolated from the client.
C
Use serverless, because Databricks Connect does not support assigned or shared clusters.
D
Disable Unity Catalog on the workspace because it conflicts with remote execution through Databricks Connect.

DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Web TestEngine demo