Databricks Databricks Certifications DATABRICKS-MACHINE-LEARNING-ASSOCIATE Questions & Answers

• Question 11:

  A machine learning engineer is trying to perform batch model inference. They want to get predictions using the linear regression model saved at the pathmodel_urifor the DataFramebatch_df.

  batch_dfhas the following schema:

  customer_id STRING

  The machine learning engineer runs the following code block to perform inference onbatch_dfusing the linear regression model atmodel_uri:

  

  In which situation will the machine learning engineer's code block perform the desired inference?

  A. When the Feature Store feature set was logged with the model at model_uri

  B. When all of the features used by the model at model_uri are in a Spark DataFrame in the PySpark

  C. When the model at model_uri only uses customer_id as a feature

  D. This code block will not perform the desired inference in any situation.

  E. When all of the features used by the model at model_uri are in a single Feature Store table

  Correct Answer: A

  The code block provided by the machine learning engineer will perform the desired inference when the Feature Store feature set was logged with the model at model_uri. This ensures that all necessary feature transformations and metadata

  are available for the model to make predictions. The Feature Store in Databricks allows for seamless integration of features and models, ensuring that the required features are correctly used during inference.

  References:

  Databricks documentation on Feature Store: Feature Store in Databricks

• Question 12:

  An organization is developing a feature repository and is electing to one-hot encode all categorical feature variables. A data scientist suggests that the categorical feature variables should not be one-hot encoded within the feature repository.

  Which of the following explanations justifies this suggestion?

  A. One-hot encoding is not supported by most machine learning libraries.

  B. One-hot encoding is dependent on the target variable's values which differ for each application.

  C. One-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.

  D. One-hot encoding is not a common strategy for representing categorical feature variables numerically.

  E. One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.

  Correct Answer: E

  One-hot encoding transforms categorical variables into a format that can be provided to machine learning algorithms to better predict the output. However, when done prematurely or universally within a feature repository, it can be

  problematic:

  Dimensionality Increase:One-hot encoding significantly increases the feature space, especially with high cardinality features, which can lead to high memory consumption and slower computation.

  Model Specificity:Some models handle categorical variables natively (like decision trees and boosting algorithms), and premature one-hot encoding can lead to inefficiency and loss of information (e.g., ordinal relationships). Sparse Matrix

  Issue:It often results in a sparse matrix where most values are zero, which can be inefficient in both storage and computation for some algorithms. Generalization vs. Specificity:Encoding should ideally be tailored to specific models and use

  cases rather than applied generally in a feature repository.

  References:

  "Feature Engineering and Selection: A Practical Approach for Predictive Models" by Max Kuhn and Kjell Johnson (CRC Press, 2019).

• Question 13:

  A data scientist has developed a linear regression model using Spark ML and computed the predictions in a Spark DataFrame preds_df with the following schema:

  prediction DOUBLE actual DOUBLE

  Which of the following code blocks can be used to compute the root mean-squared-error of the model according to the data in preds_df and assign it to the rmse variable?

  

  A. Option A

  B. Option B

  C. Option C

  D. Option D

  Correct Answer: C

  To compute the root mean-squared-error (RMSE) of a linear regression model using Spark ML, theRegressionEvaluatorclass is used. TheRegressionEvaluator is specifically designed for regression tasks and can calculate various metrics,

  including RMSE, based on the columns containing predictions and actual values. The correct code block to compute RMSE from thepreds_dfDataFrame is:

  regression_evaluator = RegressionEvaluator( predictionCol="prediction", labelCol="actual", metricName="rmse") rmse = regression_evaluator.evaluate(preds_df) This code creates an instance ofRegressionEvaluator, specifying the prediction

  and label columns, as well as the metric to be computed ("rmse"). It then evaluates the predictions in preds_dfand assigns the resulting RMSE value to thermsevariable. Options A and B incorrectly useBinaryClassificationEvaluator, which is

  not suitable for regression tasks. Option D also incorrectly usesBinaryClassificationEvaluator.

  References:

  PySpark ML Documentation

• Question 14:

  A data scientist has been given an incomplete notebook from the data engineering team. The notebook uses a Spark DataFrame spark_df on which the data scientist needs to perform further feature engineering. Unfortunately, the data scientist has not yet learned the PySpark DataFrame API.

  Which of the following blocks of code can the data scientist run to be able to use the pandas API on Spark?

  A. import pyspark.pandas as ps df = ps.DataFrame(spark_df)

  B. import pyspark.pandas as ps df = ps.to_pandas(spark_df)

  C. spark_df.to_pandas()

  D. import pandas as pd df = pd.DataFrame(spark_df)

  Correct Answer: A

  To use the pandas API on Spark, the data scientist can run the following code block:

  importpyspark.pandasasps df = ps.DataFrame(spark_df) This code imports the pandas API on Spark and converts the Spark DataFramespark_df into a pandas-on-Spark DataFrame, allowing the data scientist to use familiar pandas functions

  for further feature engineering.

  References:

  Databricks documentation on pandas API on Spark: pandas API on Spark

• Question 15:

  Which of the following machine learning algorithms typically uses bagging?

  A. Gradient boosted trees B. K-means

  C. Random forest

  D. Linear regression

  E. Decision tree

  Correct Answer: C

  Random Forest is a machine learning algorithm that typically uses bagging (Bootstrap Aggregating). Bagging involves training multiple models independently on different random subsets of the data and then combining their predictions.

  Random Forests consist of many decision trees trained on random subsets of the training data and features, and their predictions are averaged to improve accuracy and control overfitting. This method enhances model robustness and

  predictive performance.References:

  Ensemble Methods in Machine Learning (Understanding Bagging and Random Forests).

• Question 16:

  A machine learning engineer is trying to scale a machine learning pipeline by distributing its single-node model tuning process. After broadcasting the entire training data onto each core, each core in the cluster can train one model at a time. Because the tuning process is still running slowly, the engineer wants to increase the level of parallelism from 4 cores to 8 cores to speed up the tuning process. Unfortunately, the total memory in the cluster cannot be increased.

  In which of the following scenarios will increasing the level of parallelism from 4 to 8 speed up the tuning process?

  A. When the tuning process in randomized

  B. When the entire data can fit on each core

  C. When the model is unable to be parallelized

  D. When the data is particularly long in shape

  E. When the data is particularly wide in shape

  Correct Answer: B

  Increasing the level of parallelism from 4 to 8 cores can speed up the tuning process if each core can handle the entire dataset. This ensures that each core can independently work on training a model without running into memory constraints.

  If the entire dataset fits into the memory of each core, adding more cores will allow more models to be trained in parallel, thus speeding up the process.

  References:

  Parallel Computing Concepts

• Question 17:

  A data scientist has written a feature engineering notebook that utilizes the pandas library. As the size of the data processed by the notebook increases, the notebook's runtime is drastically increasing, but it is processing slowly as the size of the data included in the process increases.

  Which of the following tools can the data scientist use to spend the least amount of time refactoring their notebook to scale with big data?

  A. PySpark DataFrame API

  B. pandas API on Spark

  C. Spark SQL

  D. Feature Store

  Correct Answer: B

  The pandas API on Spark provides a way to scale pandas operations to big data while minimizing the need for refactoring existing pandas code. It allows users to run pandas operations on Spark DataFrames, leveraging Spark's distributed

  computing capabilities to handle large datasets more efficiently. This approach requires minimal changes to the existing code, making it a convenient option for scaling pandas-based feature engineering notebooks.

  References:

  Databricks documentation on pandas API on Spark: pandas API on Spark

• Question 18:

  A data scientist is utilizing MLflow Autologging to automatically track their machine learning experiments. After completing a series of runs for the experiment experiment_id, the data scientist wants to identify the run_id of the run with the best root-mean-square error (RMSE).

  Which of the following lines of code can be used to identify the run_id of the run with the best RMSE in experiment_id?

  

  A. Option A

  B. Option B

  C. Option C

  D. Option D

  Correct Answer: C

  To find the run_id of the run with the best root-mean-square error (RMSE) in an MLflow experiment, the correct line of code to use is:

  mlflow.search_runs( experiment_id, order_by=["metrics.rmse"] )["run_id"][0] This line of code searches the runs in the specified experiment, orders them by the RMSE metric in ascending order (the lower the RMSE, the better), and retrieves

  the run_id of the best-performing run. Option C correctly represents this logic.

  References:

  MLflow documentation on tracking experiments:

  https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.search_runs

• Question 19:

  A machine learning engineering team has a Job with three successive tasks. Each task runs a single notebook. The team has been alerted that the Job has failed in its latest run.

  Which of the following approaches can the team use to identify which task is the cause of the failure?

  A. Run each notebook interactively

  B. Review the matrix view in the Job's runs

  C. Migrate the Job to a Delta Live Tables pipeline

  D. Change each Task's setting to use a dedicated cluster

  Correct Answer: B

  To identify which task is causing the failure in the job, the team should review the matrix view in the Job's runs. The matrix view provides a clear and detailed overview of each task's status, allowing the team to quickly identify which task

  failed. This approach ismore efficient than running each notebook interactively, as it provides immediate insights into the job's execution flow and any issues that occurred during the run.

  References:

  Databricks documentation on Jobs: Jobs in Databricks

• Question 20:

  The implementation of linear regression in Spark ML first attempts to solve the linear regression problem using matrix decomposition, but this method does not scale well to large datasets with a large number of variables.

  Which of the following approaches does Spark ML use to distribute the training of a linear regression model for large data?

  A. Logistic regression

  B. Singular value decomposition

  C. Iterative optimization

  D. Least-squares method

  Correct Answer: C

  For large datasets, Spark ML uses iterative optimization methods to distribute the training of a linear regression model. Specifically, Spark MLlib employs techniques like Stochastic Gradient Descent (SGD) and Limited-memory Broyden