Databricks Databricks Certifications DATABRICKS-MACHINE-LEARNING-ASSOCIATE Questions & Answers

• Question 71:

  Which of the following statements describes a Spark ML estimator?

  A. An estimator is a hyperparameter arid that can be used to train a model

  B. An estimator chains multiple alqorithms toqether to specify an ML workflow

  C. An estimator is a trained ML model which turns a DataFrame with features into a DataFrame with predictions

  D. An estimator is an alqorithm which can be fit on a DataFrame to produce a Transformer

  E. An estimator is an evaluation tool to assess to the quality of a model

  Correct Answer: D

  In the context of Spark MLlib, an estimator refers to an algorithm which can be "fit" on a DataFrame to produce a model (referred to as a Transformer), which can then be used to transform one DataFrame into another, typically adding

  predictions or model scores. This is a fundamental concept in machine learning pipelines in Spark, where the workflow includes fitting estimators to data to produce transformers.

  References:

  Spark MLlib Documentation:https://spark.apache.org/docs/latest/ml-pipeline.html#estimators

• Question 72:

  Which of the following hyperparameter optimization methods automatically makes informed selections of hyperparameter values based on previous trials for each iterative model evaluation?

  A. Random Search

  B. Halving Random Search

  C. Tree of Parzen Estimators

  D. Grid Search

  Correct Answer: C

  Tree of Parzen Estimators (TPE) is a sequential model-based optimization algorithm that selects hyperparameter values based on the outcomes of previous trials. It models the probability density of good and bad hyperparameter values and

  makes informed decisions about which hyperparameters to try next. This approach contrasts with methods like random search and grid search, which do not use information from previous trials to guide the search process.

  References:

  Hyperopt and TPE

• Question 73:

  A data scientist wants to use Spark ML to one-hot encode the categorical features in their PySpark DataFramefeatures_df. A list of the names of the string columns is assigned to theinput_columnsvariable.

  They have developed this code block to accomplish this task:

  

  The code block is returning an error.

  Which of the following adjustments does the data scientist need to make to accomplish this task?

  A. They need to specify the method parameter to the OneHotEncoder.

  B. They need to remove the line with the fit operation.

  C. They need to use Stringlndexer prior to one-hot encodinq the features.

  D. They need to useVectorAssemblerprior to one-hot encoding the features.

  Correct Answer: C

  TheOneHotEncoderin Spark ML requires numerical indices as inputs rather than string labels. Therefore, you need to first convert the string columns to numerical indices usingStringIndexer. After that, you can applyOneHotEncoderto these

  indices.

  Corrected code:

  frompyspark.ml.featureimportStringIndexer, OneHotEncoder# Convert string column to indexindexers = [StringIndexer(inputCol=col,

  outputCol=col+"_index")forcolininput_columns] indexer_model = Pipeline(stages=indexers).fit(features_df) indexed_features_df = indexer_model.transform(features_df)# One-hot encode the indexed columnsohe = OneHotEncoder

  (inputCols=[col+"_index"forcolininput_columns], outputCols=output_columns) ohe_model = ohe.fit(indexed_features_df) ohe_features_df = ohe_model.transform(indexed_features_df)

  References:

  PySpark ML Documentation

• Question 74:

  A machine learning engineer has been notified that a new Staging version of a model registered to the MLflow Model Registry has passed all tests. As a result, the machine learning engineer wants to put this model into production by transitioning it to the Production stage in the Model Registry.

  From which of the following pages in Databricks Machine Learning can the machine learning engineer accomplish this task?

  A. The home page of the MLflow Model Registry

  B. The experiment page in the Experiments observatory

  C. The model version page in the MLflow ModelRegistry

  D. The model page in the MLflow Model Registry

  Correct Answer: C

  The machine learning engineer can transition a model version to the Production stage in the Model Registry from the model version page. This page provides detailed information about a specific version of a model, including its metrics,

  parameters, and current stage. From here, the engineer can perform stage transitions, moving the model from Staging to Production after it has passed all necessary tests.

  References:

  Databricks documentation on MLflow Model Registry:

  https://docs.databricks.com/applications/mlflow/model-registry.html#model-version