Databricks DATABRICKS-MACHINE-LEARNING-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-MACHINE-LEARNING-ASSOCIATE Online Questions & Answers

• Question 1:

  A machine learning engineer is trying to perform batch model inference. They want to get predictions using the linear regression model saved at the pathmodel_urifor the DataFramebatch_df.

  batch_dfhas the following schema:

  customer_id STRING

  The machine learning engineer runs the following code block to perform inference onbatch_dfusing the linear regression model atmodel_uri:

  

  In which situation will the machine learning engineer's code block perform the desired inference?

  A. When the Feature Store feature set was logged with the model at model_uri
  B. When all of the features used by the model at model_uri are in a Spark DataFrame in the PySpark
  C. When the model at model_uri only uses customer_id as a feature
  D. This code block will not perform the desired inference in any situation.
  E. When all of the features used by the model at model_uri are in a single Feature Store table
  A. When the Feature Store feature set was logged with the model at model_uri

  ---

  Explanation/Reference:

  The code block provided by the machine learning engineer will perform the desired inference when the Feature Store feature set was logged with the model at model_uri. This ensures that all necessary feature transformations and metadata

  are available for the model to make predictions. The Feature Store in Databricks allows for seamless integration of features and models, ensuring that the required features are correctly used during inference.

  References:

  Databricks documentation on Feature Store: Feature Store in Databricks

• Question 2:

  A data scientist is using MLflow to track their machine learning experiment. As a part of each of their MLflow runs, they are performing hyperparameter tuning. The data scientist would like to have one parent run for the tuning process with a child run for each unique combination of hyperparameter values. All parent and child runs are being manually started with mlflow.start_run.

  Which of the following approaches can the data scientist use to accomplish this MLflow run organization?

  A. Theycan turn on Databricks Autologging
  B. Theycan specify nested=True when startingthe child run for each unique combination of hyperparameter values
  C. Theycan start each child run inside the parentrun's indented code block usingmlflow.start runO
  D. They can start each child run with the same experiment ID as the parent run
  E. They can specify nested=True when starting the parent run for the tuningprocess
  B. Theycan specify nested=True when startingthe child run for each unique combination of hyperparameter values

  ---

  Explanation/Reference:

  To organize MLflow runs with one parent run for the tuning process and a child run for each unique combination of hyperparameter values, the data scientist can specifynested=Truewhen starting the child run. This approach ensures that

  each child run is properly nested under the parent run, maintaining a clear hierarchical structure for the experiment. This nesting helps in tracking and comparing different hyperparameter combinations within the same tuning

  process.References:

  MLflow Documentation (Managing Nested Runs).

• Question 3:

  A data scientist wants to use Spark ML to impute missing values in their PySpark DataFrame features_df. They want to replace missing values in all numeric columns in features_df with each respective numeric column's median value.

  They have developed the following code block to accomplish this task:

  

  The code block is not accomplishing the task.

  Which reasons describes why the code block is not accomplishing the imputation task?

  A. It does not impute both the training and test data sets.
  B. The inputCols and outputCols need to be exactly the same.
  C. The fit method needs to be called instead of transform.
  D. It does not fit the imputer on the data to create an ImputerModel.
  D. It does not fit the imputer on the data to create an ImputerModel.

  ---

  Explanation/Reference:

  In the provided code block, theImputerobject is created but not fitted on the data to generate anImputerModel. Thetransformmethod is being called directly on the Imputerobject, which does not yet contain the fitted median values needed for imputation. The correct approach is to fit the imputer on the dataset first. Corrected code: imputer = Imputer( strategy="median", inputCols=input_columns, outputCols=output_columns ) imputer_model = imputer.fit(features_df)# Fit the imputer to the dataimputed_features_df = imputer_model.transform(features_df)# Transform the data using the fitted imputer References: PySpark ML Documentation

• Question 4:

  Which of the Spark operations can be used to randomly split a Spark DataFrame into a training DataFrame and a test DataFrame for downstream use?

  A. TrainValidationSplit
  B. DataFrame.where
  C. CrossValidator
  D. TrainValidationSplitModel
  E. DataFrame.randomSplit
  E. DataFrame.randomSplit

  ---

  Explanation/Reference:

  The correct method to randomly split a Spark DataFrame into training and test sets is by using therandomSplitmethod. This method allows you to specify the proportions for the split as a list of weights and returns multiple DataFrames

  according to those weights. This is directly intended for splitting DataFrames randomly and is the appropriate choice for preparing data for training and testing in machine learning workflows.References:

  Apache Spark DataFrame API documentation (DataFrame Operations:randomSplit).

• Question 5:

  A data scientist is wanting to explore the Spark DataFrame spark_df. The data scientist wants visual histograms displaying the distribution of numeric features to be included in the exploration.

  Which of the following lines of code can the data scientist run to accomplish the task?

  A. spark_df.describe()
  B. dbutils.data(spark_df).summarize()
  C. This task cannot be accomplished in a single line of code.
  D. spark_df.summary()
  E. dbutils.data.summarize (spark_df)
  E. dbutils.data.summarize (spark_df)

  ---

  Explanation/Reference:

  To display visual histograms and summaries of the numeric features in a Spark DataFrame, the Databricks utility functiondbutils.data.summarizecan be used. This function provides a comprehensive summary, including visual histograms.

  Correct code:

  dbutils.data.summarize(spark_df)

  Other options likespark_df.describe()andspark_df.summary()provide textual statistical summaries but do not include visual histograms.

  References:

  Databricks Utilities Documentation

• Question 6:

  A data scientist has created a linear regression model that useslog(price)as a label variable. Using this model, they have performed inference and the predictions and actual label values are in Spark DataFramepreds_df.

  They are using the following code block to evaluate the model:

  regression_evaluator.setMetricName("rmse").evaluate(preds_df)

  Which of the following changes should the data scientist make to evaluate the RMSE in a way that is comparable withprice?

  A. They should exponentiate the computed RMSE value
  B. They should take the log of the predictions before computing the RMSE
  C. They should evaluate the MSE of the log predictions to compute the RMSE
  D. They should exponentiate the predictions before computing the RMSE
  D. They should exponentiate the predictions before computing the RMSE

  ---

  Explanation/Reference:

  When evaluating the RMSE for a model that predicts log-transformed prices, the predictions need to be transformed back to the original scale to obtain an RMSE that is comparable with the actual price values. This is done by exponentiating

  the predictions before computing the RMSE. The RMSE should be computed on the same scale as the original data to provide a meaningful measure of error.

  References:

  Databricks documentation on regression evaluation: Regression Evaluation

• Question 7:

  A data scientist is developing a machine learning pipeline using AutoML on Databricks Machine Learning.

  Which of the following steps will the data scientist need to perform outside of their AutoML experiment?

  A. Model tuning
  B. Model evaluation
  C. Model deployment
  D. Exploratory data analysis
  D. Exploratory data analysis

  ---

  Explanation/Reference:

  AutoML platforms, such as the one available in Databricks Machine Learning, streamline various stages of the machine learning pipeline including feature engineering, model selection, hyperparameter tuning, and model evaluation. However,

  exploratory data analysis (EDA) is typically performed outside the AutoML process. EDA involves understanding the dataset, visualizing distributions, identifying anomalies, and gaining insights into data before feeding it into a machine

  learning pipeline. This step is crucial for ensuring that the data is clean and suitable for model training but is generally done manually by the data scientist.

  References:

  Databricks documentation on AutoML:

  https://docs.databricks.com/applications/machine-learning/automl.html

• Question 8:

  A data scientist wants to efficiently tune the hyperparameters of a scikit-learn model in parallel. They elect to use the Hyperopt library to facilitate this process.

  Which of the following Hyperopt tools provides the ability to optimize hyperparameters in parallel?

  A. fmin
  B. SparkTrials
  C. quniform
  D. search_space
  E. objective_function
  B. SparkTrials

  ---

  Explanation/Reference:

  TheSparkTrialsclass in the Hyperopt library allows for parallel hyperparameter optimization on a Spark cluster. This enables efficient tuning of hyperparameters by distributing the optimization process across multiple nodes in a cluster.

  fromhyperoptimportfmin, tpe, hp, SparkTrials search_space = {'x': hp.uniform('x',0,1),'y':

  hp.uniform('y',0,1) }defobjective(params):returnparams['x'] **2+ params['y'] **2spark_trials = SparkTrials(parallelism=4) best = fmin(fn=objective, space=search_space, algo=tpe.suggest, max_evals=100, trials=spark_trials) References:

  Hyperopt Documentation

• Question 9:

  A data scientist has replaced missing values in their feature set with each respective feature variable's median value. A colleague suggests that the data scientist is throwing away valuable information by doing this.

  Which of the following approaches can they take to include as much information as possible in the feature set?

  A. Impute the missing values using each respective feature variable's mean value instead of the median value
  B. Refrain from imputing the missing values in favor of letting the machine learning algorithm determine how to handle them
  C. Remove all feature variables that originally contained missing values from the feature set
  D. Create a binary feature variable for each feature that contained missing values indicating whether each row's value has been imputed
  E. Create a constant feature variable for each feature that contained missing values indicating the percentage of rows from the feature that was originally missing
  D. Create a binary feature variable for each feature that contained missing values indicating whether each row's value has been imputed

  ---

  Explanation/Reference:

  By creating a binary feature variable for each feature with missing values to indicate whether a value has been imputed, the data scientist can preserve information about the original state of the data. This approach maintains the integrity of

  the dataset by marking which values are original and which are synthetic (imputed). Here are the steps to implement this approach:

  Identify Missing Values:Determine which features contain missing values. Impute Missing Values:Continue with median imputation or choose another method (mean, mode, regression, etc.) to fill missing values. Create Indicator Variables:For

  each feature that had missing values, add a new binary feature. This feature should be '1' if the original value was missing and imputed, and '0' otherwise.

  Data Integration:Integrate these new binary features into the existing dataset. This maintains a record of where data imputation occurred, allowing models to potentially weight these observations differently. Model Adjustment:Adjust machine

  learning models to account for these new features, which might involve considering interactions between these binary indicators and other features.

  References:

  "Feature Engineering for Machine Learning" by Alice Zheng and Amanda Casari (O'Reilly Media, 2018), especially the sections on handling missing data. Scikit-learn documentation on imputing missing values: https://scikit-learn.org/stable/

  modules/impute.html

• Question 10:

  Which of the following approaches can be used to view the notebook that was run to create an MLflow run?

  A. Open the MLmodel artifact in the MLflow run paqe
  B. Click the "Models" link in the row corresponding to the run in the MLflow experiment paqe
  C. Click the "Source" link in the row corresponding to the run in the MLflow experiment page
  D. Click the "Start Time" link in the row corresponding to the run in the MLflow experiment page
  C. Click the "Source" link in the row corresponding to the run in the MLflow experiment page

  ---

  Explanation/Reference:

  To view the notebook that was run to create an MLflow run, you can click the "Source" link in the row corresponding to the run in the MLflow experiment page. The "Source" link provides a direct reference to the source notebook or script that

  initiated the run, allowing you to review the code and methodology used in the experiment. This feature is particularly useful for reproducibility and for understanding the context of the experiment.

  References:

  MLflow Documentation (Viewing Run Sources and Notebooks).