Databricks DATABRICKS-CERTIFIED-GENERATIVE-AI-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Databricks DATABRICKS-CERTIFIED-GENERATIVE-AI-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 21:

  A Generative Al Engineer is responsible for developing a chatbot to enable their company's internal HelpDesk Call Center team to more quickly find related tickets and provide resolution. While creating the GenAI application work breakdown

  tasks for this project, they realize they need to start planning which data sources (either Unity Catalog volume or Delta table) they could choose for this application. They have collected several candidate data sources for consideration:

  call_rep_history: a Delta table with primary keys representative_id, call_id. This table is maintained to calculate representatives' call resolution from fields call_duration and call start_time.

  transcript Volume: a Unity Catalog Volume of all recordings as a *.wav files, but also a text transcript as *.txt files.

  call_cust_history: a Delta table with primary keys customer_id, cal1_id. This table is maintained to calculate how much internal customers use the HelpDesk to make sure that the charge back model is consistent with actual service use.

  call_detail: a Delta table that includes a snapshot of all call details updated hourly. It includes root_cause and resolution fields, but those fields may be empty for calls that are still active.

  maintenance_schedule ?a Delta table that includes a listing of both HelpDesk application outages as well as planned upcoming maintenance downtimes.

  They need sources that could add context to best identify ticket root cause and resolution.

  Which TWO sources do that? (Choose two.)

  A. call_cust_history
  B. maintenance_schedule
  C. call_rep_history
  D. call_detail
  E. transcript Volume
  D. call_detail
  E. transcript Volume

  ---

  Explanation

  In the context of developing a chatbot for a company's internal HelpDesk Call Center, the key is to select data sources that provide the most contextual and detailed information about the issues being addressed. This includes identifying the

  root cause and suggesting resolutions. The two most appropriate sources from the list are:

  Call Detail (Option D):

  Contents: This Delta table includes a snapshot of all call details updated hourly, featuring essential fields like root_cause and resolution.

  Relevance: The inclusion of root_cause and resolution fields makes this source particularly valuable, as it directly contains the information necessary to understand and resolve the issues discussed in the calls. Even if some records are

  incomplete, the data provided is crucial for a chatbot aimed at speeding up resolution identification.

  Transcript Volume (Option E):

  Contents: This Unity Catalog Volume contains recordings in .wav format and text transcripts in .txt files.

  Relevance: The text transcripts of call recordings can provide in-depth context that the chatbot can analyze to understand the nuances of each issue. The chatbot can use natural language processing techniques to extract themes, identify

  problems, and suggest resolutions based on previous similar interactions documented in the transcripts.

  Why Other Options Are Less Suitable:

  A (Call Cust History): While it provides insights into customer interactions with the HelpDesk, it focuses more on the usage metrics rather than the content of the calls or the issues discussed.

  B (Maintenance Schedule): This data is useful for understanding when services may not be available but does not contribute directly to resolving user issues or identifying root causes.

  C (Call Rep History): Though it offers data on call durations and start times, which could help in assessing performance, it lacks direct information on the issues being resolved.

  Therefore, Call Detail and Transcript Volume are the most relevant data sources for a chatbot designed to assist with identifying and resolving issues in a HelpDesk Call Center setting, as they provide direct and contextual information related

  to customer issues.

• Question 22:

  A Generative AI Engineer is designing a RAG application for answering user questions on technical regulations as they learn a new sport.

  What are the steps needed to build this RAG application and deploy it?

  A. Ingest documents from a source-->Index the documents and saves to Vector Search-->User submits queries against an LLM-->LLM retrieves relevant documents-->Evaluate model-->LLM generates a response-->Deploy it using Model Serving
  B. Ingest documents from a source-->Index the documents and save to Vector Search-->User submits queries against an LLM-->LLM retrieves relevant documents-->LLM generates a response-->Evaluate model-->Deploy it using Model Serving
  C. Ingest documents from a source-->Index the documents and save to Vector Search-->Evaluate model-->Deploy it using Model Serving
  D. User submits queries against an LLM-->Ingest documents from a source-->Index the documents and save to Vector Search-->LLM retrieves relevant documents-->LLM generates a response-->Evaluate model-->Deploy it using Model Serving
  B. Ingest documents from a source-->Index the documents and save to Vector Search-->User submits queries against an LLM-->LLM retrieves relevant documents-->LLM generates a response-->Evaluate model-->Deploy it using Model Serving

  ---

  Explanation

  The Generative AI Engineer needs to follow a methodical pipeline to build and deploy a Retrieval- Augmented Generation (RAG) application. The steps outlined in option accurately reflect this process: B

  Ingest documents from a source: This is the first step, where the engineer collects documents (e.g., technical regulations) that will be used for retrieval when the application answers user questions.

  Index the documents and save to Vector Search: Once the documents are ingested, they need to be embedded using a technique like embeddings (e.g., with a pre-trained model like BERT) and stored in a vector database (such as Pinecone

  or FAISS). This enables fast retrieval based on user queries.

  User submits queries against an LLM: Users interact with the application by submitting their queries.

  These queries will be passed to the LLM.

  LLM retrieves relevant documents: The LLM works with the vector store to retrieve the most relevant documents based on their vector representations.

  LLM generates a response: Using the retrieved documents, the LLM generates a response that is tailored to the user's question.

  Evaluate model: After generating responses, the system must be evaluated to ensure the retrieved documents are relevant and the generated response is accurate. Metrics such as accuracy, relevance, and user satisfaction can be used for

  evaluation.

  Deploy it using Model Serving: Once the RAG pipeline is ready and evaluated, it is deployed using a model-serving platform such as Databricks Model Serving. This enables real-time inference and response generation for users.

  By following these steps, the Generative AI Engineer ensures that the RAG application is both efficient and effective for the task of answering technical regulation questions.

• Question 23:

  A Generative Al Engineer is working with a retail company that wants to enhance its customer experience by automatically handling common customer inquiries. They are working on an LLM-powered Al solution that should improve response times while maintaining a personalized interaction. They want to define the appropriate input and LLM task to do this.

  Which input/output pair will do this?

  A. Input: Customer reviews; Output Group the reviews by users and aggregate per-user average rating, then respond
  B. Input: Customer service chat logs; Output Group the chat logs by users, followed by summarizing each user's interactions, then respond
  C. Input: Customer service chat logs; Output: Find the answers to similar questions and respond with a summary
  D. Input: Customer reviews: Output Classify review sentiment
  C. Input: Customer service chat logs; Output: Find the answers to similar questions and respond with a summary

  ---

  Explanation

• Question 24:

  A Generative Al Engineer would like an LLM to generate formatted JSON from emails. This will require parsing and extracting the following information: order ID, date, and sender email. Here's a sample email:

  

  They will need to write a prompt that will extract the relevant information in JSON format with the highest level of output accuracy.

  Which prompt will do that?

  A. You will receive customer emails and need to extract date, sender email, and order ID. You should return the date, sender email, and order ID information in JSON format.
  B. You will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in JSON format. Here's an example: {"date": "April 16, 2024", "sender_email": "[email protected]", "order_id": "RE987D"}
  C. You will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in a human-readable format.
  D. You will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in JSON format.
  B. You will receive customer emails and need to extract date, sender email, and order ID. Return the extracted information in JSON format. Here's an example: {"date": "April 16, 2024", "sender_email": "[email protected]", "order_id": "RE987D"}

  ---

  Explanation

  Problem Context : The goal is to parse emails to extract certain pieces of information and output this in a structured JSON format. Clarity and specificity in the prompt design will ensure higher accuracy in the LLM' s responses.

  Explanation of Options :

  Option A: Provides a general guideline but lacks an example, which helps an LLM understand the exact format expected.

  Option B: Includes a clear instruction and a specific example of the output format. Providing an example is crucial as it helps set the pattern and format in which the information should be structured, leading to more accurate results.

  Option C: Does not specify that the output should be in JSON format, thus not meeting the requirement.

  Option D: While it correctly asks for JSON format, it lacks an example that would guide the LLM on how to structure the JSON correctly.

  Therefore, Option B is optimal as it not only specifies the required format but also illustrates it with an example, enhancing the likelihood of accurate extraction and formatting by the LLM.

• Question 25:

  A Generative AI Engineer received the following business requirements for an external chatbot.

  The chatbot needs to know what types of questions the user asks and routes to appropriate models to answer the questions. For example, the user might ask about upcoming event details. Another user might ask about purchasing tickets for

  a particular event.

  What is an ideal workflow for such a chatbot?

  A. The chatbot should only look at previous event information
  B. There should be two different chatbots handling different types of user queries.
  C. The chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it's an upcoming event question, send the query to a text-to-SQL model. If it's about ticket purchasing, the customer should be redirected to a payment platform.
  D. The chatbot should only process payments
  C. The chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it's an upcoming event question, send the query to a text-to-SQL model. If it's about ticket purchasing, the customer should be redirected to a payment platform.

  ---

  Explanation

  Problem Context: The chatbot must handle various types of queries and intelligently route them to the appropriate responses or systems.

  Explanation of Options:

  Option A: Limiting the chatbot to only previous event information restricts its utility and does not meet the broader business requirements.

  Option B: Having two separate chatbots could unnecessarily complicate user interaction and increase maintenance overhead.

  Option C: Implementing a multi-step workflow where the chatbot first identifies the type of question and then routes it accordingly is the most efficient and scalable solution. This approach allows the chatbot to handle a variety of queries

  dynamically, improving user experience and operational efficiency.

  Option D: Focusing solely on payments would not satisfy all the specified user interaction needs, such as inquiring about event details.

  Option Coffers a comprehensive workflow that maximizes the chatbot's utility and responsiveness to different user needs, aligning perfectly with the business requirements.

• Question 26:

  A Generative Al Engineer is building a RAG application that answers questions about internal documents for the company SnoPen AI.

  The source documents may contain a significant amount of irrelevant content, such as advertisements, sports news, or entertainment news, or content about other companies.

  Which approach is advisable when building a RAG application to achieve this goal of filtering irrelevant information?

  A. Keep all articles because the RAG application needs to understand non-company content to avoid answering questions about them.
  B. Include in the system prompt that any information it sees will be about SnoPenAI, even if no data filtering is performed.
  C. Include in the system prompt that the application is not supposed to answer any questions unrelated to SnoPen Al.
  D. Consolidate all SnoPen AI related documents into a single chunk in the vector database.
  C. Include in the system prompt that the application is not supposed to answer any questions unrelated to SnoPen Al.

  ---

  Explanation

  In a Retrieval-Augmented Generation (RAG) application built to answer questions about internal documents, especially when the dataset contains irrelevant content, it's crucial to guide the system to focus on the right information. The best way to achieve this is by including a clear instruction in the system prompt (option C).

  System Prompt as Guidance:The system prompt is an effective way to instruct the LLM to limit its focus to SnoPen AI-related content. By clearly specifying that the model should avoid answering questions unrelated to SnoPen AI, you add an

  additional layer of control that helps the model stay on- topic, even if irrelevant content is present in the dataset.

  Why This Approach Works:The prompt acts as a guiding principle for the model, narrowing its focus to specific domains. This prevents the model from generating answers based on irrelevant content, such as advertisements or news

  unrelated to SnoPen AI.

  Why Other Options Are Less Suitable:

  A (Keep All Articles): Retaining all content, including irrelevant materials, without any filtering makes the system prone to generating answers based on unwanted data.

  B (Include in the System Prompt about SnoPen AI): This option doesn't address irrelevant content directly, and without filtering, the model might still retrieve and use irrelevant data.

  D (Consolidating Documents into a Single Chunk): Grouping documents into a single chunk makes the retrieval process less efficient and won't help filter out irrelevant content effectively. Therefore, instructing the system in the prompt not to

  answer questions unrelated to SnoPen AI (option C) is the best approach to ensure the system filters out irrelevant information.

• Question 27:

  A Generative Al Engineer is deciding between using LSH (Locality Sensitive Hashing) and HNSW (Hierarchical Navigable Small World) for indexing their vector database Their top priority is semantic accuracy.

  Which approach should the Generative Al Engineer use to evaluate these two techniques?

  A. Compare the cosine similarities of the embeddings of returned results against those of a representative sample of test inputs
  B. Compare the Bilingual Evaluation Understudy (BLEU) scores of returned results for a representative sample of test inputs
  C. Compare the Recall-Onented-Understudy for Gistmg Evaluation (ROUGE) scores of returned results for a representative sample of test inputs
  D. Compare the Levenshtein distances of returned results against a representative sample of test inputs
  A. Compare the cosine similarities of the embeddings of returned results against those of a representative sample of test inputs

  ---

  Explanation

• Question 28:

  A Generative AI Engineer is building a RAG application that will rely on context retrieved from source documents that are currently in PDF format. These PDFs can contain both text and images. They want to develop a solution using the least amount of lines of code.

  Which Python package should be used to extract the text from the source documents?

  A. flask
  B. beautifulsoup
  C. unstructured
  D. numpy
  C. unstructured

  ---

  Explanation

  Problem Context: The engineer needs to extract text from PDF documents, which may contain both text and images. The goal is to find a Python package that simplifies this task using the least amount of code.

  Explanation of Options:

  Option A: flask: Flask is a web framework for Python, not suitable for processing or extracting content from PDFs.

  Option B: beautifulsoup: Beautiful Soup is designed for parsing HTML and XML documents, not PDFs.

  Option C: unstructured: This Python package is specifically designed to work with unstructured data, including extracting text from PDFs. It provides functionalities to handle various types of content in documents with minimal coding, making it

  ideal for the task.

  Option D: numpy: Numpy is a powerful library for numerical computing in Python and does not provide any tools for text extraction from PDFs.

  Given the requirement, Option C (unstructured) is the most appropriate as it directly addresses the need to efficiently extract text from PDF documents with minimal code.

• Question 29:

  A Generative AI Engineer is tasked with deploying an application that takes advantage of a custom MLflow Pyfunc model to return some interim results. How should they configure the endpoint to pass the secrets and credentials?

  A. Use spark.conf.set ()
  B. Pass variables using the Databricks Feature Store API
  C. Add credentials using environment variables
  D. Pass the secrets in plain text
  C. Add credentials using environment variables

  ---

  Explanation

  Context : Deploying an application that uses an MLflow Pyfunc model involves managing sensitive information such as secrets and credentials securely.

  Explanation of Options :

  Option A: Use spark.conf.set(): While this method can pass configurations within Spark jobs, using it for secrets is not recommended because it may expose them in logs or Spark UI.

  Option B: Pass variables using the Databricks Feature Store API: The Feature Store API is designed for managing features for machine learning, not for handling secrets or credentials.

  Option C: Add credentials using environment variables: This is a common practice for managing credentials in a secure manner, as environment variables can be accessed securely by applications without exposing them in the codebase.

  Option D: Pass the secrets in plain text: This is highly insecure and not recommended, as it exposes sensitive information directly in the code.

  Therefore, Option C is the best method for securely passing secrets and credentials to an application, protecting them from exposure.

• Question 30:

  A Generative Al Engineer needs to design an LLM pipeline to conduct multi-stage reasoning that leverages external tools. To be effective at this, the LLM will need to plan and adapt actions while performing complex reasoning tasks.

  Which approach will do this?

  A. Tram the LLM to generate a single, comprehensive response without interacting with any external tools, relying solely on its pre-trained knowledge.
  B. Implement a framework like ReAct which allows the LLM to generate reasoning traces and perform task-specific actions that leverage external tools if necessary.
  C. Encourage the LLM to make multiple API calls in sequence without planning or structuring the calls, allowing the LLM to decide when and how to use external tools spontaneously.
  D. Use a Chain-of-Thought (CoT) prompting technique to guide the LLM through a series of reasoning steps, then manually input the results from external tools for the final answer.
  B. Implement a framework like ReAct which allows the LLM to generate reasoning traces and perform task-specific actions that leverage external tools if necessary.

  ---

  Explanation