ISTQB ISTQB-CT-AI Online Practice Questions and Exam Preparation

ISTQB ISTQB-CT-AI Online Questions & Answers

• Question 71:

  Consider a natural language processing (NLP) algorithm that attempts to predict the next word that you would like to type in a text message. An update to the algorithm has been created that should increase the accuracy of the predictions based on user typing patterns. The old algorithm was rated for accuracy by the users. Then, after the new update was released, the users rated the updated algorithm. A statistical test was used to compare the two versions of the algorithm to determine whether the update should remain in place.

  This is an example of what type of testing?

  A. Metamorphic testing
  B. A/B testing
  C. Exploratory testing
  D. Pairwise testing
  B. A/B testing

  ---

  The ISTQB CT-AI syllabus states that A/B testing can be used to evaluate updates to an AI-based system when there are agreed acceptance criteria, such as machine learning functional performance metrics. In this approach, an updated variant of the system is statistically compared with a previous variant to determine whether the change provides a measurable improvement. This scenario exactly matches that description: users rate the accuracy of both the old and the updated NLP algorithm, and a statistical test is used to compare the results to decide whether the update should be retained. Therefore, this is an example of A/B testing.

• Question 72:

  Before deployment of an AI-based system, a developer is expected to demonstrate in a test environment how decisions are made. Which of the following characteristics does decision making fall under?

  A. Explainability
  B. Autonomy
  C. Self-learning
  D. Non-determinism
  A. Explainability

  ---

  The ISTQB CT-AI syllabus defines explainability as the ability to understand and describe how an AI-based system arrives at a particular result or decision. Demonstrating how decisions are made before deployment directly relates to making the decision logic and reasoning of the system understandable to stakeholders, testers, and regulators.

  The other options do not apply. Autonomy refers to the system's ability to act without human intervention, self-learning concerns the system's capability to adapt based on data, and non-determinism relates to variability in outputs. None of these specifically address explaining decision logic.

  Therefore, Explainability is the correct answer.

• Question 73:

  "BioSearch" is creating an AI model used for predicting cancer occurrence by examining X-ray images. The accuracy of the model in isolation has been found to be good. However, users of the model started complaining about the poor quality of results---especially the inability to detect real cancer cases---when it

  was put into practice in the diagnosis lab, leading to the discontinuation of the model's usage.

  A testing expert was called in to identify deficiencies in the test planning that led to this situation.

  Which ONE of the following options would you expect to be MOST LIKELY identified by the test expert?

  A. A lack of similarity between the training and testing data.
  B. The input data was not tested for quality prior to being used for testing.
  C. A lack of focus on choosing the right functional performance metrics.
  D. A lack of focus on non-functional requirements testing.
  A. A lack of similarity between the training and testing data.

  ---

  The scenario describes a model that shows good accuracy in isolation but performs poorly in real-world operation, particularly in detecting true cancer cases. This strongly indicates a generalization problem.

  A lack of similarity between the training and testing data means that the training data was not sufficiently representative of real-world diagnostic data. This is a well-known ML risk that results in models performing well during development but failing in operational environments.

  While input data quality (Option B) is important, the issue described is not primarily about corrupted or invalid data but about representativeness.

  Choosing inappropriate functional performance metrics (Option C) could hide problems, but it would not by itself cause poor real-world detection if the data were representative.

  Non-functional requirements testing (Option D) is important but does not directly explain the inability to detect real cancer cases.

  The ISTQB CT-AI syllabus (Section 4.2 - Training, Validation, and Test Datasets) emphasizes that a lack of representative data across training, validation, and operational environments is a major cause of poor real-world ML performance.

  Therefore, Option A is the most likely deficiency discovered by the test expert.

• Question 74:

  Which ONE of the following tests is LEAST likely to be performed during the ML model testing phase?

  A. Testing the accuracy of the classification model.
  B. Testing the API of the service powered by the ML model.
  C. Testing the speed of the training of the model.
  D. Testing the speed of the prediction by the model.
  C. Testing the speed of the training of the model.

  ---

  The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option:

  Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics.

  Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected.

  Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained.

  Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real- time applications.

  ISTQB CT-AI Syllabus Section 3.2 on ML Workflow and Section 5 on ML Functional Performance Metrics discuss the focus of testing during the model testing phase, which includes accuracy and prediction speed but not the training speed.

• Question 75:

  Which ONE of the following hardware is MOST suitable for implementing AI when using machine learning (ML)?

  A. 64-bit CPUs
  B. Hardware supporting fast matrix multiplication
  C. High-powered CPUs
  D. Hardware supporting high-precision floating-point operations
  B. Hardware supporting fast matrix multiplication

  ---

  The ISTQB CT-AI syllabus explains that many machine learning algorithms---especially neural networks and deep learning---are heavily based on matrix and vector operations. As a result, hardware that supports fast matrix multiplication is particularly well suited for AI and ML workloads.

  Option B is correct because specialized hardware such as GPUs and tensor processing units (TPUs) is designed to perform large-scale parallel matrix operations efficiently, which significantly accelerates training and inference of ML models.

  Option A is incorrect because 64-bit CPUs, while important for addressing large memory spaces, are not specifically optimized for the parallel numerical computations required by ML algorithms. Option C is incorrect because high-powered CPUs are general-purpose and typically less efficient than specialized accelerators for large-scale matrix operations. Option D is incorrect because although high-precision floating-point operations are relevant in some domains, ML workloads usually benefit more from

  speed and parallelism than from very high numerical precision.

  Therefore, hardware supporting fast matrix multiplication (Option B) is the most suitable choice for implementing AI systems based on machine learning.

• Question 76:

  Which statement about testing levels for AI-based systems is correct?

  A. Input data testing checks whether the inputs from the data pipeline are received by the model correctly and exchanged with all system components.
  B. Acceptance testing checks non-functional requirements such as explainability.
  C. ML model testing ensures that the relevant ML functional performance criteria are met.
  D. If AI is offered as a service, system testing includes API tests of the service.
  C. ML model testing ensures that the relevant ML functional performance criteria are met.

  ---

  Section 4.3 - Test Levels for AI Systems clearly defines ML model testing as the level at which testers evaluate whether a machine learning model fulfills its functional performance criteria, including accuracy, precision, recall, F1-score, robustness, stability, and fairness. Therefore, Option C is the correct and syllabus-aligned statement.

  Option A is incorrect because input data testing focuses on the validity and correctness of data entering the model, not on interactions with all system components. Option B is incorrect because acceptance testing in the syllabus focuses primarily on business and stakeholder requirements, not specifically on explainability; explainability testing may occur at multiple test levels depending on context. Option D is incorrect because API testing belongs to integration testing rather than system testing, even when

  AI functionality is consumed as a service.

  Thus, Option C is the only statement that precisely matches the syllabus definitions.

• Question 77:

  Which assignment of AI techniques to testing support is BEST?

  A. Classification for the optimization of regression test cases
  B. Probabilistic methods for the prediction of system failures
  C. Fuzzy logic for the generation of test cases
  D. Computational optimization techniques for defect prediction
  B. Probabilistic methods for the prediction of system failures

  ---

  The ISTQB CT-AI syllabus (Section 5.2 - AI for Testing) explains that various AI approaches can support testing activities. Probabilistic methods - one of the three major AI technique groups --- are used to predict system failures, especially when dealing with uncertainty, likelihood estimation, and reliability analysis. This aligns precisely with Option B.

  Option A is incorrect because regression test optimization is typically performed using search-based or optimization techniques rather than classification. Option C is incorrect because fuzzy logic is mainly suited to reasoning under vagueness and imprecision, not to the generation of test cases. Option D is incorrect because defect prediction commonly relies on statistical learning or classification models rather than computational optimization techniques.

  Thus, Option B is the most syllabus-consistent mapping of AI techniques to testing tasks.

• Question 78:

  Which ONE of the following characteristics is the least likely to cause safety related issues for an Al system?

  A. Non-determinism
  B. Robustness
  C. High complexity
  D. Self-learning
  B. Robustness

  ---

  The question asks which characteristic is least likely to cause safety-related issues for an AI system. Let's evaluate each option:

  Non-determinism (A): Non-deterministic systems can produce different outcomes even with the same inputs, which can lead to unpredictable behavior and potential safety issues.

  Robustness (B): Robustness refers to the ability of the system to handle errors, anomalies, and unexpected inputs gracefully. A robust system is less likely to cause safety issues because it can maintain functionality under varied conditions.

  High complexity (C): High complexity in AI systems can lead to difficulties in understanding, predicting, and managing the system's behavior, which can cause safety-related issues.

  Self-learning (D): Self-learning systems adapt based on new data, which can lead to unexpected changes in behavior. If not properly monitored and controlled, this can result in safety issues.

  ISTQB CT-AI Syllabus Section 2.8 on Safety and AI discusses various factors affecting the safety of AI systems, emphasizing the importance of robustness in maintaining safe operation.

• Question 79:

  A bank wants to use an algorithm to determine which applicants should be given a loan. The bank hires a data scientist to construct a logistic regression model to predict whether an applicant will repay the loan or not. The bank has enough data on past customers to randomly split the data into a training data set and a test/validation data set. A logistic regression model is constructed on the training data set using the following independent variables:

  Gender Marital status Number of dependents Education Income Loan amount Loan term Credit score

  The model reveals that applicants with higher credit scores and higher total incomes are more likely to repay their loans. The data scientist has suggested that there might be bias present in the model based on previous models created for other banks.

  Given this information, what is the best test approach to check for potential bias in the model?

  A. Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.
  B. Back-to-back testing should be used to compare the model created using the training data set with another model created using the test data set. If the two models significantly differ, it will indicate that there is bias in the original model.
  C. Acceptance testing should be used to ensure that the algorithm is suitable for customers. The team can rework the acceptance criteria so that the algorithm correctly predicts the remaining applicants in the validation data set, ensuring that no bias is present.
  D. A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate that there is bias in the original model.
  A. Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set.

  ---

  The ISTQB CT-AI syllabus explains that experience-based testing and exploratory data analysis (EDA) are effective techniques for identifying potential bias in training data and models. Experience-based testing can be used to verify that the training data set is operationally relevant and representative of the real-world population. EDA supports this by enabling testers to explore distributions, correlations, and patterns in the data that may reveal imbalances or relationships leading to bias.

  This directly supports Option A, which combines experience-based testing with EDA to assess bias in the training data and, by extension, in the resulting model.

  The other options are incorrect. Back-to-back testing and A/B testing are not appropriate methods for detecting bias in training data; differences between models do not necessarily indicate bias. Acceptance testing focuses on stakeholder requirements and suitability, not on identifying bias in data or models.

• Question 80:

  Which statement regarding testing transparency, explainability, or interpretability is MOST correct?

  A. Tests for explainability and transparency are comparable to exploratory testing and can be performed with little information about development.
  B. Since different users have different backgrounds, interpretability testing depends on the comprehensibility of the ML algorithm.
  C. Dynamic testing is one way to quantify explainability; however, each method is specific to a particular model type.
  D. LIME can precisely state the decisive reason for a change in the output.
  B. Since different users have different backgrounds, interpretability testing depends on the comprehensibility of the ML algorithm.

  ---

  The ISTQB CT-AI syllabus states in Section 2.10 - Explainability, Transparency, and Interpretability that interpretability is user-dependent, meaning that different users may understand explanations in different ways. This is because interpretability depends not only on the machine learning algorithm but also on the user's domain knowledge, experience, and expectations. Option B directly reflects this syllabus principle: interpretability testing must take user background into account, and explanations must be comprehensible to the intended user group.

  Option A is incorrect because testing for explainability and transparency typically requires substantial information about the model, data, and expected behavior, and is not comparable to exploratory testing performed with little prior knowledge. Option C is incorrect because explainability is not generally quantifiable through dynamic testing alone, and the syllabus does not claim that explainability methods are strictly tied to specific model types in this manner. Option D is incorrect because LIME provides approximate, local explanations and cannot precisely identify the decisive root cause of output changes; the syllabus emphasizes that such explanations are approximations with inherent limitations.

  Therefore, Option B is the most syllabus-aligned and correct statement.