ISTQB ISTQB-CT-AI Online Practice Questions and Exam Preparation

ISTQB ISTQB-CT-AI Online Questions & Answers

• Question 1:

  Which ONE of the following statements correctly describes the importance of flexibility for AI systems?

  A. AI systems are inherently flexible.
  B. AI systems require changing operational environments; therefore, flexibility is required.
  C. Flexible AI systems allow for easier modification of the system as a whole.
  D. Self-learning systems are expected to deal with new situations without explicitly having to be programmed for them.
  C. Flexible AI systems allow for easier modification of the system as a whole.

  ---

  The ISTQB CT-AI syllabus defines flexibility as the ease with which a system can be modified to accommodate changes, such as new requirements, constraints, or operating conditions. Flexibility supports maintainability and evolution of AI systems over time.

  Option C correctly captures this concept by stating that flexible AI systems allow for easier modification of the system as a whole.

  The other options are incorrect. AI systems are not inherently flexible; flexibility depends on design choices. The need to operate in changing environments does not by itself define flexibility. Dealing with new situations without explicit programming describes adaptability, especially in self-learning systems, not flexibility.

  Therefore, Option C is the correct answer.

• Question 2:

  Upon testing a model used to detect rotten tomatoes, the following data was observed by the test engineer, based on certain number of tomato images.

  

  For this confusion matrix which combinations of values of accuracy, recall, and specificity respectively is CORRECT?

  A. 0.87, 0.9, 0.84
  B. 1, 0.87, 0.84
  C. 1, 0.9, 0.8
  D. 0.84, 1, 0.9
  A. 0.87, 0.9, 0.84

  ---

  To calculate accuracy, recall, and specificity from the confusion matrix, the following values are used: Confusion Matrix Actually Rotten: True Positive (TP) = 45 False Positive (FP) = 8 Actually Fresh: False Negative (FN) = 5 True Negative (TN) = 42 Accuracy Accuracy is the proportion of all correct predictions. Formula:

  Accuracy = (TP + TN) / (TP + TN + FP + FN)

  Calculation:

  Accuracy = (45 + 42) / (45 + 42 + 8 + 5)

  Accuracy = 87 / 100

  Accuracy = 0.87

  Recall (Sensitivity)

  Recall is the proportion of actual positives correctly identified.

  Formula:

  Recall = TP / (TP + FN)

  Calculation:

  Recall = 45 / (45 + 5)

  Recall = 45 / 50

  Recall = 0.9

  Specificity

  Specificity is the proportion of actual negatives correctly identified.

  Formula:

  Specificity = TN / (TN + FP)

  Calculation:

  Specificity = 42 / (42 + 8)

  Specificity = 42 / 50

  Specificity = 0.84

  Therefore, the correct combination of accuracy, recall, and specificity is:

  0.87, 0.9, 0.84

  Reference

  ISTQB CT-AI Syllabus v1.0, Section 5.1 - Confusion Matrix and ML Functional Performance Metrics.

• Question 3:

  You are developing a "flower" ML model. Which of the following describes an objection that you can NEGLECT in your risk assessment?

  A. The possible inputs for the "leaf" and "flower" ML models are so different that reuse has few advantages over new development.
  B. The probability of misclassification of the ML model "flower" is higher when it is reused than when it is developed from scratch.
  C. The classification behavior of the "flower" ML model is more difficult to understand when it is reused compared to when it is developed from scratch.
  D. The possible outputs of the "leaf" and "flower" ML models are so different that reuse has few advantages over new development.
  D. The possible outputs of the "leaf" and "flower" ML models are so different that reuse has few advantages over new development.

  ---

  The ISTQB CT-AI syllabus explains that reusing pre-trained models is strongly related to the similarity between the original task and the new task. Section 1.8 - Pre-trained Models and Transfer Learning states that reuse is effective when the new task is similar to the original one, such as adapting a cat classifier to classify dog breeds. The syllabus highlights risks related to differences in input data, data preparation inconsistencies, inherited shortcomings, and explainability issues. These are legitimate

  objections, corresponding to Options A, B, and C, because large differences in input characteristics can undermine transfer learning, the risk of misclassification may increase, and explainability often decreases when reusing pre-trained models.

  However, differences in outputs are not a valid concern in this context. Both the "leaf" and "flower" ML models classify the same plant species, meaning their output classes are identical. The syllabus does not identify output differences as a transfer learning risk when the label space remains the same. Instead, the primary risks relate to inputs, bias inheritance, model transparency, and training differences.

  Therefore, Option D describes an objection that can be safely neglected in the risk assessment.

• Question 4:

  Which performance metric is BEST suited to assess the quality of trained models detecting fraudulent credit card transactions?

  A. Sensitivity
  B. Accuracy
  C. F1 value
  D. --
  C. F1 value

  ---

  The ISTQB CT-AI syllabus explains in Section 3.2 - Functional Performance Criteria of ML Models that accuracy becomes unreliable when class imbalance exists. In fraud detection, more than 99% of transactions are non-fraudulent, meaning the data set is extremely imbalanced. Because accuracy counts all correctly classified non-fraudulent transactions, it can appear artificially high even when the fraud detection performance is poor. Therefore, accuracy is not suitable for evaluating fraud detection

  systems.

  The syllabus further explains that sensitivity (recall) measures the proportion of correctly identified fraudulent transactions. This metric is important because missed fraudulent events can lead to significant financial loss. However, the client also requires that legitimate transactions be correctly identified, meaning false positives must be minimized to maintain customer satisfaction.

  The F1 score, defined as the harmonic mean of precision and recall, balances both aspects: precision limits false alarms affecting legitimate customers, while recall ensures that fraudulent transactions are detected. Section 3.2 emphasizes that when both false positives and false negatives have serious consequences and the data is highly imbalanced, the F1 value is the most appropriate metric.

  Thus, Option C is the correct choice.

• Question 5:

  How can a tester check the system for bias as part of a review of data sources, acquisition, and preprocessing?

  A. During the review, it can uncover algorithmic bias by analysing the procedures used to obtain the training data.
  B. During the review of the preprocessing, the auditor can uncover whether the data has been influenced in a way that could lead to sample distortions.
  C. It may use the LIME method as part of its data collection review to detect inappropriate bias.
  D. As part of the review of preprocessing, it can reveal whether the data has been influenced in a way that could lead to algorithmic bias.
  B. During the review of the preprocessing, the auditor can uncover whether the data has been influenced in a way that could lead to sample distortions.

  ---

  Bias detection at the data level is performed by reviewing data acquisition and preprocessing steps, as explained in Section 2.3 - Data Quality and Bias of the ISTQB CT-AI syllabus. Sample bias can arise when data is distorted or when preprocessing introduces unintended shifts, for example through filtering, normalization, or labeling steps that disproportionately affect certain subsets of the data. Option B correctly reflects this approach: during preprocessing review, testers can identify whether the data has

  been influenced in a way that could lead to sample distortions.

  Option A is incorrect because algorithmic bias originates from the model and its learning process, not directly from data acquisition procedures. Option C is incorrect because LIME is an explainability technique applied to trained models, not a method used during data source or preprocessing reviews. Option D is incorrect because preprocessing affects sample bias rather than algorithmic bias.

  Thus, Option B correctly matches the syllabus guidance on detecting bias during reviews of data sources, acquisition, and preprocessing.

• Question 6:

  Which statement about AI-based test case generation is correct?

  A. AI-generated functional test cases typically result in poor requirements coverage.
  B. A different test oracle is usually required for each AI-generated functional test case.
  C. Expected results may not be available for AI-generated functional test cases.
  D. An AI-based system under test must not be used as a functional test oracle.
  C. Expected results may not be available for AI-generated functional test cases.

  ---

  The ISTQB CT-AI syllabus indicates in Section 5.2 - AI for Testing that AI-generated test cases may not come with predefined expected results. This is because AI-based test case generation techniques---such as evolutionary algorithms, reinforcement learning, or clustering-based sampling---primarily generate input data. Determining the correct expected output often still requires human judgment or additional mechanisms. Therefore, Option C is correct: expected results may not be available, especially

  when AI generates novel or previously unseen input combinations.

  Option A is incorrect because AI-based test case generation can improve requirements and input-space coverage by exploring large and complex input domains. Option B is incorrect because the need for test oracles does not inherently depend on whether test cases are generated by AI. Option D is incorrect because the syllabus allows the use of an AI-based system as a functional oracle in back-to-back testing scenarios, provided that the limitations of this approach are understood.

  Therefore, Option C is the correct syllabus-aligned statement.

• Question 7:

  A system was developed for screening X-ray images of patients for potential malignancy detection (skin cancer). A workflow system has been developed to screen multiple cancers by using several individually trained ML models chained together in a workflow. Testing the pipeline could involve multiple kinds of tests (I-III):

  A. Pairwise testing of combinations II. Testing each individual model for accuracy III. A/B testing of different sequences of models Which ONE of the following options contains the kinds of tests that would be MOST APPROPRIATE to include in the strategy for optimal detection?
  B. Only III
  C. I and II
  D. I and III
  E. Only II
  B. Only III

  ---

  For an ML workflow composed of multiple chained models, an effective testing strategy must address both individual model performance and interactions between models.

  Testing each individual model for accuracy (II) is essential to ensure that every component in the workflow meets its functional performance requirements before integration.

  Pairwise testing of combinations (I) helps identify interaction issues between models in the workflow, which is important when multiple ML components are combined.

  While A/B testing of different model sequences (III) can be useful for optimization, it is not as fundamental as validating individual model accuracy and critical interactions when establishing reliable detection performance.

  Therefore, the most appropriate combination of tests is I and II, making Option B the correct answer.

• Question 8:

  Which of the following is one of the reasons for data mislabelling?

  A. Lack of domain knowledge
  B. Expert knowledge
  C. Interoperability error
  D. Small datasets
  A. Lack of domain knowledge

  ---

• Question 9:

  A motorcycle engine repair shop owner wants to detect a leaking exhaust valve and fix it before it fails and causes catastrophic damage to the engine. The shop developed and trained a predictive model with historical data files from known healthy engines and ones which experienced a catastrophic failure due to exhaust valve failure. The shop evaluated 200 engines using this model and then disassembled the engines to assess the true state of the valves, recording the results in the confusion matrix below.

  

  What is the precision of this predictive model?

  A. 90.0%
  B. 94.5%
  C. 98.9%
  D. 94.2%
  A. 90.0%

  ---

  The ISTQB CT-AI syllabus defines precision as:

  Precision = TP / (TP + FP)

  Precision measures the proportion of predicted positive cases that are actually positive.

  From the confusion matrix:

  True Positives (TP) = 90

  False Positives (FP) = 10

  Calculation:

  Precision = 90 / (90 + 10)

  = 90 / 100 = 0.90 = 90.0%

  Therefore, the precision of the predictive model is 90.0%, which corresponds to Option A.

  The other options do not match the precision formula defined in the syllabus and therefore are incorrect.

• Question 10:

  "Splendid Healthcare" has started developing a cancer detection system based on ML. The type of cancer they plan on detecting has a 2% prevalence rate in the population of a particular geography. It is required that the model performs well for both normal and cancer patients. Which ONE of the following combinations requires MAXIMIZATION?

  A. Maximize precision and accuracy
  B. Maximize accuracy and recall
  C. Maximize recall and precision
  D. Maximize specificity and number of classes
  C. Maximize recall and precision

  ---

  The low prevalence rate (2%) indicates a highly imbalanced dataset, where cancer cases represent a very small minority of the population. In such scenarios, accuracy alone is misleading, as a model could achieve high accuracy by predominantly predicting the majority class (normal patients).

  Recall (sensitivity) is critical in medical diagnosis because it measures the proportion of actual cancer cases correctly identified. Missing cancer cases (false negatives) can have severe consequences, so recall must be maximized.

  Precision is also important because it measures how many predicted cancer cases are truly cancer cases.

  High precision reduces false positives, avoiding unnecessary anxiety and invasive follow-up procedures for healthy patients.

  When both false negatives and false positives have significant impact, as in cancer detection, the ISTQB CT-AI syllabus emphasizes the need to maximize both recall and precision, rather than relying on accuracy or specificity alone.

  Therefore, Option C (Maximize recall and precision) is the correct answer.