ISTQB ISTQB-CT-AI Online Practice Questions and Exam Preparation

ISTQB ISTQB-CT-AI Online Questions & Answers

• Question 81:

  A mobile app start-up company is implementing an AI-based chat assistant for e-commerce customers. During test planning, the team realizes that the specifications are insufficient. Which testing approach should be used to test this system?

  A. Exploratory testing
  B. Static analysis
  C. Equivalence partitioning
  D. State transition testing
  A. Exploratory testing

  ---

  The ISTQB CT-AI syllabus explains that exploratory testing is particularly suitable when specifications are incomplete or unclear, or when there are test oracle problems. This situation frequently occurs with AI-based systems, such as chat assistants, where behavior may be probabilistic, data-driven, or insufficiently specified in advance.

  The syllabus explicitly states that exploratory testing is especially useful when there are poor specifications or difficulties in defining expected results. This aligns directly with the described scenario, where the team recognizes that the specifications are insufficient. Therefore, exploratory testing is the most appropriate testing approach, making Option A the correct answer.

• Question 82:

  Which of the following is a problem with AI-generated test cases that are generated from the requirements?

  A. They are slow and will usually not be able to execute within the allowed time.
  B. They are defect-prone because they are unable to detect nuances in the requirements.
  C. They make debugging more complicated because the number of steps is usually high in order to induce the target failure.
  D. They are usually missing the expected results, so verification is difficult or must rely only on detecting significant failures.
  D. They are usually missing the expected results, so verification is difficult or must rely only on detecting significant failures.

  ---

  The ISTQB CT-AI syllabus explains a key limitation of AI-generated test cases, especially when they are generated directly from requirements. Unless a formal test model or oracle is available, AI-based test generation often suffers from a test oracle problem. This occurs because the AI tool can generate inputs and execution steps but does not inherently know what the correct expected results should be.

  As stated in the syllabus, without a defined model of required behavior, verification becomes difficult and may be limited to detecting only major failures or anomalies rather than precise correctness. This directly corresponds to Option D.

  The other options are not identified by the syllabus as primary or typical problems of AI-generated test cases. Performance, defect proneness due to requirement nuance, and debugging complexity are not cited as defining drawbacks in this context.

  Therefore, Option D is the correct answer.

• Question 83:

  Which of the following aspects is a challenge when handling test data for an AI-based system?

  A. Personal data or confidential data
  B. Output data or intermediate data
  C. Video frame speed or aspect ratio
  D. Data frameworks or machine learning frameworks
  A. Personal data or confidential data

  ---

  The ISTQB CT-AI syllabus explicitly highlights challenges related to handling personal and confidential data when testing AI-based systems. Testing often requires realistic data sets that reflect real-world usage, but such data frequently contains personal or sensitive information. The syllabus notes that it is difficult to fully anonymize data while preserving its usefulness for testing purposes, and that privacy, legal, and regulatory constraints must be carefully managed during test data handling.

  Therefore, managing personal data or confidential data represents a significant challenge in AI-based testing, which directly corresponds to Option A.

  The other options are not identified by the syllabus as primary challenges related to test data handling. Output or intermediate data management is a general testing concern, video frame speed or aspect ratio relates to media processing rather than data governance, and data or ML frameworks concern tooling rather than test data handling.

• Question 84:

  Which of the following statements about ML functional performance metrics is correct?

  A. Metrics used to measure clustering include intra-cluster metrics that measure the proximity of a cluster's data points.
  B. The R-squared metric indicates how well the model distinguishes between different classes based on the ROC curve.
  C. The silhouette coefficient describes how well the regression model fits the dependent variables.
  D. The receiver operating characteristic curve shows, depending on parameters, how well the model distinguishes between different clusters.
  A. Metrics used to measure clustering include intra-cluster metrics that measure the proximity of a cluster's data points.

  ---

  The ISTQB CT-AI syllabus explains machine learning performance metrics in Section 3.2 - Evaluating ML Models. For clustering, which is an unsupervised learning approach, the syllabus lists metrics such as intra-cluster distance, inter- cluster distance, and cohesion measures. Intra-cluster metrics evaluate how close data points are within the same cluster, which directly corresponds to Option A.

  Option B is incorrect because R-squared is a regression metric used to measure goodness of fit and is not related to classification performance or ROC curves. Option C is incorrect because the silhouette coefficient is a clustering metric that measures cohesion and separation between clusters, not regression model fit. Option D is incorrect because receiver operating characteristic curves are used to evaluate binary or multiclass classification models, not clustering models.

  Thus, Option A is the only statement that accurately reflects the syllabus.

• Question 85:

  An ML engineer is trying to determine the correctness of a new open-source implementation X of a supervised regression algorithm. R-squared (R? is one of the functional performance metrics used to assess the quality of the model. Which ONE of the following would be an APPROPRIATE strategy to achieve this goal?

  A. Add 10% of the rows randomly, create another model, and compare the R?scores of both models.
  B. Train multiple models by changing the order of input features and verify that the R?scores of these models vary significantly.
  C. Compare the R?score of the model obtained using two different implementations that use two different programming languages while applying the same algorithm and the same training and test data.
  D. Drop 10% of the rows randomly, create another model, and compare the R?scores of both models.
  C. Compare the R?score of the model obtained using two different implementations that use two different programming languages while applying the same algorithm and the same training and test data.

  ---

  To verify the correctness of an implementation, the ISTQB CT-AI syllabus recommends back-to-back comparison using the same algorithm, the same training data, and the same test data. Comparing outputs or performance metrics across independent implementations allows defects in a specific implementation to be identified.

  Option C applies exactly this principle: two independent implementations of the same supervised regression algorithm are compared under identical conditions. If the resulting R?scores are consistent (within expected numerical tolerances), this provides strong evidence that the new implementation X is correct.

  Option A and Option D change the training dataset by adding or removing data. Variations in R?caused by dataset changes do not validate implementation correctness. Option B focuses on robustness with respect to feature ordering; while feature order should not affect results in a correct implementation, this does not directly validate correctness against a trusted reference.

  Therefore, Option C is the most appropriate strategy for verifying the correctness of the new implementation.

• Question 86:

  A team of software testers is attempting to create an AI algorithm to assist in software testing. This team has gone through more than 40 testing iterations and can no longer afford the time required to execute the full regression test suite. They want the algorithm to reduce the amount of testing required, thereby shortening each testing cycle.

  How can an AI-based tool be expected to assist in this reduction?

  A. By using a clustering method to quantify relationships between test cases and assigning each test case to a category
  B. By performing optimization using data from past iterations to identify where defects most frequently occurred and selecting the corresponding test cases
  C. By performing Bayesian analysis to estimate the types of human interactions expected in the system and then selecting those test cases
  D. By using A/B testing to compare the previous update with the newest change and compare metrics between the two
  B. By performing optimization using data from past iterations to identify where defects most frequently occurred and selecting the corresponding test cases

  ---

  The ISTQB CT-AI syllabus explains that AI-based tools can be used to optimize regression test suites by analyzing historical testing information. This includes data from previous test executions, defect history, and recent code or feature changes. By identifying areas of the system that are more defect-prone or have changed frequently, the tool can prioritize or select the most relevant test cases, reducing the overall regression testing effort.

  This directly aligns with Option B, which describes optimization based on past iterations and defect patterns to select the most valuable test cases.

  The other options are incorrect. Clustering test cases alone does not directly reduce regression scope. Bayesian analysis of human interactions is not a standard approach for regression test optimization. A/B testing is used to compare system variants, not to reduce regression test execution.

• Question 87:

  Consider a machine learning model where the model is attempting to predict if a patient is at risk for stroke. The model collects information on each patient regarding their blood pressure, red blood cell count, smoking status, history of heart disease, cholesterol level, and demographics. Then, using a decision tree the model predicts whether or not the associated patient is likely to have a stroke in the near future. Once the model is created using a training dataset, it is used to predict a stroke in 80 additional patients. The table below shows a confusion matrix on whether or not the model made a correct or incorrect prediction.

  

  The testers have calculated what they believe to be an appropriate functional performance metric for the model. They calculated a value of 2/3 or 0.6667. Which metric did the testers calculate?

  A. F1-score
  B. Precision
  C. Recall
  D. Accuracy
  D. Accuracy

  ---

  From the confusion matrix: True Positives (TP) = 15 False Positives (FP) = 10

  False Negatives (FN) = 5 True Negatives (TN) = 50 First, calculate precision and recall, as defined in the ISTQB CT-AI syllabus (Section 3.2 - Functional Performance Criteria): Precision = TP / (TP + FP) = 15 / (15 + 10) = 15 / 25 = 0.6 Recall = TP / (TP + FN) = 15 / (15 + 5) = 15 / 20 = 0.75 The F1-score is the harmonic mean of precision and recall: F1 = 2 ?(Precision ?Recall) / (Precision + Recall)

  = 2 ?(0.6 ?0.75) / (0.6 + 0.75) = 0.9 / 1.35 = 0.6667

  Accuracy, for reference, would be:

  Accuracy = (TP + TN) / (TP + TN + FP + FN)

  = (15 + 50) / 80 = 0.8125 Since the calculated value 0.6667 exactly matches the F1-score, the correct answer is F1-score.

• Question 88:

  AI-enabled medical devices are used nowadays to automate certain parts of medical diagnostic processes. Since these are life-critical processes, the relevant authorities are considering introducing suitable certifications for these AI-enabled medical devices. This certification may involve several facets of AI testing (I-V).

  A. Autonomy II. Maintainability III. Safety IV. Transparency
  B. Side effects Which ONE of the following options contains the three MOST required aspects to be satisfied for the certification of AI-enabled medical devices?
  C. Aspects II, III, and IV
  D. Aspects I, II, and III
  E. Aspects III, IV, and V
  F. Aspects I, IV, and V
  C. Aspects II, III, and IV

  ---

  For AI-enabled medical devices operating in life-critical contexts, the ISTQB CT-AI syllabus emphasizes that certification must focus primarily on aspects that directly impact patient safety and regulatory trust.

  Safety (Aspect III) is essential to ensure that the AI system does not cause harm to patients.

  Transparency (Aspect IV) is required so that medical professionals and regulators can understand, assess, and justify the AI's diagnostic decisions. Side effects (Aspect V) must be identified and controlled, as unintended or emergent behaviors can have serious consequences in medical applications.

  Autonomy and maintainability are relevant quality characteristics but are secondary compared to safety-critical concerns such as preventing harm, ensuring explainability, and managing unintended effects.

  Therefore, Aspects III, IV, and V are the three most required aspects for certification in this scenario.

• Question 89:

  Which of the following are the three activities in data acquisition for data preparation?

  A. Cleaning, transforming, augmenting
  B. Feature selecting, feature growing, feature augmenting
  C. Identifying, gathering, labeling
  D. Building, approving, deploying
  C. Identifying, gathering, labeling

  ---

  The ISTQB CT-AI syllabus defines data acquisition as the activity of obtaining data relevant to the business problem addressed by an ML model. This activity typically consists of three steps: identifying relevant data sources, gathering the data, and labeling the data for use in supervised learning.

  The other options describe activities that belong to different phases of the ML workflow. Cleaning and transforming are part of data preprocessing, feature-related activities are part of feature engineering, and building or deploying relates to later stages of system development.

  Therefore, Identifying, gathering, labeling (Option C) is the correct answer.

• Question 90:

  Which option gives the correct values for accuracy and precision from the confusion matrix?

  The confusion matrix: True Positives (TP) = 15 False Positives (FP) = 5 False Negatives (FN) = 15 True Negatives (TN) = 65

  A. Accuracy = 50%, Precision = 75%
  B. Accuracy = 80%, Precision = 75%
  C. Accuracy = 75%, Precision = 80%
  D. Accuracy = 80%, Precision = 50%
  B. Accuracy = 80%, Precision = 75%

  ---

  From the confusion matrix:

  True Positives (TP) = 15

  False Positives (FP) = 5

  False Negatives (FN) = 15

  True Negatives (TN) = 65

  Total number of instances = 100

  Accuracy is calculated as:

  (TP + TN) / Total

  = (15 + 65) / 100

  = 80%

  Precision is calculated as:

  TP / (TP + FP)

  = 15 / (15 + 5)

  = 15 / 20

  = 75%

  Section 3.2 - Functional Performance Criteria in the syllabus explains accuracy and precision using exactly these definitions when evaluating machine learning classification performance.

  Option B is therefore the correct answer.