ISTQB Certified Tester AI Testing

The ISTQB CT-AI syllabus explains in Section 1.6 - Machine Learning Approaches that supervised learning is appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer's purchase probability is a classification task when the output corresponds to discrete categories such as "likely to purchase" versus "not likely to purchase." The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice.

Option A (regression) predicts continuous numeric values and is less suitable because the task centers on categorical likelihood rather than estimating exact monetary values.
Option C (unsupervised learning) is used when labels are not available; in this scenario, the company has labeled purchase histories. Option D
(reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus, supervised learning (classification) is the most suitable approach according to the syllabus.

The ISTQB CT-AI syllabus identifies accuracy of data and labels as a key data quality aspect. Incorrectly labeled data directly indicates a problem with labeling accuracy, as the labels do not correctly represent the objects in the data. Such inaccuracies lead to misleading training signals and degrade the model's learning and predictive performance.

Security issues relate to unauthorized access or data breaches, not labeling correctness. Privacy issues concern the handling of personal or sensitive data and are unrelated to labeling errors. Bias issues involve systematic skew or unfair representation in the data, rather than labels being factually wrong.

Therefore, accuracy issues (Option B) are the most likely reason for the incorrectly labeled data.

The ISTQB CT-AI syllabus explains that AI test environments differ from traditional test environments mainly due to technology-related characteristics of AI systems. These include model accuracy limitations, the emergence of undefined or unexpected scenarios caused by self-learning behavior, and the need to provide explainability for AI decisions. Options A, B, and C all directly reflect such AI-specific, technology-driven challenges.

Option D, however, concerns the creation of human handover scenarios in autonomous systems. While this is an important testing concern, it is primarily related to operational, human-machine interaction, and deployment aspects, rather than being an intrinsic AI-technology-related characteristic that differentiates AI test environments themselves.

Therefore, Option D is the correct answer.

The ISTQB CT-AI syllabus (Section 3.2 - Model Evaluation) specifies the correct usage of training, validation, and test data sets. It emphasizes that the test data set must be representative of the real operational data and must be equivalent in distribution to the training and validation data sets, ensuring a fair and unbiased evaluation. Option D precisely matches this requirement.

Option A contradicts the syllabus because validation and test data sets serve different purposes: validation is used for model tuning, while the test data set is used for final evaluation. Combining them undermines the reliability of results.
Option B is incorrect because even with limited data, the syllabus recommends maintaining a test data set or using techniques such as cross-validation rather than eliminating testing.
Option C is incorrect because an equal distribution between training, validation, and test data sets is not recommended; typically, the training data set is significantly larger (for example, 70-80%).

Thus, Option D is the only statement aligned with the syllabus guidance.

The ISTQB CT-AI syllabus explains in Section 3.2 - Functional Performance Criteria of ML Models that accuracy becomes unreliable when class imbalance exists. In fraud detection, more than 99% of transactions are non-fraudulent, meaning the data set is extremely imbalanced. Because accuracy counts all correctly classified non-fraudulent transactions, it can appear artificially high even when the fraud detection performance is poor. Therefore, accuracy is not suitable for evaluating fraud detection systems.

The syllabus further explains that sensitivity (recall) measures the proportion of correctly identified fraudulent transactions. This metric is important because missed fraudulent events can lead to significant financial loss. However, the client also requires that legitimate transactions be correctly identified, meaning false positives must be minimized to maintain customer satisfaction.

The F1 score, defined as the harmonic mean of precision and recall, balances both aspects: precision limits false alarms affecting legitimate customers, while recall ensures that fraudulent transactions are detected. Section 3.2 emphasizes that when both false positives and false negatives have serious consequences and the data is highly imbalanced, the F1 value is the most appropriate metric.

Thus, Option C is the correct choice.

The ISTQB CT-AI syllabus defines data acquisition as the activity of obtaining data relevant to the business problem addressed by an ML model. This activity typically consists of three steps: identifying relevant data sources, gathering the data, and labeling the data for use in supervised learning.

The other options describe activities that belong to different phases of the ML workflow. Cleaning and transforming are part of data preprocessing, feature-related activities are part of feature engineering, and building or deploying relates to later stages of system development.

Therefore, Identifying, gathering, labeling (Option C) is the correct answer.

The ISTQB CT-AI syllabus explains that pairwise testing is an effective test design technique for reducing the number of test cases when a system has a large number of input parameters. By focusing on all possible pairs of parameter values, many defects caused by two-way interactions can be identified without exhaustive testing.

However, the syllabus also emphasizes that even after applying pairwise testing, the resulting test suite for complex systems---such as autonomous vehicles with many environmental, sensor, and behavioral parameters---can still be large. In such cases, test automation and virtual test environments are often necessary to execute the reduced but still substantial set of test cases efficiently.

Option C correctly reflects this guidance.
Option A is incorrect because pairwise testing reduces combinations, not the number of parameters themselves.
Option B is incorrect because pairwise testing does not guarantee detection of all high-priority defects, especially those caused by higher-order interactions.
Option D is incorrect because pairwise testing is applicable to AI-based systems and is explicitly discussed in the syllabus as a useful technique despite AI complexity.

Therefore, Option C is the correct answer.

The ISTQB CT-AI syllabus discusses the use of AI-based tools to reduce the brittleness of GUI-based test automation. Frequent failures often occur when identifiers such as IDs, XPaths, or class names change between releases.
The syllabus states that:

"AI can be used to reduce the brittleness of this approach by employing AI-based tools to identify the correct objects using various criteria (e.g., XPath, label, id, class, X/Y coordinates), and to choose the historically most stable identification criteria."

This directly aligns with Option A, which describes identifying GUI objects using multiple attributes and selecting the most stable and reliable identifiers over time.

The other options are incorrect because ignoring objects reduces test value, modifying source code is not a tester responsibility, and predicting developer changes is unrealistic and unsupported by the syllabus.

Therefore, Option A is the correct and syllabus-aligned answer.

The ISTQB CT-AI syllabus defines sample bias as a bias that occurs when the training data is not representative of the population the model is expected to encounter in operation.

In this scenario, ethnicity A is significantly overrepresented in the dataset compared to ethnicities B, C, and
D. As a result, the model may learn patterns that perform better for ethnicity A and worse for the
underrepresented groups.

Expert system bias relates to biases introduced by explicitly defined rules, not data distribution.

Hyperparameter bias concerns model configuration choices, not dataset composition.

Algorithmic bias refers to bias arising from the algorithm itself, whereas this scenario describes bias originating from the training data.

Therefore, sample bias (Option B) is the correct answer.

Section 2.2 - Data Preparation and Section 4.1 - Challenges in Testing AI-Based Systems describe difficulties in obtaining and managing large, representative data sets. AI-based systems require realistic, diverse, and representative data that reflects real-world variation. The syllabus emphasizes that assembling such data sets is time-consuming, resource-intensive, and often constrained by availability, privacy concerns, or domain complexity. Option B directly corresponds to these documented challenges.

Option A is incorrect because using the same implementation for data acquisition can lead to defect masking rather than preventing it; the syllabus explicitly warns against this risk. Option C is incorrect
because real-world input data naturally changes over time, and the syllabus notes that data drift is normal in operational environments; expecting stable input data contradicts this reality. Option D is incorrect
because, while data privacy is important, the syllabus does not state that artificially generated data always requires legal approval or that sanitization and encryption are mandatory in all cases.

Thus, Option B accurately reflects the syllabus-defined difficulties related to test data in AI-based systems.