ISQI ISQI-CT-AI Online Practice Questions and Exam Preparation

ISQI ISQI-CT-AI Online Questions & Answers

• Question 41:

  Before deployment of an AI-based system, a developer is expected to demonstrate in a test environment how decisions are made. Which of the following characteristics does decision making fall under?

  A. Explainability
  B. Autonomy
  C. Self-learning
  D. Non-determinism
  A. Explainability

  ---

  The ISTQB CT-AI syllabus defines explainability as the ability to understand and describe how an AI-based system arrives at a particular result or decision. Demonstrating how decisions are made before deployment directly relates to making the decision logic and reasoning of the system understandable to stakeholders, testers, and regulators.

  The other options do not apply. Autonomy refers to the system's ability to act without human intervention, self-learning concerns the system's capability to adapt based on data, and non-determinism relates to variability in outputs. None of these specifically address explaining decision logic.

  Therefore, Explainability is the correct answer.

• Question 42:

  Consider a natural language processing (NLP) algorithm that attempts to predict the next word that you would like to type in a text message. An update to the algorithm has been created that should increase the accuracy of the predictions based on user typing patterns. The old algorithm was rated for accuracy by the users. Then, after the new update was released, the users rated the updated algorithm. A statistical test was used to compare the two versions of the algorithm to determine whether the update should remain in place.

  This is an example of what type of testing?

  A. Metamorphic testing
  B. A/B testing
  C. Exploratory testing
  D. Pairwise testing
  B. A/B testing

  ---

  The ISTQB CT-AI syllabus states that A/B testing can be used to evaluate updates to an AI-based system when there are agreed acceptance criteria, such as machine learning functional performance metrics. In this approach, an updated variant of the system is statistically compared with a previous variant to determine whether the change provides a measurable improvement. This scenario exactly matches that description: users rate the accuracy of both the old and the updated NLP algorithm, and a statistical test is used to compare the results to decide whether the update should be retained. Therefore, this is an example of A/B testing.

• Question 43:

  Which ONE of the following options represents a technology MOST TYPICALLY used to implement Al?

  A. Search engines
  B. Procedural programming
  C. Case control structures
  D. Genetic algorithms
  D. Genetic algorithms

  ---

  Technology Most Typically Used to Implement AI: Genetic algorithms are a well-known technique used in AI. They are inspired by the process of natural selection and are used to find approximate solutions to optimization and search problems. Unlike search engines, procedural programming, or case control structures, genetic algorithms are specifically designed for evolving solutions and are commonly employed in AI implementations. Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 1.4 AI Technologies, which identifies different technologies used to implement AI.

• Question 44:

  Which of the following are ML functional performance metrics used for classification problems? (Choose three)

  A. Precision
  B. Recall
  C. R-squared
  D. Specificity
  A. Precision
  B. Recall
  D. Specificity

  ---

• Question 45:

  Which of the following characteristics make ensuring safety more difficult in AI-based systems? (Choose two)

  A. Deterministic execution
  B. Self-learning behavior
  C. Probabilistic outputs
  D. Simplicity of design
  B. Self-learning behavior
  C. Probabilistic outputs

  ---

• Question 46:

  Which ONE of the following statements is a CORRECT description of adversarial examples in the context of machine learning systems that work on image classifiers?

  A. Black-box attacks based on adversarial examples create an exact duplicate model of the original.
  B. These attack examples cause a model to predict the correct class with slightly lower accuracy, even though they look like the original image.
  C. These attacks cannot be prevented by retraining the model with these examples augmented to the training data.
  D. These examples are model-specific and are not likely to cause another model trained on the same task to fail.
  D. These examples are model-specific and are not likely to cause another model trained on the same task to fail.

  ---

  Adversarial examples are inputs that are intentionally modified in a subtle way to cause a machine learning model to make an incorrect prediction, even though the input appears unchanged to a human observer.

  Option A is incorrect because black-box attacks do not create an exact duplicate of the original model. Instead, attackers may approximate behavior by querying the model, without knowing its internal structure.

  Option B is incorrect because adversarial examples are designed to cause misclassification, not merely a slight reduction in accuracy while still predicting the correct class.

  Option C is incorrect because adversarial attacks can be mitigated through techniques such as adversarial training, where adversarial examples are included in the training data to improve model robustness.

  Option D is correct. Adversarial examples are often model-specific, exploiting weaknesses in a particular model's learned decision boundaries. While transferability can occur, many adversarial examples do not reliably cause failures in other models trained on the same task.

  Therefore, Option D is the correct statement.

• Question 47:

  An airline has created an ML model to project fuel requirements for future flights. The model imports weather data such as wind speeds and temperatures, calculates flight routes based on historical routings from air traffic control, and estimates loads from average passenger and baggage weights. The model performed within an acceptable standard for the airline throughout the summer, but as winter set in, the load weights became less accurate. After some exploratory data analysis, it became apparent that luggage weights were higher in the winter than in summer.

  Which of the following statements BEST describes the problem and how it could have been prevented?

  A. The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated
  B. The model suffers from drift and therefore the performance standard should be eased until a new model with more transparency can be developed
  C. The model suffers from corruption and therefore should be reloaded into the computer system being used, preferably with a method of version control to prevent further changes
  D. The model suffers from a lack of transparency and therefore should be regularly tested to ensure that any progressive errors are detected soon enough for the problem to be mitigated
  A. The model suffers from drift and therefore should be regularly tested to ensure that any occurrences of drift are detected soon enough for the problem to be mitigated

  ---

  The ISTQB CT-AI syllabus defines concept drift as a situation where the statistical properties of the operational environment change over time while the trained model remains unchanged. As a result, model outputs become less accurate and less useful.

  In this scenario, seasonal changes caused a shift in baggage weight patterns, which were not adequately reflected in the model's assumptions. This is a classic example of concept drift caused by changing real-world conditions.

  The syllabus states that operational ML models should be regularly evaluated against their acceptance criteria so that drift can be detected early and mitigated through retraining, model adjustment, or data updates.

  Therefore, Option A correctly identifies both the problem (concept drift) and the appropriate preventive measure.

• Question 48:

  Which of the following is a problem with AI-generated test cases that are generated from the requirements?

  A. They are slow and will usually not be able to execute within the allowed time.
  B. They are defect-prone because they are unable to detect nuances in the requirements.
  C. They make debugging more complicated because the number of steps is usually high in order to induce the target failure.
  D. They are usually missing the expected results, so verification is difficult or must rely only on detecting significant failures.
  D. They are usually missing the expected results, so verification is difficult or must rely only on detecting significant failures.

  ---

  The ISTQB CT-AI syllabus explains a key limitation of AI-generated test cases, especially when they are generated directly from requirements. Unless a formal test model or oracle is available, AI-based test generation often suffers from a test oracle problem. This occurs because the AI tool can generate inputs and execution steps but does not inherently know what the correct expected results should be.

  As stated in the syllabus, without a defined model of required behavior, verification becomes difficult and may be limited to detecting only major failures or anomalies rather than precise correctness. This directly corresponds to Option D.

  The other options are not identified by the syllabus as primary or typical problems of AI-generated test cases. Performance, defect proneness due to requirement nuance, and debugging complexity are not cited as defining drawbacks in this context.

  Therefore, Option D is the correct answer.

• Question 49:

  Which of the following describes the AI effect?

  A. The changing perception of what constitutes AI
  B. The ability of AI to learn from data itself
  C. The fact that mankind can build intelligent machines
  D. The ability of AI to defeat a human, for example in chess
  A. The changing perception of what constitutes AI

  ---

  The AI Effect is clearly defined in the ISTQB Certified Tester AI Testing Syllabus v1.0 under Section 1.1 - Definition of AI and AI Effect. The syllabus explains that society's perception of what qualifies as "AI" changes over time. Technologies that were once considered artificial intelligence---such as expert systems from the 1970s and 1980s or early chess-playing systems---are no longer viewed as AI once they become well understood or widely adopted. This phenomenon is explicitly referred to as the AI

  Effect and is described as the changing perception of what constitutes AI.

  Option B is incorrect because the ability to learn from data is a characteristic of machine learning, not a description of the AI Effect. Option C refers to the general capability of building intelligent machines and does not address the perceptual shift captured by the AI Effect. Option D describes a historical achievement in AI, such as defeating a human in chess, which the syllabus explicitly cites as an example of technology that is no longer considered AI due to the AI Effect.

  Therefore, Option A accurately reflects the AI Effect as defined in the ISTQB CT-AI syllabus.

• Question 50:

  Which statement about AI-based test case generation is correct?

  A. AI-generated functional test cases typically result in poor requirements coverage.
  B. A different test oracle is usually required for each AI-generated functional test case.
  C. Expected results may not be available for AI-generated functional test cases.
  D. An AI-based system under test must not be used as a functional test oracle.
  C. Expected results may not be available for AI-generated functional test cases.

  ---

  The ISTQB CT-AI syllabus indicates in Section 5.2 - AI for Testing that AI-generated test cases may not come with predefined expected results. This is because AI-based test case generation techniques---such as evolutionary algorithms, reinforcement learning, or clustering-based sampling---primarily generate input data. Determining the correct expected output often still requires human judgment or additional mechanisms. Therefore, Option C is correct: expected results may not be available, especially

  when AI generates novel or previously unseen input combinations.

  Option A is incorrect because AI-based test case generation can improve requirements and input-space coverage by exploring large and complex input domains. Option B is incorrect because the need for test oracles does not inherently depend on whether test cases are generated by AI. Option D is incorrect because the syllabus allows the use of an AI-based system as a functional oracle in back-to-back testing scenarios, provided that the limitations of this approach are understood.

  Therefore, Option C is the correct syllabus-aligned statement.