Isaca ISACA-AAIA Online Practice Questions and Exam Preparation

Isaca ISACA-AAIA Online Questions & Answers

• Question 31:

  An organization uses an AI-powered tool to detect and respond to cybersecurity threats in real time. An IS auditor finds that the tool produces excessive false positives, increasing the workload of the security team. Which of the following techniques should the auditor recommend to BEST evaluate the tool's effectiveness in managing this issue?

  A. Use a log analysis tool to examine the types and frequency of alerts generated.
  B. Implement a benchmarking tool to compare the system's alerting capability with industry standards.
  C. Conduct penetration testing to assess the system's ability to detect genuine threats.
  D. Deploy a machine learning (ML) validation tool to increase the model's accuracy and performance.
  D. Deploy a machine learning (ML) validation tool to increase the model's accuracy and performance.

  ---

  Explanation

  The AAIATM Study Guide recommends using validation tools to fine-tune and evaluate ML models, particularly when high false positives undermine operational efficiency. ML validation can identify threshold adjustments, retraining needs, or

  feature misweighting contributing to excessive alerting.

  "Model validation enables organizations to quantify performance, reduce false alarms, and recalibrate AI behavior to align with operational needs and threat landscapes."

  While logs (A) and benchmarks (B) help with diagnosis, they don't improve the model. Penetration testing (C) evaluates detection, not alert noise. D is the most effective solution.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Operations and Performance," Subsection: "Model Tuning and False Positive Mitigation"

• Question 32:

  An IS auditor is testing an AI-based fraud detection system that flags suspicious transactions and finds that the system has a high false positive rate. Which of the following testing methods should be prioritized to BEST optimize the detection rate?

  A. Regression testing
  B. Cross-validation testing
  C. Substantive testing
  D. Benford's Law analysis
  B. Cross-validation testing

  ---

  Explanation

  Cross-validation testing is a statistical method used to assess how well a model generalizes to an independent data set. The AAIATM Study Guide recommends this method as a best practice to fine-tune model accuracy and reduce both

  false positives and false negatives. It involves splitting the dataset into training and testing subsets multiple times to ensure model robustness.

  "Cross-validation allows auditors and developers to identify overfitting and adjust model parameters to achieve better generalization and predictive accuracy, especially in fraud detection contexts."

  Regression testing (A) focuses on changes over time; substantive testing (C) is audit-specific but not model- focused. Benford's Law (D) applies to numerical distributions but is not designed for optimizing ML models.

  Hence, B is the best approach.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Operations and Performance," Subsection: "Testing and Model Validation Methods"

• Question 33:

  An IS auditor is interviewing management about implemented controls around machine learning (ML) models deployed in the production environment. Which of the following schedules for reviewing the performance of a deployed model would be of GREATEST concern to the auditor?

  A. After changes to hardware and software platforms
  B. After functionality changes
  C. One time prior to migrating to production
  D. On an annual recurring basis
  C. One time prior to migrating to production

  ---

  Explanation

  Only reviewing an ML model's performance one time prior to migrating to production (option C) is of greatest concern. The AAIATM Study Guide emphasizes that "AI and ML models require continuous monitoring and periodic performance

  reviews in production to detect issues such as data drift, model degradation, or evolving risk factors." A single pre-production review fails to capture these changes and risks, potentially resulting in undetected failures or compliance issues.

  Periodic (including annual) and event-driven reviews are necessary to ensure ongoing model reliability.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Continuous Monitoring and Review of Deployed AI Models"

• Question 34:

  When auditing a machine learning (ML) solution, false positives can BEST be assessed by examining the level of:

  A. Precision
  B. Completeness
  C. Accuracy
  D. Recall
  A. Precision

  ---

  Explanation

  Precision measures the proportion of true positives among all positive predictions. A low precision rate indicates a high rate of false positives. The AAIATM Study Guide recommends using precision when the goal is to minimize incorrect

  positive alerts, which is especially relevant in fraud detection, cybersecurity, and classification models.

  "Precision is the key metric when false positives have a significant operational cost. It provides insight into the model's ability to avoid incorrect positive classifications."

  Accuracy and recall give broader insights, but only precision directly measures false positive risk. Completeness is not a standard ML metric.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Operations and Performance," Subsection: "Performance Metrics for Classification Models"

• Question 35:

  Which of the following is the PRIMARY reason IS auditors must be aware that generative AI may return different investment recommendations from the same set of data?

  A. Limitations can arise in the quantification of risk profiles.
  B. Neural node access varies each time the process is executed.
  C. Computational logic is based on probabilities.
  D. Servers are reconfigured periodically.
  C. Computational logic is based on probabilities.

  ---

  Explanation

  Generative AI systems, particularly those based on transformer models, produce outputs using probabilistic computations. As a result, even when given the same input data, these models may generate different outputs depending on

  sampling strategies (e.g., temperature, top-k sampling).

  "Generative AI operates probabilistically, meaning that outputs can vary with each run based on stochastic sampling techniques. This variability is expected and must be accounted for in risk-sensitive environments like finance."

  While A and B refer to limitations and architecture, and D is unrelated to logic, C directly explains the output inconsistency.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Fundamentals and Technologies," Subsection: "Stochastic Behavior in Generative Models"

• Question 36:

  A bank uses a video-based know your customer (KYC) verification process. Cybercriminals exploit this process by using deepfake technology to impersonate bank customers. Which of the following countermeasures is the BEST way for the bank to mitigate this risk?

  A. Requesting additional identity and address documents for verification
  B. Leveraging AI-based liveness detection during video verification
  C. Encrypting all customer data and communication
  D. Discontinuing the use of the video-based verification process
  B. Leveraging AI-based liveness detection during video verification

  ---

  Explanation

  Liveness detection is the most effective countermeasure against deepfakes in video-based verification. AI- based liveness detection analyzes facial movement, micro-expressions, and other biometric cues to differentiate real humans from

  manipulated video content.

  "To protect against identity spoofing and deepfake exploitation, biometric systems must incorporate liveness detection protocols capable of detecting synthetic imagery or falsified video data."

  Encryption (C) protects data at rest and in transit but does not prevent impersonation. Discontinuation (D) may not be necessary if effective countermeasures like B are in place.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Biometric Security and AI Authentication Methods"

• Question 37:

  Which of the following is MOST important for an IS auditor to review during an AI system audit in order to determine compliance with intellectual property and data rights?

  A. Data performance metrics
  B. Data usage agreements
  C. Use of open-source intellectual property
  D. Model runtime efficiency logs
  B. Data usage agreements

  ---

  Explanation

  To assess compliance with intellectual property (IP) and data rights, the IS auditor must review documented data usage agreements that specify ownership, licensing, consent, and limitations of use. The AAIATM Study Guide underscores the

  importance of verifying that the data used to train or feed AI models is obtained and utilized within legal and contractual boundaries.

  "Auditors must review data usage agreements to validate whether the organization has appropriate rights to use, distribute, or transform data inputs, especially where third-party or sensitive data is involved."

  While open-source usage (C) is a concern, only B provides legal clarity. Metrics (A) and logs (D) reflect performance--not legal compliance.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "Data Rights, Licensing, and Intellectual Property"

• Question 38:

  Which of the following is the PRIMARY objective of AI governance?

  A. Implementing compliance and ethics controls for AI initiatives
  B. Defining clear roles and responsibilities for AI development, use, and oversight
  C. Ensuring controls over AI are designed well and operate effectively
  D. Promoting a positive return on investment (ROI) from AI projects
  B. Defining clear roles and responsibilities for AI development, use, and oversight

  ---

  Explanation

  The AAIATM Study Guide defines the primary objective of AI governance as establishing structure and accountability for AI initiatives. This includes clearly assigning responsibilities across development, deployment, risk management, and

  auditing roles to ensure that AI is used responsibly and transparently.

  "AI governance establishes the policies, roles, and oversight structures that guide the ethical and secure deployment of AI. Clear accountability helps prevent unauthorized use and ensures strategic alignment."

  Options A and C are essential components of governance but are not its core definition. Option D is a business outcome, not a governance goal. Thus, B is the most comprehensive and accurate objective.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Governance Objectives and Structures"

• Question 39:

  Which of the following is the MOST important risk for an IS auditor to consider when reviewing the adoption of an AI system?

  A. Costs associated with AI system maintenance
  B. Immaturity of AI systems in the industry
  C. Bias in AI system decision making
  D. Resistance to the use of AI technology
  C. Bias in AI system decision making

  ---

  Explanation

  Bias in AI decision-making is one of the most critical risks, particularly when AI influences areas like hiring, lending, or healthcare. The AAIATM Study Guide highlights the ethical and operational consequences of biased models, which may

  lead to discrimination, legal liability, and reputational damage.

  "Bias in AI outputs can originate from skewed training data or flawed algorithms. Auditors must evaluate whether mitigation techniques, such as bias detection and fairness testing, are in place."

  While costs (A) and industry maturity (B) are considerations, they do not pose the same systemic ethical risk. Resistance (D) is a change management issue. C represents the most impactful and widespread risk.

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "Bias and Fairness in AI"

• Question 40:

  Which of the following AI system characteristics would BEST help an IS auditor evaluate the system's algorithm?

  A. The AI system algorithm uses training data to inform decision output.
  B. The AI system provides multiple options for model training.
  C. The AI system provides transparent justification of decisions.
  D. The AI system uses archived transaction data to provide decisions.
  C. The AI system provides transparent justification of decisions.

  ---

  Explanation

  Transparency is a foundational principle for AI governance and effective risk management. When evaluating AI systems, IS auditors must assess not just how an algorithm functions, but also whether its processes and decisions are understandable and explainable. According to the ISACA Advanced in AI AuditTM (AAIATM) Study Guide, "algorithmic transparency--enabling auditors and stakeholders to see and understand the rationale behind automated decisions--is essential for ensuring accountability, fairness, and compliance with both organizational policies and regulatory requirements."

  Transparent justification of decisions (option C) directly supports an auditor's ability to evaluate the appropriateness, logic, and fairness of the AI's outputs. This is critical in identifying bias, ensuring ethical considerations are met, and verifying that outcomes are consistent with intended objectives. Without transparent justification, it is difficult for an IS auditor to independently assess whether decisions are being made based on valid, legal, and ethical grounds.

  By contrast, options A, B, and D are either standard operational aspects or supportive features, but they do not directly empower auditors to evaluate or scrutinize the algorithm's decision-making process. The Study Guide explicitly highlights, "Effective AI governance frameworks require that systems provide clear, interpretable explanations for their decisions, supporting auditability and accountability across the AI lifecycle."

  ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Transparency and Explainability in AI Systems," Subsection: "Auditability and Accountability"