An organization's system development process has been enhanced with AI. Which of the following features presents the GREATEST risk?
-
A
The AI allocates resources for new system development projects.
-
B
Non-technical users are validating AI results.
-
C
The AI personalizes applications for the user.
-
D
All codes are generated by AI without human oversight.
Reveal answer details
Close answer details
Correct answerD
ExplanationAllowing AI to autonomously generate code without human review introduces significant risks, including security vulnerabilities, logic errors, and noncompliance with organizational development standards. The AAIATM Study Guide strongly advocates for human-in-the-loop oversight, particularly in automated development contexts. "AI-assisted development must include manual code reviews to ensure functionality, compliance, and security. Autonomous code generation without validation increases the risk of introducing undetected flaws." While A, B, and C involve operational risks or inefficiencies, only D constitutes a direct breach of secure development life cycle principles. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Fundamentals and Technologies," Subsection: "AI in Software Development and Associated Risks"
Which of the following BEST detects model drift or unexpected changes in AI model outputs?
-
A
Standardization of AI configurations
-
B
-
C
AI model documentation reviews
-
D
Reveal answer details
Close answer details
Correct answerB
ExplanationAnomaly monitoring detects irregularities or deviations in input data or model outputs, which are key indicators of model drift. According to the AAIATM Study Guide, continuous anomaly detection is one of the most effective methods for identifying when a model is no longer functioning as expected due to changes in the data environment. "Monitoring for output anomalies enables early identification of model drift. This proactive approach allows organizations to retrain or adjust models before significant performance degradation occurs." Configuration standardization (A) and documentation reviews (C) support governance but don't detect changes. Retraining (D) is a remediation step, not a detection mechanism. Therefore, B is correct. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Operations and Performance," Subsection: "Model Drift Detection and Anomaly Monitoring"
An IS auditor is testing an AI-based fraud detection system that flags suspicious transactions and finds that the system has a high false positive rate. Which of the following testing methods should be prioritized to BEST optimize the detection rate?
-
A
-
B
-
C
-
D
Reveal answer details
Close answer details
Correct answerB
ExplanationCross-validation testing is a statistical method used to assess how well a model generalizes to an independent data set. The AAIATM Study Guide recommends this method as a best practice to fine-tune model accuracy and reduce both false positives and false negatives. It involves splitting the dataset into training and testing subsets multiple times to ensure model robustness. "Cross-validation allows auditors and developers to identify overfitting and adjust model parameters to achieve better generalization and predictive accuracy, especially in fraud detection contexts." Regression testing (A) focuses on changes over time; substantive testing (C) is audit-specific but not model-focused. Benford's Law (D) applies to numerical distributions but is not designed for optimizing ML models. Hence, B is the best approach. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Operations and Performance," Subsection: "Testing and Model Validation Methods"
Which of the following is MOST important to have in place when initially populating data into a data frame for an AI model?
-
A
The box charts, histograms, scatterplots, and Venn diagrams that identify correlations and outliers
-
B
The code for separating data into training and testing data sets
-
C
An analysis of exploratory data that checks for incorrect data types, null values, and duplicate entries
-
D
An approved risk assessment for including, excluding, or subsequently dropping data attributes from the model
Reveal answer details
Close answer details
Correct answerC
ExplanationExploratory Data Analysis (EDA) is critical during the initial stages of AI model development. According to the AAIATM Study Guide, performing EDA--including identifying null values, incorrect data types, or duplicates--ensures that the data fed into the model is clean and reliable. "Initial data frames should be subject to thorough EDA to uncover data quality issues. These issues, if not addressed early, can severely affect model training and predictive accuracy." While separating data sets (B) and visualizations (A) are important steps in later phases, C is foundational to ensure readiness for model training. Risk assessments (D) are necessary but not the first operational step. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Fundamentals and Technologies," Subsection: "Exploratory Data Analysis and Preprocessing"
An IS auditor is auditing an organization's data governance framework. The primary objective is to provide assurance that data management practices are standardized to support a trustworthy AI system. Which of the following should be the auditor's MOST important consideration?
-
A
-
B
-
C
Data practices for training models
-
D
Accountability for data management
Reveal answer details
Close answer details
Correct answerD
ExplanationAccountability for data management (option D) is the most crucial consideration. The AAIATM Study Guide emphasizes that "clear roles, responsibilities, and ownership for data management activities are central to trustworthy AI systems, as they ensure compliance, traceability, and the consistent application of policies and controls." Retention, portability, and data training practices are important, but accountability is foundational for the enforcement and monitoring of all other governance practices. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Accountability in Data Governance for AI"
Which of the following controls helps mitigate the risk of competitors poisoning data utilized by a machine learning (ML) model performing sentiment analysis of product reviews?
-
A
Peer reviewing code that acquires product reviews from social media posts
-
B
Hiring a marketing firm to text links to customers requesting product reviews for monetary compensation
-
C
Augmenting the unbalanced product review data set with the use of oversampling by the model developer
-
D
Requiring customers to authenticate access to their accounts prior to writing product reviews
Reveal answer details
Close answer details
Correct answerD
ExplanationTo prevent data poisoning, especially in systems relying on publicly submitted content such as product reviews, access authentication is critical. The AAIATM Study Guide specifies that authenticated input sources help ensure data integrity and traceability, reducing the likelihood of adversarial or malicious contributions. "Limiting review input to authenticated users restricts unauthorized actors--such as competitors or bots-- from submitting biased or harmful data. This control protects model training and outputs from being manipulated." Options A and C address technical data handling but not source authenticity. Option B may inadvertently incentivize biased reviews. Option D provides a robust control against poisoning. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Fundamentals and Technologies," Subsection: "Input Data Security and Adversarial Risk Mitigation"
During an audit of an investment organization's AI-powered software, an IS auditor identifies a potential security risk. What is the GREATEST risk associated with staff exfiltrating organizational data to a generative AI tool?
-
A
Data contamination due to biased AI model outputs
-
B
Unauthorized data disclosure
-
C
Potential business disruptions
-
D
Excessive reliance on AI-generated insights
Reveal answer details
Close answer details
Correct answerB
ExplanationThe AAIATM Study Guide stresses that inputting confidential or proprietary data into third-party generative AI tools may result in unauthorized data disclosure. These tools may store, process, or retrain on the input data, leading to privacy and intellectual property risks. "When employees input sensitive data into external AI tools, organizations risk losing control over that information. This may result in regulatory non-compliance, legal exposure, and irreversible data leakage." While business disruption (C) and reliance (D) are notable, the most severe and immediate risk is B-- unauthorized disclosure. Data contamination (A) impacts model reliability, not data security. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "Data Privacy and Use of External AI Tools"
An organization shares an AI model with external partners. One partner reports that sensitive data has been inadvertently exposed through the model's outputs. Which of the following is the IS auditor's BEST recommendation?
-
A
Limit the model's outputs to anonymized results while investigating further.
-
B
Audit the data pipelines of all partners to identify the source of the leak.
-
C
Disable the shared model and notify partners of the potential breach.
-
D
Retrain the model immediately and implement privacy-preserving techniques.
Reveal answer details
Close answer details
Correct answerC
ExplanationIn the case of a potential data exposure through AI model outputs, the first and most responsible action from an auditing and risk standpoint is to halt further risk propagation. According to the AAIATM Study Guide, immediate containment is vital, especially when regulatory and reputational risks are high. "Upon detection of a data breach risk, AI models should be immediately disabled from public or partner use, and all relevant parties should be notified as part of a responsible disclosure and containment strategy." While options A and D are longer-term remediation steps and B is investigative, none of them provide the urgent containment that is best practice in such a breach context. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "AI Data Breach and Disclosure Management"
Which of the following is the MOST important purpose of conducting a risk assessment for AI models within an organization?
-
A
Categorizing data used by the AI model
-
B
Defining mitigation strategies for AI deployment
-
C
Monitoring AI model performance on an ongoing basis
-
D
Determining whether AI model outputs align with established use cases
Reveal answer details
Close answer details
Correct answerB
ExplanationRisk assessments identify potential threats and vulnerabilities in AI systems and support the development of mitigation strategies. According to the AAIATM Study Guide, the main objective is not just identification of risks but to proactively design controls that minimize impact and ensure ethical and regulatory compliance. "The output of an AI risk assessment should lead to actionable mitigation strategies, supporting the secure and responsible deployment of AI solutions across business processes." Options A, C, and D are components of governance and monitoring, but B addresses the core intent of risk assessments. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Risk Assessment and Mitigation for AI Systems"
Question 10
Single choice
When auditing an AI system, which of the following steps ensures that AI model behavior is aligned with organizational objectives?
-
A
-
B
-
C
-
D
Reveal answer details
Close answer details
Correct answerD
ExplanationProblem framing (option D) is the process of clearly defining the purpose, scope, and desired outcomes of an AI system before it is built or deployed. According to the ISACA AAIATM Study Guide, "problem framing is the critical first step that aligns model development and subsequent behaviors with the strategic and operational objectives of the organization." If the AI problem is not framed in accordance with organizational goals, even a technically successful AI model could generate outputs that do not support the organization's mission or priorities. Algorithm debugging, data transformation, and model training are all important phases but rely on the initial problem framing to ensure their efforts are correctly directed. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Project Lifecycle Alignment"
Question 11
Single choice
Which of the following is an IS auditor's MOST important course of action when determining whether source data should be entered into approved generative AI tools to assist with an audit?
-
A
Validate that the tool is leveraging the latest model.
-
B
Validate that the tool provides a privacy notice.
-
C
Determine whether any AI model hallucinations have occurred.
-
D
Determine whether the information is reliable.
Reveal answer details
Close answer details
Correct answerD
ExplanationWhen using generative AI tools during audit activities, the most critical concern is the reliability and appropriateness of the information being entered and processed. According to the AAIATM Study Guide, auditors are accountable for ensuring that audit data is valid, confidential, and that generated outputs are factual and verifiable. "IS auditors must evaluate whether the information entered into AI tools is reliable and appropriate for the audit context. Inputting sensitive or unverified data may lead to regulatory violations or audit inaccuracies." While hallucinations (C) and privacy notices (B) are important concerns, the primary auditor responsibility is to ensure that source data is accurate and suitable. Therefore, D is the correct response. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "Auditor Responsibility and AI Input Validation"
Question 12
Single choice
The PRIMARY purpose of maintaining an audit trail in AI systems is to:
-
A
Facilitate transparency and traceability of decisions.
-
B
Analyze model accuracy and fairness.
-
C
Measure computational efficiency.
-
D
Ensure compliance with regulatory standards for AI.
Reveal answer details
Close answer details
Correct answerA
ExplanationAudit trails in AI systems serve to document the inputs, processes, and outputs of AI decisions, allowing stakeholders and auditors to trace how decisions were made. The AAIATM Study Guide identifies transparency and traceability as the core functions of an AI audit trail. "Maintaining audit logs is critical to explainability and accountability. It ensures that decisions made by AI systems can be reconstructed and assessed for accuracy, ethics, and legality." Although compliance (D) is an outcome and fairness (B) a goal, only A captures the foundational purpose of audit trails in AI systems. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Auditability and Logging Mechanisms"
Question 13
Single choice
Which of the following testing techniques would BEST validate whether an organization's data governance program effectively ensures data quality and integrity for AI model training and deployment?
-
A
Performing a business impact analysis (BIA) to assess the consequences of AI model failure
-
B
Reviewing the organization's AI software development life cycle documentation
-
C
Conducting a penetration test to identify vulnerabilities in the model
-
D
Assessing data lineage to verify the traceability of data sources
Reveal answer details
Close answer details
Correct answerD
ExplanationAssessing data lineage provides insight into the origin, flow, and transformation of data across its lifecycle, which is crucial for validating data governance. The AAIATM Study Guide states that data lineage is essential to ensure the accuracy, consistency, and trustworthiness of data used in training AI models. "Traceability of data sources is a core tenet of effective data governance. Data lineage validation ensures data quality, prevents unauthorized modifications, and maintains auditability." BIA (A) focuses on impact, not data quality. Reviewing SDLC (B) is broad and may not highlight data-specific risks. Penetration testing (C) addresses security, not governance. Therefore, D is the best method. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Data Quality, Integrity, and Governance Practices"
Question 14
Single choice
From a data appropriateness and bias perspective, which of the following should be of GREATEST concern when reviewing an AI model used in a credit scoring system?
-
A
The model incorporates the applicant's loan history to assess spending habits.
-
B
The model utilizes historical credit data to predict future credit behavior.
-
C
The model considers the applicant's income level as a key factor in the credit decision.
-
D
The model uses postal codes as a primary factor in determining creditworthiness.
Reveal answer details
Close answer details
Correct answerD
ExplanationUsing postal codes as a primary factor in credit scoring raises concerns of geographic and socioeconomic bias. Postal codes can serve as proxies for race, ethnicity, or income level--potentially violating fair lending laws and ethical guidelines. "Auditors should flag models that rely on proxies for protected attributes. Postal codes are high-risk features that may inadvertently lead to redlining or other discriminatory practices." A, B, and C are more justifiable under fair lending guidelines. Thus, D presents the highest concern. References: ISACA Advanced in AI AuditTM (AAIATM) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Bias and Fairness in Financial AI Models"
|