Amazon DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Amazon DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 151:

  A data engineer is optimizing query performance in Amazon Athena notebooks that use Apache Spark to analyze large datasets that are stored in Amazon S3. The data is partitioned. An AWS Glue crawler updates the partitions.

  The data engineer wants to minimize the amount of data that is scanned to improve efficiency of Athena queries.

  Which solution will meet these requirements?

  A. Apply partition filters in the queries.
  B. Increase the frequency of AWS Glue crawler invocations to update the data catalog more often.
  C. Organize the data that is in Amazon S3 by using a nested directory structure.
  D. Configure Spark to use in-memory caching for frequently acceed data.
  A. Apply partition filters in the queries.

  ---

  Explanation

  By including predicates on your partition columns in each Athena query (for example, WHERE year = '2025' AND month = '05'), Athena prunes partitions at the optimizer level and reads only the matching S3 folders. This directly minimizes the data scanned and improves query efficiency with no additional infrastructure changes.

• Question 152:

  A company is building data processing pipelines by using AWS Glue. The pipelines access data stored in Amazon S3. The company has organized the data into folders with prefixes that represent different classification levels. The company needs to restrict AWS Glue jobs to access only specific prefixes based on the data classification. The company must also restrict access to business hours (9 AM to 5 PM).

  Which elements must the company include in a custom IAM policy to meet these requirements?

  A. A Resource element with S3 object Amazon Resource Name (ARN) patterns that use wildcards for each prefix and a Condition element that uses the $util.time variable with TimeGreaterThan and TimeLessThan operators
  B. A Resource element with S3 object Amazon Resource Name (ARN) patterns that use wildcards for each prefix and a Condition element that uses the aws:CurrentTime condition key with DateGreaterThan and DateLessThan operators
  C. A Condition element that uses the s3:prefix condition key to restrict folder access and aws:CurrentTime with DateGreaterThanEquals and DateLessThanEquals to restrict hours of operation
  D. A Condition element that uses the s3:ResourceAccount condition key to restrict bucket access and a Deny statement that applies outside of business hours
  B. A Resource element with S3 object Amazon Resource Name (ARN) patterns that use wildcards for each prefix and a Condition element that uses the aws:CurrentTime condition key with DateGreaterThan and DateLessThan operators

• Question 153:

  A company uses Amazon Athena for one-time queries against data that is in Amazon S3. The company has several use cases. The company must implement permission controls to separate query processes and access to query history among users, teams, and applications that are in the same AWS account.

  Which solution will meet these requirements?

  A. Create an S3 bucket for each use case. Create an S3 bucket policy that grants permissions to appropriate individual IAM users. Apply the S3 bucket policy to the S3 bucket.
  B. Create an Athena workgroup for each use case. Apply tags to the workgroup. Create an IAM policy that uses the tags to apply appropriate permissions to the workgroup.
  C. Create an JAM role for each use case. Assign appropriate permissions to the role for each use case.Associate the role with Athena.
  D. Create an AWS Glue Data Catalog resource policy that grants permissions to appropriate individual IAM users for each use case. Apply the resource policy to the specific tables that Athena uses.
  B. Create an Athena workgroup for each use case. Apply tags to the workgroup. Create an IAM policy that uses the tags to apply appropriate permissions to the workgroup.

  ---

  Explanation

  Athena workgroups are a way to isolate query execution and query history among users, teams, and applications that share the same AWS account. By creating a workgroup for each use case, the company can control the access and actions on the workgroup resource using resource-level IAM permissions or identity-based IAM policies. The company can also use tags to organize and identify the workgroups, and use them as conditions in the IAM policies to grant or deny permissions to the workgroup. This solution meets the requirements of separating query processes and access to query history among users, teams, and applications that are in the same AWS account.

• Question 154:

  A data engineer needs to join data from multiple sources to perform a one-time analysis job. The data is stored in Amazon DynamoDB, Amazon RDS, Amazon Redshift, and Amazon S3.

  Which solution will meet this requirement MOST cost-effectively?

  A. Use an Amazon EMR provisioned cluster to read from all sources. Use Apache Spark to join the data and perform the analysis.
  B. Copy the data from DynamoDB, Amazon RDS, and Amazon Redshift into Amazon S3. Run Amazon Athena queries directly on the S3 files.
  C. Use Amazon Athena Federated Query to join the data from all data sources.
  D. Use Redshift Spectrum to query data from DynamoDB, Amazon RDS, and Amazon S3 directly from Redshift.
  C. Use Amazon Athena Federated Query to join the data from all data sources.

  ---

  Explanation

  Amazon Athena Federated Query is a feature that allows you to query data from multiple sources using standard SQL. You can use Athena Federated Query to join data from Amazon DynamoDB, Amazon RDS, Amazon Redshift, and Amazon S3, as well as other data sources such as MongoDB, Apache HBase, and Apache Kafka1. Athena Federated Query is a serverless and interactive service, meaning you do not need to provision or manage any infrastructure, and you only pay for the amount of data scanned by your queries. Athena Federated Query is the most cost-effective solution for performing a one-time analysis job on data from multiple sources, as it eliminates the need to copy or move data, and allows you to query data directly from the source.

  The other options are not as cost-effective as Athena Federated Query, as they involve additional steps or costs.

  Option A requires you to provision and pay for an Amazon EMR cluster, which can be expensive and time-consuming for a one-time job.

  Option B requires you to copy or move data from DynamoDB, RDS, and Redshift to S3, which can incur additional costs for data transfer and storage, and also introduce latency and complexity.

  Option D requires you to have an existing Redshift cluster, which can be costly and may not be necessary for a one-time job.

  Option D also does not support querying data from RDS directly, so you would need to use Redshift Federated Query to access RDS data, which adds another layer of complexity.

• Question 155:

  A company stores time-series data that is collected from streaming services in an Amazon S3 bucket. The company must ensure that only workloads that are deployed within the company's VPC can access the data.

  Which solution will meet this requirement?

  A. Create an S3 bucket policy that uses a condition to allow access only to traffic that originates from the company's VPC.
  B. Apply a security group to the S3 bucket that allows connections only from the company's VPC CIDR block.
  C. Define an IAM policy that denies access to all users unless the request originates from within the company's VPC.
  D. Use a network ACL on the VPC subnets to allow only specific resources to access the S3 bucket.
  A. Create an S3 bucket policy that uses a condition to allow access only to traffic that originates from the company's VPC.

• Question 156:

  A retail company uses Amazon Aurora PostgreSQL to process and store live transactional data. The company uses an Amazon Redshift cluster for a data warehouse.

  An extract, transform, and load (ETL) job runs every morning to update the Redshift cluster with new data from the PostgreSQL database. The company has grown rapidly and needs to cost optimize the Redshift cluster.

  A data engineer needs to create a solution to archive historical data. The data engineer must be able to run analytics queries that effectively combine data from live transactional data in PostgreSQL, current data in Redshift, and archived historical data. The solution must keep only the most recent 15 months of data in Amazon Redshift to reduce costs.

  Which combination of steps will meet these requirements? (Choose Two.)

  A. Configure the Amazon Redshift Federated Query feature to query live transactional data that is in the PostgreSQL database.
  B. Configure Amazon Redshift Spectrum to query live transactional data that is in the PostgreSQL database.
  C. Schedule a monthly job to copy data that is older than 15 months to Amazon S3 by using the UNLOAD command. Delete the old data from the Redshift cluster. Configure Amazon Redshift Spectrum to access historical data in Amazon S3.
  D. Schedule a monthly job to copy data that is older than 15 months to Amazon S3 Glacier Flexible Retrieval by using the UNLOAD command. Delete the old data from the Redshift duster. Configure Redshift Spectrum to access historical data from S3 Glacier Flexible Retrieval.
  E. Create a materialized view in Amazon Redshift that combines live, current, and historical data from different sources.
  A. Configure the Amazon Redshift Federated Query feature to query live transactional data that is in the PostgreSQL database.
  C. Schedule a monthly job to copy data that is older than 15 months to Amazon S3 by using the UNLOAD command. Delete the old data from the Redshift cluster. Configure Amazon Redshift Spectrum to access historical data in Amazon S3.

  ---

  Explanation

  The goal is to archive historical data from an Amazon Redshift data warehouse while combining live transactional data from Amazon Aurora PostgreSQL with current andhistorical data in a cost-efficient manner. The company wants to keep only the last 15 months of data in Redshift to reduce costs.

  Option A: "Configure the Amazon Redshift Federated Query feature to query live transactional data that is in the PostgreSQL database."Redshift Federated Queryallows querying live transactional data directly from Aurora PostgreSQL without having to move it into Redshift, thereby enabling seamless integration of the current data in Redshift and live data in PostgreSQL. This is a cost-effective approach, as it avoids unnecessary data duplication.

  Option C: "Schedule a monthly job to copy data that is older than 15 months to Amazon S3 by using the UNLOAD command. Delete the old data from the Redshift cluster. Configure Amazon Redshift Spectrum to access historical data in Amazon S3."This option usesAmazon Redshift Spectrum , which enables Redshift to query data directly in S3 without moving it into Redshift. By unloading older data (older than 15 months) to S3, and then using Spectrum to access it, this approach reduces storage costs significantly while still allowing the data to be queried when necessary.

  Option B (Redshift Spectrum for live PostgreSQL data)is not applicable, as Redshift Spectrum is intended for querying data in Amazon S3, not live transactional data in Aurora.

  Option D (S3 Glacier Flexible Retrieval)is not suitable because Glacier is designed for long-term archival storage with infrequent access, and querying data in Glacier for analytics purposes would incur higher retrieval times and costs.

  Option E (materialized views)would not meet the need to archive data or combine it from multiple sources; it is best suited for combining frequently accessed data already in Redshift.

  References:

  Amazon Redshift Federated Query

  Amazon Redshift Spectrum Documentation Amazon Redshift UNLOAD Command

• Question 157:

  A technology company currently uses Amazon Kinesis Data Streams to collect log data in real time. The company wants to use Amazon Redshift for downstream real-time queries and to enrich the log data.

  Which solution will ingest data into Amazon Redshift with the LEAST operational overhead?

  A. Set up an Amazon Data Firehose delivery stream to send data to a Redshift provisioned cluster table.
  B. Set up an Amazon Data Firehose delivery stream to send data to Amazon S3. Configure a Redshift provisioned cluster to load data every minute.
  C. Configure Amazon Managed Service for Apache Flink (previously known as Amazon Kinesis Data Analytics) to send data directly to a Redshift provisioned cluster table.
  D. Use Amazon Redshift streaming ingestion from Kinesis Data Streams and to present data as a materialized view.
  D. Use Amazon Redshift streaming ingestion from Kinesis Data Streams and to present data as a materialized view.

  ---

  Explanation

  The most efficient and low-operational-overhead solution for ingesting data into Amazon Redshift from Amazon Kinesis Data Streams is to use Amazon Redshift streaming ingestion . This feature allows Redshift to directly ingest streaming data from Kinesis Data Streams and process it in real-time.

  Amazon Redshift Streaming Ingestion:

  Redshift supports native streaming ingestion from Kinesis Data Streams, allowing real-time data to be queried using materialized views . This solution reduces operational complexity because you don't need intermediary services like Amazon Kinesis Data Firehose or S3 for batch loading.

• Question 158:

  A data governance team must reduce the risk of exposing sensitive customer data in an S3 data lake. The team needs to discover potential PII and enforce fine-grained access to the governed tables.

  Which services should the team use? (Choose two.)

  A. Amazon Macie
  B. AWS Lake Formation
  C. AWS Budgets
  D. Amazon Route 53
  E. AWS CodeDeploy
  A. Amazon Macie
  B. AWS Lake Formation

  ---

  Explanation

  Amazon Macie can help discover sensitive data such as PII in S3, and Lake Formation can enforce fine-grained permissions for governed data lake tables. Budgets tracks cost. Route 53 provides DNS.

  CodeDeploy automates application deployment and does not discover or govern data lake PII.

• Question 159:

  A data engineer maintains custom Python scripts that perform a data formatting process that many AWS Lambda functions use. When the data engineer needs to modify the Python scripts, the data engineer must manually update all the Lambda functions.

  The data engineer requires a less manual way to update the Lambda functions.

  Which solution will meet this requirement?

  A. Store the custom Python scripts in a shared Amazon S3 bucket. Store a pointer to the custom scripts in the execution context object.
  B. Package the custom Python scripts into Lambda layers. Apply the Lambda layers to the Lambda functions.
  C. Store the custom Python scripts in a shared Amazon S3 bucket. Store a pointer to the customer scripts in environment variables.
  D. Assign the same alias to each Lambda function. Call each Lambda function by specifying the function's alias.
  B. Package the custom Python scripts into Lambda layers. Apply the Lambda layers to the Lambda functions.

  ---

  Explanation

  Lambda layers allow you to package external libraries, custom scripts, or dependencies (like Python scripts) separately from the Lambda function code. By using Lambda layers, you can update the custom Python scripts in one central place (the layer) without manually updating each Lambda function. When the layer is updated, all Lambda functions that use the layer automatically get the latest version of the scripts.

• Question 160:

  A data engineer must orchestrate a data pipeline that consists of one AWS Lambda function and one AWS Glue job. The solution must integrate with AWS services.

  Which solution will meet these requirements with the LEAST management overhead?

  A. Use an AWS Step Functions workflow that includes a state machine. Configure the state machine to run the Lambda function and then the AWS Glue job.
  B. Use an Apache Airflow workflow that is deployed on an Amazon EC2 instance. Define a directed acyclic graph (DAG) in which the first task is to call the Lambda function and the second task is to call the AWS Glue job.
  C. Use an AWS Glue workflow to run the Lambda function and then the AWS Glue job.
  D. Use an Apache Airflow workflow that is deployed on Amazon Elastic Kubernetes Service (Amazon EKS). Define a directed acyclic graph (DAG) in which the first task is to call the Lambda function and the second task is to call the AWS Glue job.
  A. Use an AWS Step Functions workflow that includes a state machine. Configure the state machine to run the Lambda function and then the AWS Glue job.

  ---

  Explanation

  AWS Step Functions is a service that allows you to coordinate multiple AWS services into serverless workflows. You can use Step Functions to create state machines that define the sequence and logic of the tasks in your workflow. Step Functions supports various types of tasks, such as Lambda functions, AWS Glue jobs, Amazon EMR clusters, Amazon ECS tasks, etc. You can use Step Functions to monitor and troubleshoot your workflows, as well as to handle errors and retries.

  Using an AWS Step Functions workflow that includes a state machine to run the Lambda function and then the AWS Glue job will meet the requirements with the leastmanagement overhead, as it leverages the serverless and managed capabilities of Step Functions. You do not need to write any code to orchestrate the tasks in your workflow, as you can use the Step Functions console or the AWS Serverless Application Model (AWS SAM) to define and deploy your state machine. You also do not need to provision or manage any servers or clusters, as Step Functions scales automatically based on the demand.

  The other options are not as efficient as using an AWS Step Functions workflow. Using an Apache Airflow workflow that is deployed on an Amazon EC2 instance or on Amazon Elastic Kubernetes Service (Amazon EKS) will require more management overhead, as you will need to provision, configure, and maintain the EC2 instance or the EKS cluster, as well as the Airflow components. You will also need to write and maintain the Airflow DAGs to orchestrate the tasks in your workflow. Using an AWS Glue workflow to run the Lambda function and then the AWS Glue job will not work, as AWS Glue workflows only support AWS Glue jobs and crawlers as tasks, not Lambda functions.