Amazon DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Amazon DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 31:

  A data engineer needs to run a data transformation job whenever a user adds a file to an Amazon S3 bucket. The job will run for le than 1 minute. The job must send the output through an email meage to the data engineer. The data engineer expects users to add one file every hour of the day.

  Which solution will meet these requirements in the MOST operationally efficient way?

  A. Create a small Amazon EC2 instance that polls the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email meages.
  B. Run an Amazon Elastic Container Service (Amazon ECS) task to poll the S3 bucket for new files. Run transformation code on a schedule to generate the output. Use operating system commands to send email meages.
  C. Create an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to invoke the Lambda function when a new object is created. Publish the output to an Amazon Simple Notification Service (Amazon SNS) topic. Subscribe the data engineer's email account to the topic.
  D. Deploy an Amazon EMR cluster. Use EMR File System (EMRFS) to acce the files in the S3 bucket. Run transformation code on a schedule to generate the output to a second S3 bucket. Create an Amazon Simple Notification Service (Amazon SNS) topic. Configure Amazon S3 Event Notifications to notify the topic when a new object is created.
  C. Create an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to invoke the Lambda function when a new object is created. Publish the output to an Amazon Simple Notification Service (Amazon SNS) topic. Subscribe the data engineer's email account to the topic.

  ---

  Explanation

  An S3 event notification on ObjectCreated can directly invoke a Lambda function as soon as a new file arrives. Lambda can perform your <1 minute transformation and then publish the result to an SNS topic.

  With the data engineer subscribed to that topic by email, the output is delivered automatically - no servers to manage, no polling, and minimal ongoing operations.

• Question 32:

  h A company uses Amazon Retdshift for its data warehouse. A data engineer must query a table named orders.complete_orders_history, which contains 100 columns.

  The query must return all columns except columns named companyId and uniq/ue/_system_id.

  Which Amazon Redshift SQL statement will meet this requirement?

  

  A. Option A
  B. Option B
  C. Option C
  D. Option D
  A. Option A

• Question 33:

  A company receives test results from testing facilities that are located around the world. The company stores the test results in millions of 1 KB JSON files in an Amazon S3 bucket. A data engineer needs to process the files, convert them into Apache Parquet format, and load them into Amazon Redshift tables.

  The data engineer uses AWS Glue to process the files, AWS Step Functions to orchestrate the processes, and Amazon EventBridge to schedule jobs.

  The company recently added more testing facilities. The time required to process files is increasing. The data engineer must reduce the data processing time.

  Which solution will MOST reduce the data processing time?

  A. Use AWS Lambda to group the raw input files into larger files. Write the larger files back to Amazon S3. Use AWS Glue to process the files. Load the files into the Amazon Redshift tables.
  B. Use the AWS Glue dynamic frame file-grouping option to ingest the raw input files. Process the files. Load the files into the Amazon Redshift tables.
  C. Use the Amazon Redshift COPY command to move the raw input files from Amazon S3 directly into the Amazon Redshift tables. Process the files in Amazon Redshift.
  D. Use Amazon EMR instead of AWS Glue to group the raw input files. Process the files in Amazon EMR. Load the files into the Amazon Redshift tables.
  B. Use the AWS Glue dynamic frame file-grouping option to ingest the raw input files. Process the files. Load the files into the Amazon Redshift tables.

  ---

  Explanation

  Problem Analysis:

  Millions of 1 KB JSON files in S3 are being processed and converted to Apache Parquet format using AWS Glue.

  Processing time is increasing due to the additional testing facilities.

  The goal is to reduce processing time while using the existing AWS Glue framework.

  Key Considerations:

  AWS Glue offers the dynamic frame file-grouping feature, which consolidates small files into larger, more efficient datasets during processing.

  Grouping smaller files reduces overhead and speeds up processing.

  Solution Analysis:

  Option A: Lambda for File Grouping Using Lambda to group files would add complexity and operational overhead. Glue already offers built-in grouping functionality.

  Option B: AWS Glue Dynamic Frame File-Grouping

  This option directly addresses the issue by grouping small files during Glue job execution.

  Minimizes data processing time with no extra overhead.

  Option C: Redshift COPY Command COPY directly loads raw files but is not designed for pre-processing (conversion to Parquet).

  Option D: Amazon EMR

  While EMR is powerful, replacing Glue with EMR increases operational complexity.

  Final Recommendation:

  Use AWS Glue dynamic frame file-grouping for optimized data ingestion and processing.

  References:

  AWS Glue Dynamic Frames

  Optimizing Glue Performance

• Question 34:

  A data engineer uses AWS Lake Formation to manage access to data that is stored in an Amazon S3 bucket. The data engineer configures an AWS Glue crawler to discover data at a specific file location in the bucket, s3://examplepath. The crawler execution fails with the following error: "The S3 location: s3:// examplepath is not registered."

  The data engineer needs to resolve the error.

  Which solution will meet this requirement?

  A. Attach an appropriate IAM policy to the IAM role of the AWS Glue crawler to grant the crawler permiion to read the S3 location.
  B. Register the S3 location in Lake Formation to allow the crawler to acce the data.
  C. Create a new AWS Glue database. Aign the correct permiions to the database for the crawler.
  D. Configure the S3 bucket policy to allow cro-account acce.
  B. Register the S3 location in Lake Formation to allow the crawler to acce the data.

  ---

  Explanation

  In Lake Formation, you must explicitly register each S3 path as a data location before any Glue crawler or user can acce it. Registering s3://examplepath in Lake Formation grants the crawler the metadata permiions it needs to crawl that location.

• Question 35:

  A nance company uses Amazon Redshift as a data warehouse. The company stores the data in a shared Amazon S3 bucket. The company uses Amazon Redshift Spectrum to access the data that is stored in the S3 bucket. The data comes from certi ed third-party data providers. Each third-party data provider has unique connection details.

  To comply with regulations, the company must ensure that none of the data is accessible from outside the company's AWS environment.

  Which combination of steps should the company take to meet these requirements? (Choose two.)

  A. Replace the existing Redshift cluster with a new Redshift cluster that is in a private subnet. Use an interface VPC endpoint to connect to the Redshift cluster. Use a NAT gateway to give Redshift access to the S3 bucket.
  B. Create an AWS CloudHSM hardware security module (HSM) for each data provider. Encrypt each data provider's data by using the corresponding HSM for each data provider.
  C. Turn on enhanced VPC routing for the Amazon Redshift cluster. Set up an AWS Direct Connect connection and configure a connection between each data provider and the nance company's VPC.
  D. De fine table constraints for the primary keys and the foreign keys.
  E. Use federated queries to access the data from each data provider. Do not upload the data to the S3 bucket. Perform the federated queries through a gateway VPC endpoint.
  A. Replace the existing Redshift cluster with a new Redshift cluster that is in a private subnet. Use an interface VPC endpoint to connect to the Redshift cluster. Use a NAT gateway to give Redshift access to the S3 bucket.
  C. Turn on enhanced VPC routing for the Amazon Redshift cluster. Set up an AWS Direct Connect connection and configure a connection between each data provider and the nance company's VPC.

• Question 36:

  A data engineer must implement a data cataloging solution to track schema changes in an Amazon Redshift table.

  Which solution will meet these requirements?

  A. Schedule an AWS Glue crawler to run every day on the table by using the Java Database Connectivity (JDBC) driver. Configure the crawler to update an AWS Glue Data Catalog.
  B. Use AWS DataSync to log the table metadata to an AWS Glue Data Catalog. Use an AWS Glue crawler to update the Data Catalog every day.
  C. Use the AWS Schema Conversion Tool (AWS SCT) to log the table metadata to an Apache Hive metastore. Use Amazon EventBridge Scheduler to update the metastore every day.
  D. Schedule an AWS Glue crawler to run every day on the table. Configure the crawler to update an Apache Hive metastore.
  A. Schedule an AWS Glue crawler to run every day on the table by using the Java Database Connectivity (JDBC) driver. Configure the crawler to update an AWS Glue Data Catalog.

  ---

  Explanation

  An AWS Glue crawler can connect to Amazon Redshift via JDBC, detect schema changes, and automatically update the AWS Glue Data Catalog on a schedule, providing the required ongoing schema tracking.

• Question 37:

  A financial company wants to run real-time queries on Amazon Redshift from a web-based trading app with minimal operational overhead.

  Which solution to choose?

  A. Establish WebSocket connections to Redshift
  B. Use the Amazon Redshift Data API
  C. Set up JDBC connections to Redshift
  D. Store frequent data in S3 and use S3 Select
  B. Use the Amazon Redshift Data API

• Question 38:

  A global ecommerce company occasionally receives customer data files in its Amazon S3 data lake. The company needs to automatically detect new data and mask sensitive data before making the data available to the company's analytics team.

  Which solution will meet these requirements with the LEAST operational overhead?

  A. Configure Amazon S3 Event Notifications to detect the new data and trigger an AWS Glue job. Use Amazon Macie to detect and mask the sensitive data. Store the processed data in Amazon Redshift.
  B. Use Amazon Managed Streaming for Apache Kafka (Amazon MSK) to detect incoming data. Use Amazon EMR workflows to detect and mask sensitive data. Store the processed data in Amazon S3.
  C. Use Amazon Kinesis Data Streams to capture new data. Use Amazon Comprehend to detect and mask the sensitive data. Store the processed data in Amazon DynamoDB tables.
  D. Use Amazon EventBridge to detect new data and run AWS Glue workflows. Use AWS Glue DataBrew to detect and mask the sensitive data. Store the processed data in an S3 bucket.
  D. Use Amazon EventBridge to detect new data and run AWS Glue workflows. Use AWS Glue DataBrew to detect and mask the sensitive data. Store the processed data in an S3 bucket.

• Question 39:

  A media streaming company is moving its application to AWS. The company wants to avoid data quality issues in user engagement analytics. The company needs to be able to reprocess several days of historical data without duplicates and without interrupting current real-time processing.

  Which solution will meet these requirements?

  A. Configure Amazon Kinesis Data Streams with extended retention. Implement idempotent consumers with Amazon DynamoDB checkpointing. Use separate processing paths for replay data and real-time data.
  B. Configure Amazon Kinesis Data Streams with standard retention. Implement idempotent consumers with Amazon RDS checkpointing. Use separate processing paths with conditional logic for replay data and real-time data.
  C. Configure Amazon Kinesis Data Streams with standard retention. Implement idempotent consumers with Amazon DynamoDB checkpointing. Use shared processing paths with conditional logic for replay data and real-time data.
  D. Configure Amazon Kinesis Data Streams with extended retention. Implement idempotent consumers with Amazon RDS checkpointing. Use shared processing paths for replay data and real-time data.
  A. Configure Amazon Kinesis Data Streams with extended retention. Implement idempotent consumers with Amazon DynamoDB checkpointing. Use separate processing paths for replay data and real-time data.

• Question 40:

  A data engineer needs to design Amazon S3 retention for compliance data. Current objects must transition to colder storage after 180 days. Noncurrent versions must be deleted when they are no longer needed.

  Incomplete multipart uploads should not accumulate.

  Which S3 Lifecycle actions can help meet these requirements? (Choose three.)

  A. Transition current object versions to another storage class after a specified number of days.
  B. Expire noncurrent object versions based on lifecycle criteria.
  C. Abort incomplete multipart uploads after a specified number of days.
  D. Create IAM users for each object prefix.
  E. Run MSCK REPAIR TABLE after each object transition.
  F. Enable Lambda reserved concurrency on the bucket.
  A. Transition current object versions to another storage class after a specified number of days.
  B. Expire noncurrent object versions based on lifecycle criteria.
  C. Abort incomplete multipart uploads after a specified number of days.

  ---

  Explanation

  S3 Lifecycle can transition current versions, expire noncurrent versions, and abort incomplete multipart uploads. IAM users are not a lifecycle mechanism. Athena partition repair is unrelated to object retention.

  Lambda reserved concurrency controls Lambda invocation limits, not S3 storage lifecycle.