Amazon DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Amazon DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 391:

  A company uses an Amazon RDS for Oracle database to store data for an ecommerce application. The company needs a solution to make the data in all tables available for future analytics. The solution must process data from a table named customer_complaints in near real time by using a REST API.

  Which solution will meet these requirements MOST cost-effectively?

  A. Use AWS Database Migration Service (AWS DMS) to replicate the Oracle database to an Amazon RDS for PostgreSQL database instance. Create an AWS Lambda function to process data from the PostgreSQL customer_complaints table and to call the REST API.
  B. Create a task in AWS Database Migration Service (AWS DMS) to replicate the Oracle database to Amazon Redshift. Set the Oracle database as the source. Configure an AWS Lambda function to use the Amazon Redshift Data API to query the data from the customer_complaints table and to call the REST API. Unload the data from Amazon Redshift to Amazon S3.
  C. Set up one AWS Database Migration Service (AWS DMS) task to stream the customer_complaints table data to Amazon Kinesis Data Streams. Set up a second AWS DMS task to send data to an Amazon S3 bucket. Configure an AWS Lambda function to process Kinesis dataand to call the REST API.
  D. Create a read replica of the Amazon RDS for Oracle database. Configure an AWS Lambda function to process data from the read replica customer_complaints table and to call the REST API.
  C. Set up one AWS Database Migration Service (AWS DMS) task to stream the customer_complaints table data to Amazon Kinesis Data Streams. Set up a second AWS DMS task to send data to an Amazon S3 bucket. Configure an AWS Lambda function to process Kinesis dataand to call the REST API.

• Question 392:

  A data engineer is configuring an AWS Glue job to read data from an Amazon S3 bucket via a VPC gateway endpoint but encounters connection errors.

  Which solution will resolve the issue?

  A. Update the AWS Glue security group to allow inbound traffic from the S3 VPC gateway endpoint
  B. Configure an S3 bucket policy to grant the Glue job access permissions
  C. Verify that the VPC's route table includes inbound and outbound routes for the S3 VPC gateway endpoint
  D. Ensure the Glue connection details include a fully qualified domain name
  C. Verify that the VPC's route table includes inbound and outbound routes for the S3 VPC gateway endpoint

• Question 393:

  A company stores 10 to 15 TB of uncompressed .csv les in Amazon S3. The company is evaluating Amazon Athena as a one-time query engine.

  The company wants to transform the data to optimize query runtime and storage costs.

  Which file format and compression solution will meet these requirements for Athena queries?

  A. .csv format compressed with zip
  B. JSON format compressed with bzip2
  C. Apache Parquet format compressed with Snappy
  D. Apache Avro format compressed with LZO
  C. Apache Parquet format compressed with Snappy

• Question 394:

  A company needs to use Amazon Athena to analyze data that is in an Amazon S3 bucket. A data engineer needs to configure AWS Glue table partitions for year, month, and day. The data engineer needs to create the partitions every day to adjust to schema changes in the data.

  Which solution will meet these requirements?

  A. Use AWS Glue DataBrew to create the partitions for the AWS Glue table.
  B. Use an AWS Lambda function to create the partitions for the AWS Glue table.
  C. Set partition projection properties for the AWS Glue table.
  D. Configure an AWS Glue crawler to run on a set schedule.
  C. Set partition projection properties for the AWS Glue table.

  ---

  Explanation

  Partition projection in AWS Glue lets Athena automatically calculate partitions based on table properties without explicitly creating them each day. This eliminates the need for crawlers or Lambda jobs, reduces operational overhead, and adapts to schema changes while supporting year, month, and day partitions efficiently.

• Question 395:

  A company uses Amazon Redshift to store order transactions from the current day. The company has an orders table that contains the previous order data. The company also has a staging table that contains new or updated order records.

  The company needs to remove stale records from the orders table and insert the most recent data in the orders table from the staging table. Several downstream applications need the orders table to display up-to-date information.

  Which solution will meet these requirements?

  A. Use Amazon Redshift Spectrum to delete stale records from the orders table and insert records from the staging table into the orders table.
  B. Unload the orders table and the staging table to Amazon S3. Delete stale orders table data and insert new staging table data in Amazon S3 by using Amazon Athena. Copy the orders S3 table to the orders Amazon Redshift table.
  C. Use Amazon Athena federated queries to read stale records from the orders table. Delete the stale records and insert the records from the staging table into the orders table.
  D. Write an Amazon Redshift stored procedure that deletes the stale records from the orders table and inserts new records from the staging table.
  D. Write an Amazon Redshift stored procedure that deletes the stale records from the orders table and inserts new records from the staging table.

• Question 396:

  Which execution class should a data engineer choose for AWS Glue jobs that don't require specific run/ finish times to be cost-effective?

  A. FLEX execution class
  B. Spot Instance type
  C. STANDARD execution class
  D. Latest GlueVersion
  A. FLEX execution class

• Question 397:

  A transportation company wants to track vehicle movements by capturing geolocation records. The records are 10 bytes in size. The company receives up to 10,000 records every second. Data transmission delays of a few minutes are acceptable because of unreliable network conditions.

  The transportation company wants to use Amazon Kinesis Data Streams to ingest the geolocation data.

  The company needs a reliable mechanism to send data to Kinesis Data Streams. The company needs to maximize the throughput efficiency of the Kinesis shards.

  Which solution will meet these requirements in the MOST operationally efficient way?

  A. Kinesis Agent
  B. Kinesis Producer Library (KPL)
  C. Amazon Kinesis Data Firehose
  D. Kinesis SDK
  B. Kinesis Producer Library (KPL)

  ---

  Explanation

  Problem Analysis:

  The company ingests geolocation records (10 bytes each) at 10,000 records per second into Kinesis Data Streams .

  Data transmission delays are acceptable, but the solution must maximize throughput efficiency.

  Key Considerations:

  The Kinesis Producer Library (KPL) batches records and uses aggregation to optimize shard throughput.

  Efficiently handles high-throughput scenarios with minimal operational overhead.

  Solution Analysis:

  Option A: Kinesis Agent Designed for file-based ingestion; not optimized for geolocation records.

  Option B: KPL

  Aggregates records into larger payloads, significantly improving shard throughput.

  Suitable for applications generating small, high-frequency records.

  Option C: Kinesis Firehose

  Firehose is for delivery to destinations like S3 or Redshift and is not optimized for direct ingestion to Kinesis Data Streams.

  Option D: Kinesis SDK

  The SDK lacks advanced features like aggregation, resulting in lower throughput efficiency.

  Final Recommendation:

  Use Kinesis Producer Library (KPL) for its built-in aggregation and batching capabilities.

  References:

  Kinesis Producer Library (KPL) Overview Best Practices for Amazon Kinesis

• Question 398:

  A company is planning to use a provisioned Amazon EMR cluster that runs Apache Spark jobs to perform big data analysis. The company requires high reliability. A big data team must follow best practices for running cost-optimized and long-running workloads on Amazon EMR. The team must find a solution that will maintain the company's current level of performance.

  Which combination of resources will meet these requirements MOST cost-effectively? (Choose two.)

  A. Use Hadoop Distributed File System (HDFS) as a persistent data store.
  B. Use Amazon S3 as a persistent data store.
  C. Use x86-based instances for core nodes and task nodes.
  D. Use Graviton instances for core nodes and task nodes.
  E. Use Spot Instances for all primary nodes.
  B. Use Amazon S3 as a persistent data store.
  D. Use Graviton instances for core nodes and task nodes.

  ---

  Explanation

  The best combination of resources to meet the requirements of high reliability, cost-optimization, and performance for running Apache Spark jobs on Amazon EMR is to useAmazon S3 as a persistent data store and Graviton instances for core nodes and task nodes.

  Amazon S3 is a highly durable, scalable, and secure object storage service that can store any amount of data for a variety of use cases, including big data analytics. Amazon S3 is a better choice than HDFS as a persistent data store for Amazon EMR, as it decouples the storage from the compute layer, allowing for more flexibility and cost-efficiency. Amazon S3 also supports data encryption, versioning, lifecycle management, and cross-region replication. Amazon EMR integrates seamlessly with Amazon S3, using EMR File System (EMRFS) to access data stored in Amazon S3 buckets. EMRFS also supports consistent view, which enables Amazon EMR to provide read-after-write consistency for Amazon S3 objects that are accessed through EMRFS.

  Graviton instances are powered by Arm-based AWS Graviton processors that deliver up to 40% better price performance over comparable current generation x86-based instances. Graviton instances are ideal for running workloads that are CPU-bound, memory-bound, or network-bound, such as big data analytics, web servers, and open-source databases. Graviton instances are compatible with Amazon EMR, and can be used for both core nodes and task nodes. Core nodes are responsible for running the data processing frameworks, such as Apache Spark, and storing data in HDFS or the local file system. Task nodes are optional nodes that can be added to a cluster to increase the processing power and throughput. By using Graviton instances for both core nodes and task nodes, you can achieve higher performance and lower cost than using x86-based instances. Using Spot Instances for all primary nodes is not a good option, as it can compromise the reliability and availability of the cluster. Spot Instances are spare EC2 instances that are available at up to 90% discount compared to On-Demand prices, but they can be interrupted by EC2 with a two-minute notice when EC2 needs the capacity back. Primary nodes are the nodes that run the cluster software, such as Hadoop, Spark, Hive, and Hue, and are essential for the cluster operation. If a primary node is interrupted by EC2, the cluster will fail or become unstable. Therefore, it is recommended to use On-Demand Instances or Reserved Instances for primary nodes, and use Spot Instances only for task nodes that can tolerate interruptions.

  References:

  Amazon S3 - Cloud Object Storage

  EMR File System (EMRFS)

  AWS Graviton2 Processor-Powered Amazon EC2 Instances [Plan and Configure EC2 Instances] [Amazon EC2 Spot Instances] [Best Practices for Amazon EMR]

• Question 399:

  A company uses a variety of AWS and third-party data stores. The company wants to consolidate all the data into a central data warehouse to perform analytics. Users need fast response times for analytics queries.

  The company uses Amazon QuickSight in direct query mode to visualize the data. Users normally run queries during a few hours each day with unpredictable spikes.

  Which solution will meet these requirements with the LEAST operational overhead?

  A. Use Amazon Redshift Serverless to load all the data into Amazon Redshift managed storage (RMS).
  B. Use Amazon Athena to load all the data into Amazon S3 in Apache Parquet format.
  C. Use Amazon Redshift provisioned clusters to load all the data into Amazon Redshift managed storage (RMS).
  D. Use Amazon Aurora PostgreSQL to load all the data into Aurora.
  A. Use Amazon Redshift Serverless to load all the data into Amazon Redshift managed storage (RMS).

  ---

  Explanation

  Problem Analysis:

  The company requires a centralized data warehouse for consolidating data from various sources.

  They use Amazon QuickSight in direct query mode , necessitating fast response times for analytical queries.

  Users query the data intermittently, with unpredictable spikes during the day.

  Operational overhead should be minimal.

  Key Considerations:

  The solution must support fast, SQL-based analytics.

  It must handle unpredictable spikes efficiently.

  Must integrate seamlessly with QuickSight for direct querying.

  Minimize operational complexity and scaling concerns.

  Solution Analysis:

  Option A: Amazon Redshift Serverless Redshift Serverless eliminates the need for provisioning and managing clusters.

  Automatically scales compute capacity up or down based on query demand.

  Reduces operational overhead by handling performance optimization.

  Fully integrates with Amazon QuickSight , ensuring low-latency analytics.

  Reduces costs as it charges only for usage, making it ideal for workloads with intermittent spikes.

  Option B: Amazon Athena with S3 (Apache Parquet)

  Athena supports querying data directly from S3 in Parquet format.

  While it's cost-effective, performance depends on the size and complexity of the data.

  It is not optimized for high-speed analytics needed by QuickSight in direct query mode.

  Option C: Amazon Redshift Provisioned Clusters

  Requires manual cluster provisioning, scaling, and maintenance.

  Higher operational overhead compared to Redshift Serverless.

  Option D: Amazon Aurora PostgreSQL

  Aurora is optimized for transactional databases, not data warehousing or analytics.

  Does not meet the requirement for fast analytics queries.

  Final Recommendation:

  Amazon Redshift Serverlessis the best choice for this use case because it provides fast analytics, integrates natively with QuickSight, and minimizes operational complexity while efficiently handling unpredictable spikes.

  References:

  Amazon Redshift Serverless Overview Amazon QuickSight and Redshift Integration Athena vs. Redshift

• Question 400:

  A data engineer is using an Apache Iceberg framework to build a data lake that contains 100 TB of data.

  The data engineer wants to run AWS Glue Apache Spark jobs that use the Iceberg framework.

  What combination of steps will meet these requirements? (Choose two.)

  A. Create a key named --conf for an AWS Glue job. Set Iceberg as a value for the --datalake-formats job parameter.
  B. Specify the path to a specific version of Iceberg by using the --extra-jars job parameter. Set Iceberg as a value for the --datalake-formats job parameter.
  C. Set Iceberg as a value for the --datalake-formats job parameter.
  D. Set the --enable-auto-scaling parameter to true.
  E. Add the --job-bookmark-option job-bookmark-enable parameter to an AWS Glue job.
  B. Specify the path to a specific version of Iceberg by using the --extra-jars job parameter. Set Iceberg as a value for the --datalake-formats job parameter.
  C. Set Iceberg as a value for the --datalake-formats job parameter.

  ---

  Explanation

  To run AWS Glue Apache Spark jobs with the Apache Iceberg framework, you must: Set --datalake-formats to iceberg so Glue knows to use the Iceberg format.

  Specify the Iceberg library by using the -extra-jars parameter to include the appropriate Iceberg JAR files for Glue to access the Iceberg table format.