Amazon DATA-ENGINEER-ASSOCIATE Online Practice Questions and Exam Preparation

Amazon DATA-ENGINEER-ASSOCIATE Online Questions & Answers

• Question 351:

  A media company wants to improve a system that recommends media content to customer based on user behavior and preferences. To improve the recommendation system, the company needs to incorporate insights from third-party datasets into the company's existing analytics platform.

  The company wants to minimize the effort and time required to incorporate third-party datasets.

  Which solution will meet these requirements with the LEAST operational overhead?

  A. Use API calls to access and integrate third-party datasets from AWS Data Exchange.
  B. Use API calls to access and integrate third-party datasets from AWS
  C. Use Amazon Kinesis Data Streams to access and integrate third-party datasets from AWS CodeCommit repositories.
  D. Use Amazon Kinesis Data Streams to access and integrate third-party datasets from Amazon Elastic Container Registry (Amazon ECR).
  A. Use API calls to access and integrate third-party datasets from AWS Data Exchange.

  ---

  Explanation

  AWS Data Exchange is a service that makes it easy to find, subscribe to, and use third-party data in the cloud. It provides a secure and reliable way to access and integrate data from various sources, such as data providers, public datasets, or AWS services. Using AWS Data Exchange, you can browse and subscribe to data products that suit your needs, and then use API calls or the AWS Management Console to export the data to Amazon S3, where you can use it with your existing analytics platform. This solution minimizes the effort and time required to incorporate third-party datasets, as you do not need to set up and manage data pipelines, storage, or access controls. You also benefit from the data quality and freshness provided by the data providers, who can update their data products as frequently as needed.

  The other options are not optimal for the following reasons:

  Option B: Use API calls to access and integrate third-party datasets from AWS. This option is vague and does not specify which AWS service or feature is used to access and integrate third-party datasets. AWS offers a variety of services and features that can help with data ingestion, processing, and analysis, but not all of them are suitable for the given scenario. For example, AWS Glue is a serverless data integration service that canhelp you discover, prepare, and combine data from various sources, but it requires you to create and run data extraction, transformation, and loading (ETL) jobs, which can add operational overhead.

  Option C: Use Amazon Kinesis Data Streams to access and integrate third-party datasets from AWS CodeCommit repositories. This option is not feasible, as AWS CodeCommit is a source control service that hosts secure Git-based repositories, not a data source that can be accessed by Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is a service that enables you to capture, process, and analyze data streams in real time, such as clickstream data, application logs, or IoT telemetry. It does not support accessing and integrating data from AWS CodeCommit repositories, which are meant for storing and managing code, not data .

  Option D: Use Amazon Kinesis Data Streams to access and integrate third-party datasets from Amazon Elastic Container Registry (Amazon ECR). This option is also not feasible, as Amazon ECR is a fully managed container registry service that stores, manages, and deploys container images, not a data source that can be accessed by Amazon Kinesis Data Streams.

  Amazon Kinesis Data Streams does not support accessing and integrating data from Amazon ECR, which is meant for storing and managing container images, not data.

  References:

  1: AWS Data Exchange User Guide

  2: AWS Data Exchange FAQs

  3: AWS Glue Developer Guide

  AWS CodeCommit User Guide

  Amazon Kinesis Data Streams Developer Guide

  Amazon Elastic Container Registry User Guide

  Build a Continuous Delivery Pipeline for Your Container Images with Amazon ECR as Source

• Question 352:

  A company uses an organization in AWS Organizations to manage multiple AWS accounts. The company uses an enhanced fanout data stream in Amazon Kinesis Data Streams to receive streaming data from multiple producers. The company runs the data stream in an account named Account A. The company wants to use an AWS Lambda function in an account named Account B to process the data from the data stream. The company creates a Lambda execution role in Account B that has permissions to access data from the data stream in Account A.

  What additional step must the company take to meet this requirement?

  C. Create a service control policy (SCP) to grant the data stream read access to the cross-account Lambda execution role. Attach the SCP to Account
  D. Add a resource-based policy to the cross-account Lambda function to grant the data stream read access to the function.
  B

  ---

  Explanation

  To enable cross-account Lambda processing of Kinesis Data Streams, the stream in Account A must explicitly allow the Lambda execution role from Account B. This is done by adding a resource-based policy on the Kinesis data stream to grant kinesis:SubscribeToShard and related read permissions to the cross-account role. Without this resource-based policy, the Lambda in Account B cannot consume the data.

• Question 353:

  A company generates yearly financial statements for customers and stores the statements in an Amazon

  S3 bucket. Customers rarely access the documents after 1 week. The company must retain the statements for 7 years. The statements must remain readily accessible for customers.

  Which solution will meet these requirements in the MOST cost-effective way?

  A. Create an S3 Lifecycle rule to transition objects to S3 Glacier Deep Archive after 7 days. Expire the objects after 7 years.
  B. Set the S3 bucket to use S3 Intelligent-Tiering when new objects are uploaded. Set objects to expire after 7 years.
  C. Create an S3 Lifecycle rule to transition objects to S3 Glacier Instant Retrieval after 7 days. Expire the objects after 7 years.
  D. Set the S3 bucket to use S3 Glacier Instant Retrieval when new objects are uploaded. Create an AWS Lambda function that runs daily to delete any objects that are older than 7 years.
  C. Create an S3 Lifecycle rule to transition objects to S3 Glacier Instant Retrieval after 7 days. Expire the objects after 7 years.

• Question 354:

  A company needs a solution to process streaming data by using Apache Spark in a Kubernetes environment. The solution must support event-driven scaling and optimize resource utilization. The company needs to integrate the solution with existing Kubernetes infrastructure deployed on Amazon Elastic Kubernetes Service (Amazon EKS).

  Which solution will meet these requirements with the LEAST operational overhead?

  A. Deploy self-managed Apache Spark on Amazon EKS by using custom auto-scaling policies based on Kubernetes Event-Driven Autoscaling (KEDA) operators.
  B. Use Amazon EMR on Amazon EKS with Kubernetes Event-Driven Autoscaling (KEDA) for event-driven scaling of Spark applications.
  C. Use Amazon EMR on Amazon EKS with the Kubernetes Cluster Autoscaler to scale Spark applications based on metrics events generated by the Kubernetes Metrics Server.
  D. Implement Apache Spark on Amazon EKS with AWS Auto Scaling groups and custom scaling policies based on metrics events collected by the Kubernetes Metrics Server.
  B. Use Amazon EMR on Amazon EKS with Kubernetes Event-Driven Autoscaling (KEDA) for event-driven scaling of Spark applications.

• Question 355:

  A data engineer runs Amazon Athena queries on data that is in an Amazon S3 bucket. The Athena queries use AWS Glue Data Catalog as a metadata table.

  The data engineer notices that the Athena query plans are experiencing a performance bottleneck. The data engineer determines that the cause of the performance bottleneck is the large number of partitions that are in the S3 bucket. The data engineer must resolve the performance bottleneck and reduce Athena query planning time.

  Which solutions will meet these requirements? (Choose two.)

  A. Create an AWS Glue partition index. Enable partition filtering.
  B. Bucket the data based on a column that the data have in common in a WHERE clause of the user query
  C. Use Athena partition projection based on the S3 bucket prefix.
  D. Transform the data that is in the S3 bucket to Apache Parquet format.
  E. Use the Amazon EMR S3DistCP utility to combine smaller objects in the S3 bucket into larger objects.
  A. Create an AWS Glue partition index. Enable partition filtering.
  C. Use Athena partition projection based on the S3 bucket prefix.

  ---

  Explanation

  The best solutions to resolve the performance bottleneck and reduce Athena query planning time are to create an AWS Glue partition index and enable partition filtering, and to use Athena partition projection based on the S3 bucket prefix.

  AWS Glue partition indexes are a feature that allows you to speed up query processing of highly partitioned tables cataloged in AWS Glue Data Catalog. Partition indexes are available for queries in Amazon EMR, Amazon Redshift Spectrum, and AWS Glue ETL jobs. Partition indexes are sublists of partition keys defined in the table. When you create a partition index, you specify a list of partition keys that already exist on a given table. AWS Glue then creates an index for the specified keys and stores it in the DataCatalog. When you run a query that filters on the partition keys, AWS Glue uses the partition index to quickly identify the relevant partitions without scanning the entire table metadata. This reduces the query planning time and improves the query performance.

  Athena partition projection is a feature that allows you to speed up query processing of highly partitioned tables and automate partition management. In partition projection, Athena calculates partition values and locations using the table properties that you configure directly on your table in AWS Glue. The table properties allow Athena to `project', or determine, the necessary partition information instead of having to do a more time-consuming metadata lookup in the AWS Glue Data Catalog. Because in-memory operations are often faster than remote operations, partition projection can reduce the runtime of queries against highly partitioned tables. Partition projection also automates partition management because it removes the need to manually create partitions in Athena, AWS Glue, or your external Hive metastore.

  Option B is not the best solution, as bucketing the data based on a column that the data have in common in a WHERE clause of the user query would not reduce the query planning time. Bucketing is a technique that divides data into buckets based on a hash function applied to a column. Bucketing can improve the performance of join queries by reducing the amount of data that needs to be shuffled between nodes.

  However, bucketing does not affect the partition metadata retrieval, which is the main cause of the performance bottleneck in this scenario.

  Option D is not the best solution, as transforming the data that is in the S3 bucket to Apache Parquet format would not reduce the query planning time. Apache Parquet is a columnar storage format that can improve the performance of analytical queries by reducing the amount of data that needs to be scanned and providing efficient compression and encoding schemes. However, Parquet does not affect the partition metadata retrieval, which is the main cause of the performance bottleneck in this scenario.

  Option E is not the best solution, as using the Amazon EMR S3DistCP utility to combine smaller objects in the S3 bucket into larger objects would not reduce the query planning time. S3DistCP is a tool that can copy large amounts of data between Amazon S3 buckets or from HDFS to Amazon S3. S3DistCP can also aggregate smaller files into larger files to improve the performance of sequential access. However, S3DistCP does not affect the partition metadata retrieval, which is the main cause of the performance bottleneck in this scenario.

• Question 356:

  A company runs concurrent analytical queries on Amazon Redshift tables multiple times each day. The queries require consistent data views three times each day. The company runs extract, transform, and load (ETL) operations that update dimension tables while the queries run.

  The company has noticed that the queries cause table-level locks during the ETL operations. The company's current solution experiences query timeouts and deadlocks during peak processing hours, which affects analytical reporting and on-demand analysis.

  Which solution will fix this issue?

  A. Use Amazon Redshift materialized views for analytical queries. Schedule ETL operations during off-peak hours to minimize lock contention.
  B. Configure Amazon Redshift federated queries to access source data directly. Use read replicas to isolate analytical workloads from ETL operations.
  C. Use Amazon Redshift Spectrum to query data in Amazon S3 for analytical workloads. Maintain ETL operations on Amazon Redshift tables with transaction isolation.
  D. Deploy separate Amazon Redshift clusters for ETL and analytics workloads. Use cross-database queries and data sharing to maintain data consistency.
  D. Deploy separate Amazon Redshift clusters for ETL and analytics workloads. Use cross-database queries and data sharing to maintain data consistency.

• Question 357:

  A company runs an extract, transform, and load (ETL) job in AWS Glue. The job processes personally identifiable information (PII) data and writes logs to an Amazon CloudWatch Logs log group. A data engineer needs to mask PII data in the CloudWatch logs group.

  Which solution will meet these requirements?

  A. Attach an AWS Glue security configuration to the ETL job.
  B. Configure a data protection policy. Attach the policy to the CloudWatch log group.
  C. Run an Amazon Macie sensitive data discovery job.
  D. Call AWS Glue sensitive data detection APIs in the ETL job.
  B. Configure a data protection policy. Attach the policy to the CloudWatch log group.

• Question 358:

  A data engineer maintains a materialized view that is based on an Amazon Redshift database. The view has a column named load_date that stores the date when each row was loaded.

  The data engineer needs to reclaim database storage space by deleting all the rows from the materialized view.

  Which command will reclaim the MOST database storage space?

  A. DELETE FROM materialized_view_name where 1=1
  B. TRUNCATE materialized_view_name
  C. VACUUM table_name where load_date<=current_date materializedview
  D. DELETE FROM materialized_view_name where load_date<=current_date
  B. TRUNCATE materialized_view_name

• Question 359:

  A data engineer is building a new data pipeline that stores metadata in an Amazon DynamoDB table. The data engineer must ensure that all items that are older than a specified age are removed from the DynamoDB table daily.

  Which solution will meet this requirement with the LEAST configuration effort?

  A. Enable DynamoDB TTL on the DynamoDB table. Adjust the application source code to set the TTL attribute appropriately.
  B. Create an Amazon EventBridge rule that uses a daily cron expression to trigger an AWS Lambda function to delete items that are older than the specified age.
  C. Add a lifecycle configuration to the DynamoDB table that deletes items that are older than the specified age.
  D. Create a DynamoDB stream that has an AWS Lambda function that reacts to data modifications. Configure the Lambda function to delete items that are older than the specified age.
  A. Enable DynamoDB TTL on the DynamoDB table. Adjust the application source code to set the TTL attribute appropriately.

  ---

  Explanation

  DynamoDB Time to Live (TTL) is a fully managed feature that automatically expires and deletes items based on a timestamp attribute you define. By enabling TTL and setting the TTL attribute in the application code, items older than the specified age are removed without the need for custom scheduling or Lambda functions, offering the least configuration effort.

• Question 360:

  An airline company is collecting metrics about flight activities for analytics. The company is conducting a proof of concept (POC) test to show how analytics can provide insights that the company can use to increase on-time departures.

  The POC test uses objects in Amazon S3 that contain the metrics in .csv format. The POC test uses Amazon Athena to query the data. The data is partitioned in the S3 bucket by date. As the amount of data increases, the company wants to optimize the storage solution to improve query performance.

  Which combination of solutions will meet these requirements? (Choose two.)

  A. Add a randomized string to the beginning of the keys in Amazon S3 to get more throughput across partitions.
  B. Use an S3 bucket that is in the same account that uses Athena to query the data.
  C. Use an S3 bucket that is in the same AWS Region where the company runs Athena queries.
  D. Preprocess the .csv data to JSON format by fetching only the document keys that the query requires.
  E. Preprocess the .csv data to Apache Parquet format by fetching only the data blocks that are needed for predicates.
  C. Use an S3 bucket that is in the same AWS Region where the company runs Athena queries.
  E. Preprocess the .csv data to Apache Parquet format by fetching only the data blocks that are needed for predicates.

  ---

  Explanation

  Using an S3 bucket that is in the same AWS Region where the company runs Athena queries can improve query performance by reducing data transfer latency and costs. Preprocessing the .csv data to Apache Parquet format can also improve query performance by enabling columnar storage, compression, and partitioning, which can reduce the amount of data scanned and fetched by the query. These solutions can optimize the storage solution for the POC test without requiring much effort or changes to the existing data pipeline. The other solutions are not optimal or relevant for this requirement. Adding a randomized string to the beginning of the keys in Amazon S3 can improve the throughput across partitions, but it can also make the data harder to query and manage. Using an S3 bucket that is in the same account that uses Athena to query the data does not have any significant impact on query performance, as long as the proper permissions are granted. Preprocessing the .csv data to JSON format does not offer any benefits over the .csv format, as both are row-based and verbose formats that require more data scanning and fetching than columnar formats like Parquet.