Google Google Certifications PROFESSIONAL-DATA-ENGINEER Questions & Answers

• Question 171:

  Data Analysts in your company have the Cloud IAM Owner role assigned to them in their projects to allow them to work with multiple GCP products in their projects. Your organization requires that all BigQuery data access logs be retained for 6 months. You need to ensure that only audit personnel in your company can access the data access logs for all projects. What should you do?

  A. Enable data access logs in each Data Analyst's project. Restrict access to Stackdriver Logging via Cloud IAM roles.

  B. Export the data access logs via a project-level export sink to a Cloud Storage bucket in the Data Analysts' projects. Restrict access to the Cloud Storage bucket.

  C. Export the data access logs via a project-level export sink to a Cloud Storage bucket in a newly created projects for audit logs. Restrict access to the project with the exported logs.

  D. Export the data access logs via an aggregated export sink to a Cloud Storage bucket in a newly created project for audit logs. Restrict access to the project that contains the exported logs.

  Correct Answer: D

• Question 172:

  You maintain ETL pipelines. You notice that a streaming pipeline running on Dataflow is taking a long time to process incoming data, which causes output delays. You also noticed that the pipeline graph was automatically optimized by Dataflow and merged into one step. You want to identify where the potential bottleneck is occurring. What should you do?

  A. Insert a Reshuffle operation after each processing step, and monitor the execution details in the Dataflow console.

  B. Log debug information in each ParDo function, and analyze the logs at execution time.

  C. Insert output sinks after each key processing step, and observe the writing throughput of each block.

  D. Verify that the Dataflow service accounts have appropriate permissions to write the processed data to the output sinks

  Correct Answer: A

  A Reshuffle operation is a way to force Dataflow to split the pipeline into multiple stages, which can help isolate the performance of each step and identify bottlenecks. By monitoring the execution details in the Dataflow console, you can see the time, CPU, memory, and disk usage of each stage, as well as the number of elements and bytes processed. This can help you diagnose where the pipeline is slowing down and optimize it accordingly. References:

  1: Reshuffling your data

  2: Monitoring pipeline performance using the Dataflow monitoring interface

  3: Optimizing pipeline performance

• Question 173:

  You used Cloud Dataprep to create a recipe on a sample of data in a BigQuery table. You want to reuse this recipe on a daily upload of data with the same schema, after the load job with variable execution time completes. What should you do?

  A. Create a cron schedule in Cloud Dataprep.

  B. Create an App Engine cron job to schedule the execution of the Cloud Dataprep job.

  C. Export the recipe as a Cloud Dataprep template, and create a job in Cloud Scheduler.

  D. Export the Cloud Dataprep job as a Cloud Dataflow template, and incorporate it into a Cloud Composer job.

  Correct Answer: D

• Question 174:

  The marketing team at your organization provides regular updates of a segment of your customer dataset. The marketing team has given you a CSV with 1 million records that must be updated in BigQuery. When you use the UPDATE statement in BigQuery, you receive a quotaExceeded error. What should you do?

  A. Reduce the number of records updated each day to stay within the BigQuery UPDATE DML statement limit.

  B. Increase the BigQuery UPDATE DML statement limit in the Quota management section of the Google Cloud Platform Console.

  C. Split the source CSV file into smaller CSV files in Cloud Storage to reduce the number of BigQuery UPDATE DML statements per BigQuery job.

  D. Import the new records from the CSV file into a new BigQuery table. Create a BigQuery job that merges the new records with the existing records and writes the results to a new BigQuery table.

  Correct Answer: D

• Question 175:

  A shipping company has live package-tracking data that is sent to an Apache Kafka stream in real time. This is then loaded into BigQuery. Analysts in your company want to query the tracking data in BigQuery to analyze geospatial trends in the lifecycle of a package. The table was originally created with ingest-date partitioning. Over time, the query processing time has increased. You need to implement a change that would improve query performance in BigQuery. What should you do?

  A. Implement clustering in BigQuery on the ingest date column.

  B. Implement clustering in BigQuery on the package-tracking ID column.

  C. Tier older data onto Cloud Storage files, and leverage extended tables.

  D. Re-create the table using data partitioning on the package delivery date.

  Correct Answer: A

• Question 176:

  You have a variety of files in Cloud Storage that your data science team wants to use in their models Currently, users do not have a method to explore, cleanse, and validate the data in Cloud Storage. You are looking for a low code solution that can be used by your data science team to quickly cleanse and explore data within Cloud Storage. What should you do?

  A. Load the data into BigQuery and use SQL to transform the data as necessary Provide the data science team access to staging tables to explore the raw data.

  B. Provide the data science team access to Dataflow to create a pipeline to prepare and validate the raw data and load data into BigQuery for data exploration.

  C. Provide the data science team access to Dataprep to prepare, validate, and explore the data within Cloud Storage.

  D. Create an external table in BigQuery and use SQL to transform the data as necessary Provide the data science team access to the external tables to explore the raw data.

  Correct Answer: C

  Dataprep is a low code, serverless, and fully managed service that allows users to visually explore, cleanse, and validate data in Cloud Storage. It also provides features such as data profiling, data quality, data transformation, and data lineage. Dataprep is integrated with BigQuery, so users can easily export the prepared data to BigQuery for further analysis or modeling. Dataprep is a suitable solution for the data science team to quickly and easily work with the data in Cloud Storage, without having to write code or manage infrastructure. The other options are not as suitable as Dataprep for this use case, because they either require more coding, more infrastructure management, or more data movement. Loading the data into BigQuery, either directly or through Dataflow, would incur additional costs and latency, and may not provide the same level of data exploration and validation as Dataprep. Creating an external table in BigQuery would allow users to query the data in Cloud Storage, but would not provide the same level of data cleansing and transformation as Dataprep. References: Dataprep overview Dataprep features Dataprep and BigQuery integration

• Question 177:

  You have enabled the free integration between Firebase Analytics and Google BigQuery. Firebase now automatically creates a new table daily in BigQuery in the format app_events_YYYYMMDD. You want to query all of the tables for the past 30 days in legacy SQL. What should you do?

  A. Use the TABLE_DATE_RANGE function

  B. Use the WHERE_PARTITIONTIME pseudo column

  C. Use WHERE date BETWEEN YYYY-MM-DD AND YYYY-MM-DD

  D. Use SELECT IF.(date >= YYYY-MM-DD AND date <= YYYY-MM-DD

  Correct Answer: A

  Reference: https://cloud.google.com/blog/products/gcp/using-bigquery-and-firebase-analytics-to-understandyour-mobile-app?hl=am

• Question 178:

  You are designing an Apache Beam pipeline to enrich data from Cloud Pub/Sub with static reference data from BigQuery. The reference data is small enough to fit in memory on a single worker. The pipeline should write enriched results to BigQuery for analysis. Which job type and transforms should this pipeline use?

  A. Batch job, PubSubIO, side-inputs

  B. Streaming job, PubSubIO, JdbcIO, side-outputs

  C. Streaming job, PubSubIO, BigQueryIO, side-inputs

  D. Streaming job, PubSubIO, BigQueryIO, side-outputs

  Correct Answer: C

• Question 179:

  You have developed three data processing jobs. One executes a Cloud Dataflow pipeline that transforms data uploaded to Cloud Storage and writes results to BigQuery. The second ingests data from on-premises servers and uploads it to Cloud Storage. The third is a Cloud Dataflow pipeline that gets information from third-party data providers and uploads the information to Cloud Storage. You need to be able to schedule and monitor the execution of these three workflows and manually execute them when needed. What should you do?

  A. Create a Direct Acyclic Graph in Cloud Composer to schedule and monitor the jobs.

  B. Use Stackdriver Monitoring and set up an alert with a Webhook notification to trigger the jobs.

  C. Develop an App Engine application to schedule and request the status of the jobs using GCP API calls.

  D. Set up cron jobs in a Compute Engine instance to schedule and monitor the pipelines using GCP API calls.

  Correct Answer: D

• Question 180:

  You are planning to migrate your current on-premises Apache Hadoop deployment to the cloud. You need to ensure that the deployment is as fault-tolerant and cost-effective as possible for long-running batch jobs. You want to use a managed service. What should you do?

  A. Deploy a Cloud Dataproc cluster. Use a standard persistent disk and 50% preemptible workers. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://

  B. Deploy a Cloud Dataproc cluster. Use an SSD persistent disk and 50% preemptible workers. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://

  C. Install Hadoop and Spark on a 10-node Compute Engine instance group with standard instances. Install the Cloud Storage connector, and store the data in Cloud Storage. Change references in scripts from hdfs:// to gs://

  D. Install Hadoop and Spark on a 10-node Compute Engine instance group with preemptible instances. Store data in HDFS. Change references in scripts from hdfs:// to gs://

  Correct Answer: A