Hortonworks HDP Certified Developer HDPCD Questions & Answers

• Question 41:

  You have user profile records in your OLPT database, that you want to join with web logs you have already ingested into the Hadoop file system. How will you obtain these user records?

  A. HDFS command

  B. Pig LOAD command

  C. Sqoop import

  D. Hive LOAD DATA command

  E. Ingest with Flume agents

  F. Ingest with Hadoop Streaming

  Correct Answer: C

  Reference: Hadoop and Pig for Large-Scale Web Log Analysis

• Question 42:

  You use the hadoop fs –put command to write a 300 MB file using and HDFS block size of 64 MB. Just after this command has finished writing 200 MB of this file, what would another user see when trying to access this life?

  A. They would see Hadoop throw an ConcurrentFileAccessException when they try to access this file.

  B. They would see the current state of the file, up to the last bit written by the command.

  C. They would see the current of the file through the last completed block.

  D. They would see no content until the whole file written and closed.

  Correct Answer: C

• Question 43:

  What is the disadvantage of using multiple reducers with the default HashPartitioner and distributing your workload across you cluster?

  A. You will not be able to compress the intermediate data.

  B. You will longer be able to take advantage of a Combiner.

  C. By using multiple reducers with the default HashPartitioner, output files may not be in globally sorted order.

  D. There are no concerns with this approach. It is always advisable to use multiple reduces.

  Correct Answer: C

  Explanation: Multiple reducers and total ordering If your sort job runs with multiple reducers (either because mapreduce.job.reduces in mapred-site.xml has been set to a number larger than 1, or because you've used the -r option to specify the number of reducers on the command-line), then by default Hadoop will use the HashPartitioner to distribute records across the reducers. Use of the HashPartitioner means that you can't concatenate your output files to create a single sorted output file. To do this you'll need total ordering,

  Reference: Sorting text files with MapReduce

• Question 44:

  You need to move a file titled "weblogs" into HDFS. When you try to copy the file, you can't. You know you have ample space on your DataNodes. Which action should you take to relieve this situation and store more files in HDFS?

  A. Increase the block size on all current files in HDFS.

  B. Increase the block size on your remaining files.

  C. Decrease the block size on your remaining files.

  D. Increase the amount of memory for the NameNode.

  E. Increase the number of disks (or size) for the NameNode.

  F. Decrease the block size on all current files in HDFS.

  Correct Answer: C

• Question 45:

  What does the following WebHDFS command do?

  Curl -1 -L "http://host:port/webhdfs/v1/foo/bar?op=OPEN"

  A. Make a directory /foo/bar

  B. Read a file /foo/bar

  C. List a directory /foo

  D. Delete a directory /foo/bar

  Correct Answer: B

• Question 46:

  In a MapReduce job, the reducer receives all values associated with same key. Which statement best describes the ordering of these values?

  A. The values are in sorted order.

  B. The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce job.

  C. The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.

  D. Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

  Correct Answer: B

  Note:

  *

  Input to the Reducer is the sorted output of the mappers.

  *

  The framework calls the application's Reduce function once for each unique key in the sorted order.

  *

  Example:

  For the given sample input the first map emits:

  < Hello, 1>

  < World, 1>

  < Bye, 1>

  < World, 1>

  The second map emits:

  < Hello, 1>

  < Hadoop, 1>

  < Goodbye, 1>

  < Hadoop, 1>

• Question 47:

  Consider the following two relations, A and B.

  

  What is the output of the following Pig commands?

  X = GROUP A BY S1;

  DUMP X;

  

  A. Option A

  B. Option B

  C. Option C

  D. Option D

  Correct Answer: D

• Question 48:

  Your client application submits a MapReduce job to your Hadoop cluster. Identify the Hadoop daemon on which the Hadoop framework will look for an available slot schedule a MapReduce operation.

  A. TaskTracker

  B. NameNode

  C. DataNode

  D. JobTracker

  E. Secondary NameNode

  Correct Answer: D

  Explanation: JobTracker is the daemon service for submitting and tracking MapReduce jobs in Hadoop. There is only One Job Tracker process run on any hadoop cluster. Job Tracker runs on its own JVM process. In a typical production cluster its run on a separate machine. Each slave node is configured with job tracker node location. The JobTracker is single point of failure for the Hadoop MapReduce service. If it goes down, all running jobs are halted. JobTracker in Hadoop performs following actions(from Hadoop Wiki:)

  Client applications submit jobs to the Job tracker. The JobTracker talks to the NameNode to determine the location of the data The JobTracker locates TaskTracker nodes with available slots at or near the data The JobTracker submits the work to the chosen TaskTracker nodes. The TaskTracker nodes are monitored. If they do not submit heartbeat signals often enough, they are deemed to have failed and the work is scheduled on a different TaskTracker. A TaskTracker will notify the JobTracker when a task fails. The JobTracker decides what to do then: it may resubmit the job elsewhere, it may mark that specific record as something to avoid, and it may may even blacklist the TaskTracker as unreliable. When the work is completed, the JobTracker updates its status.

  Client applications can poll the JobTracker for information.

  Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers, What is a JobTracker in Hadoop? How many instances of JobTracker run on a Hadoop Cluster?

• Question 49:

  What does the following command do?

  register andapos;/piggyban):/pig-files.jarandapos;;

  A. Invokes the user-defined functions contained in the jar file

  B. Assigns a name to a user-defined function or streaming command

  C. Transforms Pig user-defined functions into a format that Hive can accept

  D. Specifies the location of the JAR file containing the user-defined functions

  Correct Answer: D

• Question 50:

  Indentify which best defines a SequenceFile?

  A. A SequenceFile contains a binary encoding of an arbitrary number of homogeneous Writable objects

  B. A SequenceFile contains a binary encoding of an arbitrary number of heterogeneous Writable objects

  C. A SequenceFile contains a binary encoding of an arbitrary number of WritableComparable objects, in sorted order.

  D. A SequenceFile contains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be the same type.

  Correct Answer: D

  Explanation: SequenceFile is a flat file consisting of binary key/value pairs.

  There are 3 different SequenceFile formats:

  Uncompressed key/value records.

  Record compressed key/value records - only 'values' are compressed here. Block compressed key/value

  records - both keys and values are collected in 'blocks' separately and compressed. The size of the 'block'

  is configurable.

  Reference: http://wiki.apache.org/hadoop/SequenceFile