Hortonworks Hortonworks Certifications HDPCD Questions & Answers

• Question 51:

  Table metadata in Hive is:

  A. Stored as metadata on the NameNode.

  B. Stored along with the data in HDFS.

  C. Stored in the Metastore.

  D. Stored in ZooKeeper.

  Correct Answer: C

  Explanation: By default, hive use an embedded Derby database to store metadata information. The metastore is the "glue" between Hive and HDFS. It tells Hive where your data files live in HDFS, what type of data they contain, what tables they belong to, etc.

  The Metastore is an application that runs on an RDBMS and uses an open source ORM layer called DataNucleus, to convert object representations into a relational schema and vice versa. They chose this approach as opposed to storing this information in hdfs as they need the Metastore to be very low latency. The DataNucleus layer allows them to plugin many different RDBMS technologies.

  Note:

  *

  By default, Hive stores metadata in an embedded Apache Derby database, and other client/server

  databases like MySQL can optionally be used.

  *

  features of Hive include:

  Metadata storage in an RDBMS, significantly reducing the time to perform semantic checks during query

  execution.

  Reference: Store Hive Metadata into RDBMS

• Question 52:

  Assuming the following Hive query executes successfully:

  

  Which one of the following statements describes the result set?

  A. A bigram of the top 80 sentences that contain the substring "you are" in the lines column of the input data A1 table.

  B. An 80-value ngram of sentences that contain the words "you" or "are" in the lines column of the inputdata table.

  C. A trigram of the top 80 sentences that contain "you are" followed by a null space in the lines column of the inputdata table.

  D. A frequency distribution of the top 80 words that follow the subsequence "you are" in the lines column of the inputdata table.

  Correct Answer: D

• Question 53:

  Which two of the following are true about this trivial Pig program' (choose Two)

  

  A. The contents of myfile appear on stdout

  B. Pig assumes the contents of myfile are comma delimited

  C. ABC has a schema associated with it

  D. myfile is read from the user's home directory in HDFS

  Correct Answer: AD

• Question 54:

  You want to populate an associative array in order to perform a map-side join. You've decided to put this information in a text file, place that file into the DistributedCache and read it in your Mapper before any records are processed.

  Indentify which method in the Mapper you should use to implement code for reading the file and populating the associative array?

  A. combine

  B. map

  C. init

  D. configure

  Correct Answer: D

  Reference: org.apache.hadoop.filecache , Class DistributedCache

• Question 55:

  You have the following key-value pairs as output from your Map task:

  (the, 1)

  (fox, 1) (faster, 1)

  (than, 1)

  (the, 1)

  (dog, 1)

  How many keys will be passed to the Reducer's reduce method?

  A. Six

  B. Five

  C. Four

  D. Two

  E. One

  F. Three

  Correct Answer: B

  Explanation: Only one key value pair will be passed from the two (the, 1) key value pairs.

• Question 56:

  Which one of the following statements describes the relationship between the ResourceManager and the ApplicationMaster?

  A. The ApplicationMaster requests resources from the ResourceManager

  B. The ApplicationMaster starts a single instance of the ResourceManager

  C. The ResourceManager monitors and restarts any failed Containers of the ApplicationMaster

  D. The ApplicationMaster starts an instance of the ResourceManager within each Container

  Correct Answer: A

• Question 57:

  You need to create a job that does frequency analysis on input data. You will do this by writing a Mapper that uses TextInputFormat and splits each value (a line of text from an input file) into individual characters. For each one of these characters, you will emit the character as a key and an InputWritable as the value. As this will produce proportionally more intermediate data than input data, which two resources should you expect to be bottlenecks?

  A. Processor and network I/O

  B. Disk I/O and network I/O

  C. Processor and RAM

  D. Processor and disk I/O

  Correct Answer: B

• Question 58:

  Assuming default settings, which best describes the order of data provided to a reducer's reduce method:

  A. The keys given to a reducer aren't in a predictable order, but the values associated with those keys always are.

  B. Both the keys and values passed to a reducer always appear in sorted order.

  C. Neither keys nor values are in any predictable order.

  D. The keys given to a reducer are in sorted order but the values associated with each key are in no predictable order

  Correct Answer: D

  Explanation: Reducer has 3 primary phases:

  1.

  Shuffle

  The Reducer copies the sorted output from each Mapper using HTTP across the network.

  2.

  Sort

  The framework merge sorts Reducer inputs by keys (since different Mappers may have output the same key).

  The shuffle and sort phases occur simultaneously i.e. while outputs are being fetched they are merged.

  SecondarySort

  To achieve a secondary sort on the values returned by the value iterator, the application should extend the key with the secondary key and define a grouping comparator. The keys will be sorted using the entire key, but will be grouped using the grouping comparator to decide which keys and values are sent in the same call to reduce.

  3. Reduce

  In this phase the reduce(Object, Iterable, Context) method is called for each in

  the sorted inputs.

  The output of the reduce task is typically written to a RecordWriter via TaskInputOutputContext.write

  (Object, Object).

  The output of the Reducer is not re-sorted.

  Reference: org.apache.hadoop.mapreduce, Class Reducer

• Question 59:

  Which HDFS command displays the contents of the file x in the user's HDFS home directory?

  A. hadoop fs -Is x

  B. hdfs fs -get x

  C. hadoop fs -cat x

  D. hadoop fs -cp x

  Correct Answer: C

• Question 60:

  For each intermediate key, each reducer task can emit:

  A. As many final key-value pairs as desired. There are no restrictions on the types of those key-value pairs (i.e., they can be heterogeneous).

  B. As many final key-value pairs as desired, but they must have the same type as the intermediate key-value pairs.

  C. As many final key-value pairs as desired, as long as all the keys have the same type and all the values have the same type.

  D. One final key-value pair per value associated with the key; no restrictions on the type.

  E. One final key-value pair per key; no restrictions on the type.

  Correct Answer: C

  Reference: Hadoop Map-Reduce Tutorial; Yahoo! Hadoop Tutorial, Module 4: MapReduce