Hortonworks HCAHD APACHE-HADOOP-DEVELOPER Questions & Answers

• Question 11:

  To process input key-value pairs, your mapper needs to lead a 512 MB data file in memory. What is the best way to accomplish this?

  A. Serialize the data file, insert in it the JobConf object, and read the data into memory in the configure method of the mapper.

  B. Place the data file in the DistributedCache and read the data into memory in the map method of the mapper.

  C. Place the data file in the DataCache and read the data into memory in the configure method of the mapper.

  D. Place the data file in the DistributedCache and read the data into memory in the configure method of the mapper.

  Correct Answer: C

• Question 12:

  Which one of the following statements is true about a Hive-managed table?

  A. Records can only be added to the table using the Hive INSERT command.

  B. When the table is dropped, the underlying folder in HDFS is deleted.

  C. Hive dynamically defines the schema of the table based on the FROM clause of a SELECT query.

  D. Hive dynamically defines the schema of the table based on the format of the underlying data.

  Correct Answer: B

• Question 13:

  You wrote a map function that throws a runtime exception when it encounters a control character in input data. The input supplied to your mapper contains twelve such characters totals, spread across five file splits. The first four file splits each have two control characters and the last split has four control characters.

  Indentify the number of failed task attempts you can expect when you run the job with mapred.max.map.attempts set to 4:

  A. You will have forty-eight failed task attempts

  B. You will have seventeen failed task attempts

  C. You will have five failed task attempts

  D. You will have twelve failed task attempts

  E. You will have twenty failed task attempts

  Correct Answer: E

  Explanation: There will be four failed task attempts for each of the five file splits.

  Note:

  

• Question 14:

  How are keys and values presented and passed to the reducers during a standard sort and shuffle phase of MapReduce?

  A. Keys are presented to reducer in sorted order; values for a given key are not sorted.

  B. Keys are presented to reducer in sorted order; values for a given key are sorted in ascending order.

  C. Keys are presented to a reducer in random order; values for a given key are not sorted.

  D. Keys are presented to a reducer in random order; values for a given key are sorted in ascending order.

  Correct Answer: A

  Explanation: Reducer has 3 primary phases:

  1.

  Shuffle

  The Reducer copies the sorted output from each Mapper using HTTP across the network.

  2.

  Sort

  The framework merge sorts Reducer inputs by keys (since different Mappers may have output the same key).

  The shuffle and sort phases occur simultaneously i.e. while outputs are being fetched they are merged.

  SecondarySort

  To achieve a secondary sort on the values returned by the value iterator, the application should extend the key with the secondary key and define a grouping comparator. The keys will be sorted using the entire key, but will be grouped using the grouping comparator to decide which keys and values are sent in the same call to reduce.

  3. Reduce In this phase the reduce(Object, Iterable, Context) method is called for each in the sorted inputs.

  The output of the reduce task is typically written to a RecordWriter via TaskInputOutputContext.write (Object, Object).

  The output of the Reducer is not re-sorted.

  Reference: org.apache.hadoop.mapreduce, Class Reducer

• Question 15:

  Which one of the following is NOT a valid Oozie action?

  A. mapreduce

  B. pig

  C. hive

  D. mrunit

  Correct Answer: D

• Question 16:

  A NameNode in Hadoop 2.0 manages ______________.

  A. Two namespaces: an active namespace and a backup namespace

  B. A single namespace

  C. An arbitrary number of namespaces

  D. No namespaces

  Correct Answer: B

• Question 17:

  What is a SequenceFile?

  A. A SequenceFile contains a binary encoding of an arbitrary number of homogeneous writable objects.

  B. A SequenceFile contains a binary encoding of an arbitrary number of heterogeneous writable objects.

  C. A SequenceFile contains a binary encoding of an arbitrary number of WritableComparable objects, in sorted order.

  D. A SequenceFile contains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be same type.

  Correct Answer: D

  Explanation: SequenceFile is a flat file consisting of binary key/value pairs.

  There are 3 different SequenceFile formats:

  Uncompressed key/value records.

  Record compressed key/value records - only 'values' are compressed here. Block compressed key/value

  records - both keys and values are collected in 'blocks' separately and compressed. The size of the 'block'

  is configurable.

  Reference: http://wiki.apache.org/hadoop/SequenceFile

• Question 18:

  You've written a MapReduce job that will process 500 million input records and generated 500 million key-value pairs. The data is not uniformly distributed. Your MapReduce job will create a significant amount of intermediate data that it needs to transfer between mappers and reduces which is a potential bottleneck. A custom implementation of which interface is most likely to reduce the amount of intermediate data transferred across the network?

  A. Partitioner

  B. OutputFormat

  C. WritableComparable

  D. Writable

  E. InputFormat

  F. Combiner

  Correct Answer: F

  Explanation: Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative.

  Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?

• Question 19:

  For each intermediate key, each reducer task can emit:

  A. As many final key-value pairs as desired. There are no restrictions on the types of those key-value pairs (i.e., they can be heterogeneous).

  B. As many final key-value pairs as desired, but they must have the same type as the intermediate key-value pairs.

  C. As many final key-value pairs as desired, as long as all the keys have the same type and all the values have the same type.

  D. One final key-value pair per value associated with the key; no restrictions on the type.

  E. One final key-value pair per key; no restrictions on the type.

  Correct Answer: C

  Reference: Hadoop Map-Reduce Tutorial; Yahoo! Hadoop Tutorial, Module 4: MapReduce

• Question 20:

  You want to populate an associative array in order to perform a map-side join. You've decided to put this information in a text file, place that file into the DistributedCache and read it in your Mapper before any records are processed.

  Indentify which method in the Mapper you should use to implement code for reading the file and populating the associative array?

  A. combine

  B. map

  C. init

  D. configure

  Correct Answer: D

  Reference: org.apache.hadoop.filecache , Class DistributedCache