Hortonworks Hortonworks Certifications HDPCD Questions & Answers

• Question 71:

  Which best describes what the map method accepts and emits?

  A. It accepts a single key-value pair as input and emits a single key and list of corresponding values as output.

  B. It accepts a single key-value pairs as input and can emit only one key-value pair as output.

  C. It accepts a list key-value pairs as input and can emit only one key-value pair as output.

  D. It accepts a single key-value pairs as input and can emit any number of key-value pair as output, including zero.

  Correct Answer: D

  Explanation: public class Mapper extends Object Maps input key/value pairs to a set of intermediate key/value pairs.

  Maps are the individual tasks which transform input records into a intermediate records. The transformed intermediate records need not be of the same type as the input records. A given input pair may map to zero or many output pairs.

  Reference: org.apache.hadoop.mapreduce

  Class Mapper

• Question 72:

  You wrote a map function that throws a runtime exception when it encounters a control character in input data. The input supplied to your mapper contains twelve such characters totals, spread across five file splits. The first four file splits each have two control characters and the last split has four control characters.

  Indentify the number of failed task attempts you can expect when you run the job with mapred.max.map.attempts set to 4:

  A. You will have forty-eight failed task attempts

  B. You will have seventeen failed task attempts

  C. You will have five failed task attempts

  D. You will have twelve failed task attempts

  E. You will have twenty failed task attempts

  Correct Answer: E

  Explanation: There will be four failed task attempts for each of the five file splits.

  Note:

  

• Question 73:

  What are the TWO main components of the YARN ResourceManager process? Choose 2 answers

  A. Job Tracker

  B. Task Tracker

  C. Scheduler

  D. Applications Manager

  Correct Answer: CD

• Question 74:

  Consider the following two relations, A and B.

  

  Which Pig statement combines A by its first field and B by its second field?

  A. C = DOIN B BY a1, A by b2;

  B. C = JOIN A by al, B by b2;

  C. C = JOIN A a1, B b2;

  D. C = JOIN A SO, B $1;

  Correct Answer: B

• Question 75:

  Which process describes the lifecycle of a Mapper?

  A. The JobTracker calls the TaskTracker's configure () method, then its map () method and finally its close () method.

  B. The TaskTracker spawns a new Mapper to process all records in a single input split.

  C. The TaskTracker spawns a new Mapper to process each key-value pair.

  D. The JobTracker spawns a new Mapper to process all records in a single file.

  Correct Answer: B

  Explanation: For each map instance that runs, the TaskTracker creates a new instance of your mapper.

  Note:

  *

  The Mapper is responsible for processing Key/Value pairs obtained from the InputFormat. The mapper

  may perform a number of Extraction and Transformation functions on the Key/Value pair before ultimately

  outputting none, one or many Key/Value pairs of the same, or different Key/Value type.

  *

  With the new Hadoop API, mappers extend the org.apache.hadoop.mapreduce.Mapper class. This class

  defines an 'Identity' map function by default - every input Key/Value pair obtained from the InputFormat is

  written out.

  Examining the run() method, we can see the lifecycle of the mapper:

  /**

  *

  Expert users can override this method for more complete control over the

  *

  execution of the Mapper.

  *

  @param context

  *

  @throws IOException

  */

  public void run(Context context) throws IOException, InterruptedException { setup(context);

  while (context.nextKeyValue()) {

  map(context.getCurrentKey(), context.getCurrentValue(), context); }

  cleanup(context);

  }

  setup(Context) - Perform any setup for the mapper. The default implementation is a no-op method. map(Key, Value, Context) - Perform a map operation in the given Key / Value pair. The default implementation calls Context.write(Key, Value) cleanup(Context) - Perform any cleanup for the mapper. The default implementation is a no-op method.

  Reference: Hadoop/MapReduce/Mapper

• Question 76:

  Which one of the following classes would a Pig command use to store data in a table defined in HCatalog?

  A. org.apache.hcatalog.pig.HCatOutputFormat

  B. org.apache.hcatalog.pig.HCatStorer

  C. No special class is needed for a Pig script to store data in an HCatalog table

  D. Pig scripts cannot use an HCatalog table

  Correct Answer: B

• Question 77:

  A client application creates an HDFS file named foo.txt with a replication factor of 3. Identify which best describes the file access rules in HDFS if the file has a single block that is stored on data nodes A, B and C?

  A. The file will be marked as corrupted if data node B fails during the creation of the file.

  B. Each data node locks the local file to prohibit concurrent readers and writers of the file.

  C. Each data node stores a copy of the file in the local file system with the same name as the HDFS file.

  D. The file can be accessed if at least one of the data nodes storing the file is available.

  Correct Answer: D

  Explanation: HDFS keeps three copies of a block on three different datanodes to protect against true data corruption. HDFS also tries to distribute these three replicas on more than one rack to protect against data availability issues. The fact that HDFS actively monitors any failed datanode(s) and upon failure detection immediately schedules re-replication of blocks (if needed) implies that three copies of data on three different nodes is sufficient to avoid corrupted files. Note: HDFS is designed to reliably store very large files across machines in a large cluster. It stores each file as a sequence of blocks; all blocks in a file except the last block are the same size. The blocks of a file are replicated for fault tolerance. The block size and replication factor are configurable per file. An application can specify the number of replicas of a file. The replication factor can be specified at file creation time and can be changed later. Files in HDFS are write-once and have strictly one writer at any time. The NameNode makes all decisions regarding replication of blocks. HDFS uses rack-aware replica placement policy. In default configuration there are total 3 copies of a datablock on HDFS, 2 copies are stored on datanodes on same rack and 3rd copy on a different rack. Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers , How the HDFS Blocks are replicated?

• Question 78:

  Which YARN component is responsible for monitoring the success or failure of a Container?

  A. ResourceManager

  B. ApplicationMaster

  C. NodeManager

  D. JobTracker

  Correct Answer: A

• Question 79:

  You want to count the number of occurrences for each unique word in the supplied input data. You've decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner. Will you be able to reuse your existing Reduces as your combiner in this case and why or why not?

  A. Yes, because the sum operation is both associative and commutative and the input and output types to the reduce method match.

  B. No, because the sum operation in the reducer is incompatible with the operation of a Combiner.

  C. No, because the Reducer and Combiner are separate interfaces.

  D. No, because the Combiner is incompatible with a mapper which doesn't use the same data type for both the key and value.

  E. Yes, because Java is a polymorphic object-oriented language and thus reducer code can be reused as a combiner.

  Correct Answer: A

  Explanation: Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative. The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend on the combiners execution.

  Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?

• Question 80:

  Review the following andapos;dataandapos; file and Pig code.

  

  Which one of the following statements is true?

  A. The Output Of the DUMP D command IS (M,{(M,62.95102),(M,38,95111)})

  B. The output of the dump d command is (M, {(38,95in),(62,95i02)})

  C. The code executes successfully but there is not output because the D relation is empty

  D. The code does not execute successfully because D is not a valid relation

  Correct Answer: A