Hortonworks HCAHD APACHE-HADOOP-DEVELOPER Questions & Answers

• Question 101:

  What does the following WebHDFS command do?

  Curl -1 -L "http://host:port/webhdfs/v1/foo/bar?op=OPEN"

  A. Make a directory /foo/bar

  B. Read a file /foo/bar

  C. List a directory /foo

  D. Delete a directory /foo/bar

  Correct Answer: B

• Question 102:

  Which one of the following statements is FALSE regarding the communication between DataNodes and a federation of NameNodes in Hadoop 2.0?

  A. Each DataNode receives commands from one designated master NameNode.

  B. DataNodes send periodic heartbeats to all the NameNodes.

  C. Each DataNode registers with all the NameNodes.

  D. DataNodes send periodic block reports to all the NameNodes.

  Correct Answer: A

• Question 103:

  What is the disadvantage of using multiple reducers with the default HashPartitioner and distributing your workload across you cluster?

  A. You will not be able to compress the intermediate data.

  B. You will longer be able to take advantage of a Combiner.

  C. By using multiple reducers with the default HashPartitioner, output files may not be in globally sorted order.

  D. There are no concerns with this approach. It is always advisable to use multiple reduces.

  Correct Answer: C

  Explanation: Multiple reducers and total ordering

  If your sort job runs with multiple reducers (either because mapreduce.job.reduces in mapred-site.xml has been set to a number larger than 1, or because you've used the -r option to specify the number of reducers on the command-line), then by default Hadoop will use the HashPartitioner to distribute records across the reducers. Use of the HashPartitioner means that you can't concatenate your output files to create a single sorted output file. To do this you'll need total ordering,

  Reference: Sorting text files with MapReduce

• Question 104:

  Which best describes what the map method accepts and emits?

  A. It accepts a single key-value pair as input and emits a single key and list of corresponding values as output.

  B. It accepts a single key-value pairs as input and can emit only one key-value pair as output.

  C. It accepts a list key-value pairs as input and can emit only one key-value pair as output.

  D. It accepts a single key-value pairs as input and can emit any number of key-value pair as output, including zero.

  Correct Answer: D

  Explanation: public class Mapper extends Object Maps input key/value pairs to a set of intermediate key/value pairs.

  Maps are the individual tasks which transform input records into a intermediate records. The transformed intermediate records need not be of the same type as the input records. A given input pair may map to zero or many output pairs. Reference: org.apache.hadoop.mapreduce

  Class Mapper

• Question 105:

  When can a reduce class also serve as a combiner without affecting the output of a MapReduce program?

  A. When the types of the reduce operation's input key and input value match the types of the reducer's output key and output value and when the reduce operation is both communicative and associative.

  B. When the signature of the reduce method matches the signature of the combine method.

  C. Always. Code can be reused in Java since it is a polymorphic object-oriented programming language.

  D. Always. The point of a combiner is to serve as a mini-reducer directly after the map phase to increase performance.

  E. Never. Combiners and reducers must be implemented separately because they serve different purposes.

  Correct Answer: A

  Explanation: You can use your reducer code as a combiner if the operation performed is commutative and associative.

  Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?

• Question 106:

  Which of the following tool was designed to import data from a relational database into HDFS?

  A. HCatalog B. Sqoop

  C. Flume

  D. Ambari

  Correct Answer: B

• Question 107:

  You want to count the number of occurrences for each unique word in the supplied input data. You've decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner. Will you be able to reuse your existing Reduces as your combiner in this case and why or why not?

  A. Yes, because the sum operation is both associative and commutative and the input and output types to the reduce method match.

  B. No, because the sum operation in the reducer is incompatible with the operation of a Combiner.

  C. No, because the Reducer and Combiner are separate interfaces.

  D. No, because the Combiner is incompatible with a mapper which doesn't use the same data type for both the key and value.

  E. Yes, because Java is a polymorphic object-oriented language and thus reducer code can be reused as a combiner.

  Correct Answer: A

  Explanation: Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative. The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times. Therefore your MapReduce jobs should not depend on the combiners execution.

  Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers, What are combiners? When should I use a combiner in my MapReduce Job?

• Question 108:

  You need to perform statistical analysis in your MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file. Which is the best way to make this library available to your MapReducer job at runtime?

  A. Have your system administrator copy the JAR to all nodes in the cluster and set its location in the HADOOP_CLASSPATH environment variable before you submit your job.

  B. Have your system administrator place the JAR file on a Web server accessible to all cluster nodes and then set the HTTP_JAR_URL environment variable to its location.

  C. When submitting the job on the command line, specify the 璴ibjars option followed by the JAR file path.

  D. Package your code and the Apache Commands Math library into a zip file named JobJar.zip

  Correct Answer: C

  Explanation: The usage of the jar command is like this,

  Usage: hadoop jar [mainClass] args...

  If you want the commons-math3.jar to be available for all the tasks you can do any one of these

  1. Copy the jar file in $HADOOP_HOME/lib dir

  2.

  or

  Use the generic option -libjars.