Hortonworks HDP Certified Developer HDPCD Questions & Answers

• Question 11:

  In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?

  A. Increase the parameter that controls minimum split size in the job configuration.

  B. Write a custom MapRunner that iterates over all key-value pairs in the entire file.

  C. Set the number of mappers equal to the number of input files you want to process.

  D. Write a custom FileInputFormat and override the method isSplitable to always return false.

  Correct Answer: D

  Explanation: FileInputFormat is the base class for all file-based InputFormats. This provides a generic implementation of getSplits(JobContext). Subclasses of FileInputFormat can also override the isSplitable (JobContext, Path) method to ensure input-files are not split-up and are processed as a whole by Mappers.

  Reference: org.apache.hadoop.mapreduce.lib.input, Class FileInputFormat

• Question 12:

  What does Pig provide to the overall Hadoop solution?

  A. Legacy language Integration with MapReduce framework

  B. Simple scripting language for writing MapReduce programs

  C. Database table and storage management services

  D. C++ interface to MapReduce and data warehouse infrastructure

  Correct Answer: B

• Question 13:

  All keys used for intermediate output from mappers must:

  A. Implement a splittable compression algorithm.

  B. Be a subclass of FileInputFormat.

  C. Implement WritableComparable.

  D. Override isSplitable.

  E. Implement a comparator for speedy sorting.

  Correct Answer: C

  Explanation: The MapReduce framework operates exclusively on pairs, that is, the framework views the input to the job as a set of pairs and produces a set of pairs as the output of the job, conceivably of different types.

  The key and value classes have to be serializable by the framework and hence need to implement the Writable interface. Additionally, the key classes have to implement the WritableComparable interface to facilitate sorting by the framework.

  Reference: MapReduce Tutorial

• Question 14:

  A NameNode in Hadoop 2.2 manages ______________.

  A. Two namespaces: an active namespace and a backup namespace

  B. A single namespace

  C. An arbitrary number of namespaces

  D. No namespaces

  Correct Answer: B

• Question 15:

  Examine the following Pig commands: Which one of the following statements is true?

  

  A. The SAMPLE command generates an "unexpected symbol" error

  B. Each MapReduce task will terminate after executing for 0.2 minutes

  C. The reducers will only output the first 20% of the data passed from the mappers

  D. A random sample of approximately 20% of the data will be output

  Correct Answer: D

• Question 16:

  Identify the MapReduce v2 (MRv2 / YARN) daemon responsible for launching application containers and monitoring application resource usage?

  A. ResourceManager

  B. NodeManager

  C. ApplicationMaster

  D. ApplicationMasterService

  E. TaskTracker

  F. JobTracker

  Correct Answer: B

  Reference: Apache Hadoop YARN ?Conceptsand; Applications

• Question 17:

  You need to perform statistical analysis in your MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file. Which is the best way to make this library available to your MapReducer job at runtime?

  A. Have your system administrator copy the JAR to all nodes in the cluster and set its location in the HADOOP_CLASSPATH environment variable before you submit your job.

  B. Have your system administrator place the JAR file on a Web server accessible to all cluster nodes and then set the HTTP_JAR_URL environment variable to its location.

  C. When submitting the job on the command line, specify the 璴ibjars option followed by the JAR file path.

  D. Package your code and the Apache Commands Math library into a zip file named JobJar.zip

  Correct Answer: C

  Explanation: The usage of the jar command is like this,

  Usage: hadoop jar [mainClass] args...

  If you want the commons-math3.jar to be available for all the tasks you can do any one of these

  1. Copy the jar file in $HADOOP_HOME/lib dir

  2.

  or

  Use the generic option -libjars.

• Question 18:

  Which TWO of the following statements are true regarding Hive? Choose 2 answers

  A. Useful for data analysts familiar with SQL who need to do ad-hoc queries

  B. Offers real-time queries and row level updates

  C. Allows you to define a structure for your unstructured Big Data

  D. Is a relational database

  Correct Answer: AC

• Question 19:

  Which best describes how TextInputFormat processes input files and line breaks?

  A. Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReader of the split that contains the beginning of the broken line.

  B. Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReaders of both splits containing the broken line.

  C. The input file is split exactly at the line breaks, so each RecordReader will read a series of complete lines.

  D. Input file splits may cross line breaks. A line that crosses file splits is ignored.

  E. Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReader of the split that contains the end of the broken line.

  Correct Answer: A

  Reference: How Map and Reduce operations are actually carried out

• Question 20:

  What is the term for the process of moving map outputs to the reducers?

  A. Reducing

  B. Combining

  C. Partitioning

  D. Shuffling and sorting

  Correct Answer: D