Hortonworks HDP Certified Developer HDPCD Questions & Answers

• Question 21:

  You want to perform analysis on a large collection of images. You want to store this data in HDFS and process it with MapReduce but you also want to give your data analysts and data scientists the ability to process the data directly from HDFS with an interpreted high- level programming language like Python. Which format should you use to store this data in HDFS?

  A. SequenceFiles

  B. Avro

  C. JSON

  D. HTML

  E. XML

  F. CSV

  Correct Answer: B

  Reference: Hadoop binary files processing introduced by image duplicates finder

• Question 22:

  Which one of the following statements is false about HCatalog?

  A. Provides a shared schema mechanism

  B. Designed to be used by other programs such as Pig, Hive and MapReduce

  C. Stores HDFS data in a database for performing SQL-like ad-hoc queries

  D. Exists as a subproject of Hive

  Correct Answer: C

• Question 23:

  For each input key-value pair, mappers can emit:

  A. As many intermediate key-value pairs as designed. There are no restrictions on the types of those key-value pairs (i.e., they can be heterogeneous).

  B. As many intermediate key-value pairs as designed, but they cannot be of the same type as the input key-value pair.

  C. One intermediate key-value pair, of a different type.

  D. One intermediate key-value pair, but of the same type.

  E. As many intermediate key-value pairs as designed, as long as all the keys have the same types and all the values have the same type.

  Correct Answer: E

  Explanation: Mapper maps input key/value pairs to a set of intermediate key/value pairs.

  Maps are the individual tasks that transform input records into intermediate records. The transformed intermediate records do not need to be of the same type as the input records. A given input pair may map to zero or many output pairs.

  Reference: Hadoop Map-Reduce Tutorial

• Question 24:

  Analyze each scenario below and indentify which best describes the behavior of the default partitioner?

  A. The default partitioner assigns key-values pairs to reduces based on an internal random number generator.

  B. The default partitioner implements a round-robin strategy, shuffling the key-value pairs to each reducer in turn. This ensures an event partition of the key space.

  C. The default partitioner computes the hash of the key. Hash values between specific ranges are associated with different buckets, and each bucket is assigned to a specific reducer.

  D. The default partitioner computes the hash of the key and divides that valule modulo the number of reducers. The result determines the reducer assigned to process the key-value pair.

  E. The default partitioner computes the hash of the value and takes the mod of that value with the number of reducers. The result determines the reducer assigned to process the key-value pair.

  Correct Answer: D

  Explanation: The default partitioner computes a hash value for the key and assigns the partition based on this result.

  The default Partitioner implementation is called HashPartitioner. It uses the hashCode() method of the key objects modulo the number of partitions total to determine which partition to send a given (key, value) pair to.

  In Hadoop, the default partitioner is HashPartitioner, which hashes a record's key to determine which partition (and thus which reducer) the record belongs in.The number of partition is then equal to the number of reduce tasks for the job.

  Reference: Getting Started With (Customized) Partitioning

• Question 25:

  In a large MapReduce job with m mappers and n reducers, how many distinct copy operations will there be in the sort/shuffle phase?

  A. mXn (i.e., m multiplied by n)

  B. n

  C. m

  D. m+n (i.e., m plus n) E. mn (i.e., m to the power of n)

  Correct Answer: A

  Explanation: A MapReduce job with m mappers and r reducers involves up to m * r distinct copy operations, since each mapper may have intermediate output going to every reducer.

• Question 26:

  Which project gives you a distributed, Scalable, data store that allows you random, realtime read/write access to hundreds of terabytes of data?

  A. HBase

  B. Hue

  C. Pig

  D. Hive

  E. Oozie

  F. Flume

  G. Sqoop

  Correct Answer: A

  Explanation: Use Apache HBase when you need random, realtime read/write access to your Big Data.

  Note: This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop

  clusters of commodity hardware. Apache HBase is an open-source, distributed, versioned, column-

  oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data by

  Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System,

  Apache HBase provides Bigtable-like capabilities on top of Hadoop and HDFS.

  Features

  Linear and modular scalability.

  Strictly consistent reads and writes.

  Automatic and configurable sharding of tables

  Automatic failover support between RegionServers.

  Convenient base classes for backing Hadoop MapReduce jobs with Apache HBase tables.

  Easy to use Java API for client access.

  Block cache and Bloom Filters for real-time queries.

  Query predicate push down via server side Filters

  Thrift gateway and a REST-ful Web service that supports XML, Protobuf, and binary data encoding options

  Extensible jruby-based (JIRB) shell

  Support for exporting metrics via the Hadoop metrics subsystem to files or Ganglia; or via JMX

  Reference: http://hbase.apache.org/ (when would I use HBase? First sentence)

• Question 27:

  Which one of the following Hive commands uses an HCatalog table named x?

  A. SELECT * FROM x;

  B. SELECT x.-FROM org.apache.hcatalog.hive.HCatLoader('x');

  C. SELECT * FROM org.apache.hcatalog.hive.HCatLoader('x');

  D. Hive commands cannot reference an HCatalog table

  Correct Answer: C

• Question 28:

  What is a SequenceFile?

  A. A SequenceFile contains a binary encoding of an arbitrary number of homogeneous writable objects.

  B. A SequenceFile contains a binary encoding of an arbitrary number of heterogeneous writable objects.

  C. A SequenceFile contains a binary encoding of an arbitrary number of WritableComparable objects, in sorted order.

  D. A SequenceFile contains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be same type.

  Correct Answer: D

  Explanation: SequenceFile is a flat file consisting of binary key/value pairs.

  There are 3 different SequenceFile formats:

  Uncompressed key/value records.

  Record compressed key/value records - only 'values' are compressed here. Block compressed key/value

  records - both keys and values are collected in 'blocks' separately and compressed. The size of the 'block'

  is configurable.

  Reference: http://wiki.apache.org/hadoop/SequenceFile

• Question 29:

  You have written a Mapper which invokes the following five calls to the OutputColletor.collect method:

  output.collect (new Text ("Apple"), new Text ("Red") ) ;

  output.collect (new Text ("Banana"), new Text ("Yellow") ) ;

  output.collect (new Text ("Apple"), new Text ("Yellow") ) ;

  output.collect (new Text ("Cherry"), new Text ("Red") ) ;

  output.collect (new Text ("Apple"), new Text ("Green") ) ;

  How many times will the Reducer's reduce method be invoked?

  A. 6

  B. 3

  C. 1

  D. 0

  E. 5

  Correct Answer: B

  Explanation: reduce() gets called once for each [key, (list of values)] pair. To explain, let's say you called:

  out.collect(new Text("Car"),new Text("Subaru");

  out.collect(new Text("Car"),new Text("Honda");

  out.collect(new Text("Car"),new Text("Ford");

  out.collect(new Text("Truck"),new Text("Dodge");

  out.collect(new Text("Truck"),new Text("Chevy");

  Then reduce() would be called twice with the pairs reduce(Car, )

  reduce(Truck, )

  Reference: Mapper output.collect()?

• Question 30:

  Which one of the following statements describes the relationship between the NodeManager and the ApplicationMaster?

  A. The ApplicationMaster starts the NodeManager in a Container

  B. The NodeManager requests resources from the ApplicationMaster

  C. The ApplicationMaster starts the NodeManager outside of a Container

  D. The NodeManager creates an instance of the ApplicationMaster

  Correct Answer: D