Hortonworks HDP Certified Developer HDPCD Questions & Answers

• Question 91:

  Identify the tool best suited to import a portion of a relational database every day as files into HDFS, and generate Java classes to interact with that imported data?

  A. Oozie

  B. Flume

  C. Pig

  D. Hue

  E. Hive

  F. Sqoop

  G. fuse-dfs

  Correct Answer: F

  Sqoop ("SQL-to-Hadoop") is a straightforward command-line tool with the following capabilities:

  Imports individual tables or entire databases to files in HDFS Generates Java classes to allow you to

  interact with your imported data Provides the ability to import from SQL databases straight into your Hive

  data warehouse

  Note:

  Data Movement Between Hadoop and Relational Databases Data can be moved between Hadoop and a

  relational database as a bulk data transfer, or relational tables can be accessed from within a MapReduce

  map function.

  Note:

  * Cloudera's Distribution for Hadoop provides a bulk data transfer tool (i.e., Sqoop) that imports individual tables or entire databases into HDFS files. The tool also generates Java classes that support interaction with the imported data. Sqoop supports all relational databases over JDBC, and Quest Software provides a connector (i.e., OraOop) that has been optimized for access to data residing in Oracle databases.

  Reference: http://log.medcl.net/item/2011/08/hadoop-and-mapreduce-big-data-analytics- gartner/ (Data Movement between hadoop and relational databases, second paragraph)

• Question 92:

  Review the following data and Pig code.

  M,38,95111

  F,29,95060

  F,45,95192

  M,62,95102

  F,56,95102

  A = LOAD andapos;dataandapos; USING PigStorage(andapos;.andapos;) as (gender:Chararray, age:int,

  zlp:chararray);

  B = FOREACH A GENERATE age;

  Which one of the following commands would save the results of B to a folder in hdfs named myoutput?

  A. STORE A INTO andapos;myoutputandapos; USING PigStorage(andapos;,andapos;);

  B. DUMP B using PigStorage(andapos;myoutputandapos;);

  C. STORE B INTO andapos;myoutputandapos;;

  D. DUMP B INTO andapos;myoutputandapos;;

  Correct Answer: C

• Question 93:

  In the reducer, the MapReduce API provides you with an iterator over Writable values. What does calling the next () method return?

  A. It returns a reference to a different Writable object time.

  B. It returns a reference to a Writable object from an object pool.

  C. It returns a reference to the same Writable object each time, but populated with different data.

  D. It returns a reference to a Writable object. The API leaves unspecified whether this is a reused object or a new object.

  E. It returns a reference to the same Writable object if the next value is the same as the previous value, or a new Writable object otherwise.

  Correct Answer: C

  Explanation: Calling Iterator.next() will always return the SAME EXACT instance of IntWritable, with the contents of that instance replaced with the next value.

  Reference: manupulating iterator in mapreduce

• Question 94:

  In a MapReduce job with 500 map tasks, how many map task attempts will there be?

  A. It depends on the number of reduces in the job.

  B. Between 500 and 1000.

  C. At most 500.

  D. At least 500.

  E. Exactly 500.

  Correct Answer: D

  From Cloudera Training Course: Task attempt is a particular instance of an attempt to execute a task ?There will be at least as many task attempts as there are tasks ?If a task attempt fails, another will be started by the JobTracker ?Speculative execution can also result in more task attempts than completed tasks

• Question 95:

  Which one of the following statements describes a Pig bag. tuple, and map, respectively?

  A. Unordered collection of maps, ordered collection of tuples, ordered set of key/value pairs

  B. Unordered collection of tuples, ordered set of fields, set of key value pairs

  C. Ordered set of fields, ordered collection of tuples, ordered collection of maps

  D. Ordered collection of maps, ordered collection of bags, and unordered set of key/value pairs

  Correct Answer: B

• Question 96:

  You write MapReduce job to process 100 files in HDFS. Your MapReduce algorithm uses TextInputFormat: the mapper applies a regular expression over input values and emits key- values pairs with the key consisting of the matching text, and the value containing the filename and byte offset. Determine the difference between setting the number of reduces to one and settings the number of reducers to zero.

  A. There is no difference in output between the two settings.

  B. With zero reducers, no reducer runs and the job throws an exception. With one reducer, instances of matching patterns are stored in a single file on HDFS.

  C. With zero reducers, all instances of matching patterns are gathered together in one file on HDFS. With one reducer, instances of matching patterns are stored in multiple files on HDFS.

  D. With zero reducers, instances of matching patterns are stored in multiple files on HDFS. With one reducer, all instances of matching patterns are gathered together in one file on HDFS.

  Correct Answer: D

  Explanation: * It is legal to set the number of reduce-tasks to zero if no reduction is desired.

  In this case the outputs of the map-tasks go directly to the FileSystem, into the output path set by setOutputPath(Path). The framework does not sort the map-outputs before writing them out to the FileSystem.

  * Often, you may want to process input data using a map function only. To do this, simply set mapreduce.job.reduces to zero. The MapReduce framework will not create any reducer tasks. Rather, the outputs of the mapper tasks will be the final output of the job.

  Note:

  Reduce

  In this phase the reduce(WritableComparable, Iterator, OutputCollector, Reporter) method is called for

  each pair in the grouped inputs.

  The output of the reduce task is typically written to the FileSystem via OutputCollector.collect

  (WritableComparable, Writable).

  Applications can use the Reporter to report progress, set application-level status messages and update

  Counters, or just indicate that they are alive.

  The output of the Reducer is not sorted.

• Question 97:

  Can you use MapReduce to perform a relational join on two large tables sharing a key? Assume that the two tables are formatted as comma-separated files in HDFS.

  A. Yes.

  B. Yes, but only if one of the tables fits into memory

  C. Yes, so long as both tables fit into memory.

  D. No, MapReduce cannot perform relational operations.

  E. No, but it can be done with either Pig or Hive.

  Correct Answer: A

  Explanation: Note:

  *

  Join Algorithms in MapReduce A) Reduce-side join B) Map-side join C) In-memory join / Striped Striped variant variant / Memcached variant

  *

  Which join to use? / In-memory join > map-side join > reduce-side join / Limitations of each? In-memory join: memory Map-side join: sort order and partitioning Reduce-side join: general purpose

• Question 98:

  Which HDFS command copies an HDFS file named foo to the local filesystem as localFoo?

  A. hadoop fs -get foo LocalFoo

  B. hadoop -cp foo LocalFoo

  C. hadoop fs -Is foo

  D. hadoop fs -put foo LocalFoo

  Correct Answer: A

• Question 99:

  In Hadoop 2.2, which TWO of the following processes work together to provide automatic failover of the NameNode? Choose 2 answers

  A. ZKFailoverController

  B. ZooKeeper

  C. QuorumManager

  D. JournalNode

  Correct Answer: AD

• Question 100:

  Review the following data and Pig code: What command to define B would produce the output (M,62,95l02) when invoking the DUMP operator on B?

  

  A. B = FILTER A BY (zip = = '95102' AND gender = = M");

  B. B= FOREACH A BY (gender = = 'M' AND zip = = '95102');

  C. B = JOIN A BY (gender = = 'M' AND zip = = '95102');

  D. B= GROUP A BY (zip = = '95102' AND gender = = 'M');

  Correct Answer: A