Hortonworks HCAHD APACHE-HADOOP-DEVELOPER Questions & Answers

• Question 31:

  Examine the following Pig commands:

  

  Which one of the following statements is true?

  A. The SAMPLE command generates an "unexpected symbol" error

  B. Each MapReduce task will terminate after executing for 0.2 minutes

  C. The reducers will only output the first 20% of the data passed from the mappers

  D. A random sample of approximately 20% of the data will be output

  Correct Answer: D

• Question 32:

  A client application creates an HDFS file named foo.txt with a replication factor of 3. Identify which best describes the file access rules in HDFS if the file has a single block that is stored on data nodes A, B and C?

  A. The file will be marked as corrupted if data node B fails during the creation of the file.

  B. Each data node locks the local file to prohibit concurrent readers and writers of the file.

  C. Each data node stores a copy of the file in the local file system with the same name as the HDFS file.

  D. The file can be accessed if at least one of the data nodes storing the file is available.

  Correct Answer: D

  Explanation: HDFS keeps three copies of a block on three different datanodes to protect against true data corruption. HDFS also tries to distribute these three replicas on more than one rack to protect against data availability issues. The fact that HDFS actively monitors any failed datanode(s) and upon failure detection immediately schedules re-replication of blocks (if needed) implies that three copies of data on three different nodes is sufficient to avoid corrupted files. Note: HDFS is designed to reliably store very large files across machines in a large cluster. It stores each file as a sequence of blocks; all blocks in a file except the last block are the same size. The blocks of a file are replicated for fault tolerance. The block size and replication factor are configurable per file. An application can specify the number of replicas of a file. The replication factor can be specified at file creation time and can be changed later. Files in HDFS are write-once and have strictly one writer at any time. The NameNode makes all decisions regarding replication of blocks. HDFS uses rack-aware replica placement policy. In default configuration there are total 3 copies of a datablock on HDFS, 2 copies are stored on datanodes on same rack and 3rd copy on a different rack. Reference: 24 Interview Questions and Answers for Hadoop MapReduce developers , How the HDFS Blocks are replicated?

• Question 33:

  In a MapReduce job, the reducer receives all values associated with same key. Which statement best describes the ordering of these values?

  A. The values are in sorted order.

  B. The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce job.

  C. The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.

  D. Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

  Correct Answer: B

  Note:

  *

  Input to the Reducer is the sorted output of the mappers.

  *

  The framework calls the application's Reduce function once for each unique key in the sorted order.

  *

  Example:

  For the given sample input the first map emits:

  < Hello, 1>

  < World, 1>

  < Bye, 1>

  < World, 1>

  The second map emits:

  < Hello, 1>

  < Hadoop, 1>

  < Goodbye, 1>

  < Hadoop, 1>

• Question 34:

  For each input key-value pair, mappers can emit:

  A. As many intermediate key-value pairs as designed. There are no restrictions on the types of those key-value pairs (i.e., they can be heterogeneous).

  B. As many intermediate key-value pairs as designed, but they cannot be of the same type as the input key-value pair.

  C. One intermediate key-value pair, of a different type.

  D. One intermediate key-value pair, but of the same type.

  E. As many intermediate key-value pairs as designed, as long as all the keys have the same types and all the values have the same type.

  Correct Answer: E

  Explanation: Mapper maps input key/value pairs to a set of intermediate key/value pairs.

  Maps are the individual tasks that transform input records into intermediate records. The transformed intermediate records do not need to be of the same type as the input records. A given input pair may map to zero or many output pairs.

  Reference: Hadoop Map-Reduce Tutorial

• Question 35:

  Which HDFS command copies an HDFS file named foo to the local filesystem as localFoo?

  A. hadoop fs -get foo LocalFoo

  B. hadoop -cp foo LocalFoo

  C. hadoop fs -Is foo

  D. hadoop fs -put foo LocalFoo

  Correct Answer: A

• Question 36:

  Given the following Hive command:

  

  Which one of the following statements is true?

  A. The files in the mydata folder are copied to a subfolder of /apps/hlve/warehouse

  B. The files in the mydata folder are moved to a subfolder of /apps/hive/wa re house

  C. The files in the mydata folder are copied into Hive's underlying relational database

  D. The files in the mydata folder do not move from their current location In HDFS

  Correct Answer: D

• Question 37:

  You use the hadoop fs –put command to write a 300 MB file using and HDFS block size of 64 MB. Just after this command has finished writing 200 MB of this file, what would another user see when trying to access this life?

  A. They would see Hadoop throw an ConcurrentFileAccessException when they try to access this file.

  B. They would see the current state of the file, up to the last bit written by the command.

  C. They would see the current of the file through the last completed block.

  D. They would see no content until the whole file written and closed.

  Correct Answer: C

• Question 38:

  Review the following data and Pig code.

  M,38,95111

  F,29,95060

  F,45,95192

  M,62,95102

  F,56,95102

  A = LOAD andapos;dataandapos; USING PigStorage(andapos;.andapos;) as (gender:Chararray, age:int,

  zlp:chararray);

  B = FOREACH A GENERATE age;

  Which one of the following commands would save the results of B to a folder in hdfs named myoutput?

  A. STORE A INTO andapos;myoutputandapos; USING PigStorage(andapos;,andapos;);

  B. DUMP B using PigStorage(andapos;myoutputandapos;);

  C. STORE B INTO andapos;myoutputandapos;;

  D. DUMP B INTO andapos;myoutputandapos;;

  Correct Answer: C

• Question 39:

  Which one of the following statements regarding the components of YARN is FALSE?

  A. A Container executes a specific task as assigned by the ApplicationMaster

  B. The ResourceManager is responsible for scheduling and allocating resources

  C. A client application submits a YARW job to the ResourceManager

  D. The ResourceManager monitors and restarts any failed Containers

  Correct Answer: D

• Question 40:

  Your cluster's HDFS block size in 64MB. You have directory containing 100 plain text files, each of which

  is 100MB in size. The InputFormat for your job is TextInputFormat.

  Determine how many Mappers will run?

  A. 64

  B. 100

  C. 200

  D. 640

  Correct Answer: C

  Explanation: Each file would be split into two as the block size (64 MB) is less than the file size (100 MB),

  so 200 mappers would be running.

  Note:

  If you're not compressing the files then hadoop will process your large files (say 10G), with a number of

  mappers related to the block size of the file.

  Say your block size is 64M, then you will have ~160 mappers processing this 10G file (160*64 ~= 10G).

  Depending on how CPU intensive your mapper logic is, this might be an

  acceptable blocks size, but if you find that your mappers are executing in sub minute times, then you might

  want to increase the work done by each mapper (by increasing the block size to 128, 256, 512m - the

  actual size depends on how you intend to process the data). Reference: http://stackoverflow.com/