Free CCD-410 Exam Dumps PDF Preview

Question 1

Single choice

Your cluster's HDFS block size in 64MB. You have directory containing 100 plain text files, each of which is 100MB in size. The InputFormat for your job is TextInputFormat.
Determine how many Mappers will run?

A
64
B
100
C
200
D
640

Reveal answer details Close answer details

Correct answerC

Explanation

Each file would be split into two as the block size (64 MB) is less than the file size (100 MB), so 200 mappers would be running.

Note:

If you're not compressing the files then hadoop will process your large files (say 10G), with a number of mappers related to the block size of the file.

Say your block size is 64M, then you will have ~160 mappers processing this 10G file (160*64 ~= 10G).
Depending on how CPU intensive your mapper logic is, this might be an acceptable blocks size, but if you find that your mappers are executing in sub minute times, then you might want to increase the work done by each mapper (by increasing the block size to 128, 256, 512m - the actual size depends on how you intend to process the data).

References:
http://stackoverflow.com/questions/11014493/hadoop-mapreduce-appropriate-input-files-size
(first answer, second paragraph)

Question 2

Single choice

Identify the MapReduce v2 (MRv2 / YARN) daemon responsible for launching application containers and monitoring application resource usage?

A
ResourceManager
B
NodeManager
C
ApplicationMaster
D
ApplicationMasterService
E
TaskTracker
F
JobTracker

Reveal answer details Close answer details

Correct answerB

Explanation

References:
Apache Hadoop YARN Concepts & Applications

Question 3

Single choice

You have a directory named jobdata in HDFS that contains four files: _first.txt, second.txt, .third.txt and #data.txt.

How many files will be processed by the FileInputFormat.setInputPaths () command when it's given a path object representing this directory?

A
Four, all files will be processed
B
Three, the pound sign is an invalid character for HDFS file names
C
Two, file names with a leading period or underscore are ignored
D
None, the directory cannot be named jobdata
E
One, no special characters can prefix the name of an input file

Reveal answer details Close answer details

Correct answerC

Explanation

Files starting with '_' are considered 'hidden' like unix files starting with '.'.

# characters are allowed in HDFS file names.

Question 4

Single choice

What is a SequenceFile?

A
A SequenceFile contains a binary encoding of an arbitrary number of homogeneous writable objects.
B
A SequenceFile contains a binary encoding of an arbitrary number of heterogeneous writable objects.
C
A SequenceFile contains a binary encoding of an arbitrary number of WritableComparable objects, in sorted order.
D
A SequenceFile contains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be same type.

Reveal answer details Close answer details

Correct answerD

Explanation

SequenceFile is a flat file consisting of binary key/value pairs.

There are 3 different SequenceFile formats:

Uncompressed key/value records.

Record compressed key/value records - only 'values' are compressed here.

Block compressed key/value records - both keys and values are collected in 'blocks' separately and compressed. The size of the 'block' is configurable.

References:
http://wiki.apache.org/hadoop/SequenceFile

Question 5

Single choice

Assuming default settings, which best describes the order of data provided to a reducer's reduce method:

A
The keys given to a reducer aren't in a predictable order, but the values associated with those keys always are.
B
Both the keys and values passed to a reducer always appear in sorted order.
C
Neither keys nor values are in any predictable order.
D
The keys given to a reducer are in sorted order but the values associated with each key are in no predictable order

Reveal answer details Close answer details

Correct answerD

Explanation

Reducer has 3 primary phases:

1. Shuffle

The Reducer copies the sorted output from each Mapper using HTTP across the network.

2. Sort

The framework merge sorts Reducer inputs by keys (since different Mappers may have output the same key).

The shuffle and sort phases occur simultaneously i.e. while outputs are being fetched they are merged.

SecondarySort

To achieve a secondary sort on the values returned by the value iterator, the application should extend the key with the secondary key and define a grouping comparator. The keys will be sorted using the entire key, but will be grouped using the grouping comparator to decide which keys and values are sent in the same call to reduce.

3. Reduce

In this phase the reduce(Object, Iterable, Context) method is called for each <key, (collection of values)> in the sorted inputs.

The output of the reduce task is typically written to a RecordWriter via TaskInputOutputContext.write (Object, Object).

The output of the Reducer is not re-sorted.

References:
org.apache.hadoop.mapreduce, Class
Reducer<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

Question 6

Single choice

A combiner reduces:

A
The number of values across different keys in the iterator supplied to a single reduce method call.
B
The amount of intermediate data that must be transferred between the mapper and reducer.
C
The number of input files a mapper must process.
D
The number of output files a reducer must produce.

Reveal answer details Close answer details

Correct answerB

Explanation

Combiners are used to increase the efficiency of a MapReduce program. They are used to aggregate intermediate map output locally on individual mapper outputs. Combiners can help you reduce the amount of data that needs to be transferred across to the reducers. You can use your reducer code as a combiner if the operation performed is commutative and associative. The execution of combiner is not guaranteed, Hadoop may or may not execute a combiner. Also, if required it may execute it more then 1 times.
Therefore your MapReduce jobs should not depend on the combiners execution.

References:
24 Interview Questions & Answers for Hadoop MapReduce developers, What are combiners?
When should I use a combiner in my MapReduce Job?

Question 7

Single choice

In a MapReduce job, the reducer receives all values associated with same key.

Which statement best describes the ordering of these values?

A
The values are in sorted order.
B
The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce
job.
C
The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.
D
Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

Reveal answer details Close answer details

Correct answerB

Explanation

Note:
* Input to the Reducer is the sorted output of the mappers.
* The framework calls the application's Reduce function once for each unique key in the sorted order.
* Example:
For the given sample input the first map emits:
< Hello, 1>
< World, 1>
< Bye, 1>
< World, 1> The second map emits:
< Hello, 1>
< Hadoop, 1>
< Goodbye, 1>
< Hadoop, 1>

Question 8

Single choice

Which describes how a client reads a file from HDFS?

A
The client queries the NameNode for the block location(s). The NameNode returns the block location (s) to the client. The client reads the data directory off the DataNode(s).
B
The client queries all DataNodes in parallel. The DataNode that contains the requested data responds directly to the client. The client reads the data directly off the DataNode.
C
The client contacts the NameNode for the block location(s). The NameNode then queries the DataNodes for block locations. The DataNodes respond to the NameNode, and the NameNode redirects the client to the DataNode that holds the requested data block(s). The client then reads the data directly off the DataNode.
D
The client contacts the NameNode for the block location(s). The NameNode contacts the DataNode that holds the requested data block. Data is transferred from the DataNode to the NameNode, and then from the NameNode to the client.

Reveal answer details Close answer details

Correct answerA

Explanation

References:
24 Interview Questions & Answers for Hadoop MapReduce developers, How the Client
communicates with HDFS?

Question 9

Single choice

Identify the tool best suited to import a portion of a relational database every day as files into HDFS, and generate Java classes to interact with that imported data?

A
Oozie
B
Flume
C
Pig
D
Hue
E
Hive
F
Sqoop
G
fuse-dfs

Reveal answer details Close answer details

Correct answerF

Explanation

Sqoop ("SQL-to-Hadoop") is a straightforward command-line tool with the following capabilities:

Imports individual tables or entire databases to files in HDFS

Generates Java classes to allow you to interact with your imported data

Provides the ability to import from SQL databases straight into your Hive data warehouse

Note:

Data Movement Between Hadoop and Relational Databases

Data can be moved between Hadoop and a relational database as a bulk data transfer, or relational tables can be accessed from within a MapReduce map function.
Note:

* Cloudera's Distribution for Hadoop provides a bulk data transfer tool (i.e., Sqoop) that imports individual tables or entire databases into HDFS files. The tool also generates Java classes that support interaction with the imported data. Sqoop supports all relational databases over JDBC, and Quest Software provides a connector (i.e., OraOop) that has been optimized for access to data residing in Oracle databases.

References:
http://log.medcl.net/item/2011/08/hadoop-and-mapreduce-big-data-analytics-gartner/
(Data Movement between hadoop and relational databases, second paragraph)

Free CCD-410 Exam Dumps Preview

Cloudera Certified Developer for Apache Hadoop (CCDH)

PREVIEW CONTENTS · 1 VOLUME

QUESTION 1

QUESTION 2

QUESTION 3

QUESTION 4

QUESTION 5

QUESTION 6

QUESTION 7

QUESTION 8

QUESTION 9