What is the graphs
Map and Reduce are derived from Functional Language
A map is a meta-function that processes each element in a collection sequentially and independently
Reduce is a meta-function that processes set elements in batches
Example : Word Count
Let’s use the following document as an example
The Map phase
The Map task generates a key/value pair for each record.
For example, (Welcome, 1) this means that the word Welcome has come across before
These records can be handled independently of each other.
In this example, there is only one Map task, so it processes the records in order.
This process can easily be interrupted when data volumes are particularly large.
The input data set needs to be shard or split, assigned to each mirror or each independent Map task, and output generated
Reduce
Reduce merges and processes intermediate results on a per-key basis
Each record is assigned to a different Reduce task based on the key. The Hash Paritrees algorithm is typically used to assign keys. Reduce tasks process and merge intermediate values in parallel based on assigned keys.
