|
|
Reduce the number of shuffled records |
On the flow above, the number of shuffled records, equals to number of records in all partitions.
The following figure, illustrates the word-count calculation and shuffle process without combiner.
On the flow above, before Group By, We use Combiner to calculation local/partition aggregation.
So the number of shuffled record are reduced. It will increase performance.
