ds.granules.operation

Interface Reduce

All Superinterfaces:

ds.funnel.data.format.FormatMarshaller, MapReduce, Operation

All Known Implementing Classes:

MapReduceBase, StreamBase, StreamProcessor, StreamSource

public interface Reduce
extends MapReduce

This interface corresponds to the Reduce role in a Map-Reduce operation. It incorporates support for adding and removing Map functions. Additionally, the Reducer can keep track of Map from which Outputs have been received, and those from which Outputs are currently pending.

Author:

Shrideep Pallickara

Nested Class Summary

Nested classes/interfaces inherited from interface ds.granules.operation.Operation

Operation.OperationType

Method Summary

Methods

Modifier and Type	Method and Description
int	addMap(Map map) Adds a Map that will be reduced.
Iterator<String>	getAwaitingOutputsFrom() Retrieves the set of Map identifiers from which an Output has not been received so far.
Enumeration<String>	getMapIdentifiers() Retrieves the set of Map identifiers
int	getNumberOfMaps() Retrieves the number of registered Map instances
boolean	isAwaitingOutputs() Check to see is the Reducer is missing outputs from any of the registered Maps.
int	removeMap(String mapIdentifier) Removes a Map from the list of Maps that are registered to this reducer.
void	resetOutputTracker() This method resets the output tracker associated with the reducer.
void	setReceivedOutputFrom(String mapIdentifier) Mark the receipt of Output from a given Map instance

Methods inherited from interface ds.granules.operation.MapReduce

createResults, writeResults, writeResults, writeResults, writeResults

Methods inherited from interface ds.granules.operation.Operation

dispose, execute, getDatasetCollection, getDomain, getExecutionProfile, getInstanceIdentifier, getOperationIdentifier, getProcessingDirectives, getVersionInformation, hasDatasetCollection, hasExecutionProfile, hasProcessingDirectives, initialize, isInitialized, needsInitialization, terminationConditionReached, toString, unmarshall

Methods inherited from interface ds.funnel.data.format.FormatMarshaller

marshall

Method Detail

getNumberOfMaps

int getNumberOfMaps()

Retrieves the number of registered Map instances

Returns:

The number of Maps

getMapIdentifiers

Enumeration<String> getMapIdentifiers()

Retrieves the set of Map identifiers

Returns:

An enumeration of the list of registered identifiers

isAwaitingOutputs

boolean isAwaitingOutputs()

Check to see is the Reducer is missing outputs from any of the registered Maps.

Returns:

true is the Reducer is still awaiting outputs; false otherwise.

getAwaitingOutputsFrom

Iterator<String> getAwaitingOutputsFrom()

Retrieves the set of Map identifiers from which an Output has not been received so far.

Returns:

An iterator of the pending identifiers

resetOutputTracker

void resetOutputTracker()

This method resets the output tracker associated with the reducer. This is useful when the stages are part of an iterative computation.

setReceivedOutputFrom

void setReceivedOutputFrom(String mapIdentifier)

Mark the receipt of Output from a given Map instance

Parameters:

mapIdentifier - The ID of the Map from which an output has been received.

addMap

int addMap(Map map)
           throws MapReduceException

Adds a Map that will be reduced.

Parameters:

map - Add a Map instance for this reducer

Returns:

The number of maps that are registered with this reducer

Throws:

MapReduceException

removeMap

int removeMap(String mapIdentifier)
              throws MapReduceException

Removes a Map from the list of Maps that are registered to this reducer.

Parameters:

mapIdentifier - The identifier of the Map to removed

Returns:

The number of maps that are registered with this reducer

Throws:

MapReduceException - If there are problems removing the Map.