Falcon uses distcp (distributed copy) for data replication. If you need to optimize bandwidth between data centers, you can throttle bandwidth during Falcon data replication as needed and limit the number of maps used during replication.
To throttle distcp:
If you already have Falcon running on your clusters, suspend your current active feeds and processes:
$FALCON_HOME/bin/falcon entity -type $feedName -name $name -suspend
$FALCON_HOME/bin/falcon entity -type $processName -name $name -suspend
Edit your feed entity or entities. Add the following lines:
<properties> <property name="maxMaps" value="$integerValue" /> <property name="maxBandwidth" value="$MB/svalue" /> </properties>
Note Specify the maximum number of mappers for Falcon to use in
maxMaps
. Specify the bandwidth in MB for each mapper inmapBandwidth
.Sumbit your updated feed entity.
$FALCON_HOME/bin/falcon entity -submit -type feed -file ~$feedFileName
Resume your processes.
$FALCON_HOME/bin/falcon entity -type $processName -name $name -resume
$FALCON_HOME/bin/falcon entity -type $feedName -name $name -resume