You can set retention policies on a per-cluster basis. You must specify the amount of time to retain data before deletion.
Falcon kicks off the retention policy on the basis of the time value you specify:
Less than 24 hours: Falcon kicks off the retention policy every 6 hours.
More than 24 hours: Falcon kicks off the retention policy every 24 hours.
When a feed is scheduled: Falcon kicks off the retention policy immediately.
Note | |
---|---|
When a feed is successfully scheduled, Falcon triggers the retention policy immediately regardless of the current timestamp/state of the cluster. |
To set a retention policy, add the following lines to your cluster entity as part of the <cluster> definition:
<clusters> <cluster name="corp" type="source"> <validity start="2012-01-30T00:00Z" end="2013-03-31T23:59Z" timezone="UTC" /> <retention limit="$unitOfTime($n)" action="delete" /> <!--Retention policy. --> </cluster> </clusters>
Where limit
can be minutes, hours, days, or months and then a specified
numeric value. Falcon then retains data spanning from the current moment back to the
time specified in the attribute. Any data beyond the limit (past or future) is
erased.
Note | |
---|---|
Delete is the only supported action for HDP 2.1. |