only keep dedupIds no older than 6 months to make the rocksdb size consistent

[jackyjia]

Background

Each journal entry's dedup id is kept in rocksdb, to avoid processing the same journal entry multiple times. However if we keep adding all dedup ids in rocksdb, the rocksdb size will increase and eventually run out of disk space. To make the size consistent, we can remove some dedup ids whose validTime is for example 6 months old (configurable). And if ledger receives a journal entry older than 6 months, it'll reject to avoid processing the same request twice.

Acceptance Criteria

• A scheduled job to remove dedup ids older than a x days
• If ledger receives a journal entry older than x days, reject
• x days can be configured on the fly