Skip to content

Housekeeping

Applies to: All subscriptions

Purpose

Keep storage growth under control without deleting anything that must be retained — and make clear which of the two obligations wins when they conflict.

Audience

Infrastructure teams, database administrators, and the customer's compliance lead for the retention decisions.

Prerequisites

  • Sizing and Capacity
  • The customer's statutory retention obligation, confirmed by their tax and legal leads

Reference

What grows, and whether it may be trimmed

Data Grows with May be purged?
Documents and compliance records Documents processed No — statutory retention
Reference numbers, signed QR codes, acknowledgements Documents registered No — this is the evidence
Stored submission payloads Documents submitted No — evidence of what was sent
Raw ERP payloads Documents ingested Yes, after a configured period
Cached rendered documents Documents printed Yes — regenerable from stored data
Delivered write-back events Events emitted Yes, after a configured period
Dead-lettered write-back events Failures Yes, but see the warning below
Delivery attempt history Delivery attempts Yes, after a period
Upload files Uploads Yes, after processing and a grace period
Application logs Activity Yes, per log retention
Audit trail Activity Per the compliance requirement, not per convenience
Telemetry Activity Yes

The first three rows are the reason storage forecasting matters: they cannot be trimmed to recover space, ever.

The rule when obligations conflict

Statutory retention wins over a data-minimisation preference.

Compliance evidence has a retention obligation that may well exceed what a privacy-minded policy would otherwise prefer. Any deletion policy must be checked against that obligation before implementation, not after a purge.

Verify: The retention period Complifly applies to compliance evidence, and its statutory basis, are not stated in this portal. Confirm before configuring any purge. Recorded as assumption C2 in the Assumptions Register.

The retention settings that matter

Setting Controls Consideration
Raw ERP payload retention How long the original ERP payload is kept Long enough to investigate a mapping dispute
Delivered event retention How long successfully delivered events are kept Long enough to answer "did you send it?"
Dead-letter retention How long failed events are kept Must exceed your realistic reaction time
Cached document retention How long rendered documents are cached Purely a performance and space trade-off
Print horizon How far back documents can be reprinted Affects what must remain retrievable

The dead-letter row is the one with a real trap. A weekend outage with a two-day retention purges the evidence before anyone returns to investigate. Set it against how quickly your team actually responds, including over holidays — not how quickly it intends to.

The pattern for any purge

Step Action
1 Confirm the statutory position with the tax and legal leads
2 Establish what will be affected, precisely
3 Verify a backup exists and has been verified
4 Run against a non-production copy first
5 Verify the count against storage metadata, not against another scoped query
6 Run in production, in a maintenance window
7 Verify again, the same way
8 Record what was purged, when, and on whose authority

Step 5 exists because of how this database behaves: a query written without tenant scope returns zero rows and reports success. A purge written that way deletes nothing and reports success, and a verification query written the same way agrees with it. Read counts from storage metadata, which isolation policies do not filter.

Controlling growth without deleting

Options that reduce cost without touching retention:

Option Effect
Archive older data to cheaper storage Retained, retrievable, cheaper
Compress at the database level Smaller, transparent
Shorten cache retention Regenerable data only
Shorten raw payload retention Accepting a shorter investigation window
Move log retention to a log platform Off the database entirely

Archival is usually the right first move. It satisfies the retention obligation while removing the cost pressure.

Validation

Check Method Pass condition
Growth matches forecast Measure monthly Within 25 percent, or the forecast revised
Retention jobs run Inspect job history Completing successfully
Purges are effective Count against storage metadata before and after The expected reduction, verified independently
Nothing statutory purged Review the policy against the obligation Confirmed by the tax lead
Dead-letter retention adequate Compare against actual reaction times Exceeds them, including holidays
Archive retrievable Retrieve a sample Succeeds
Free space adequate Check every volume Above the warning threshold
Purges are recorded Review the change log What, when, and on whose authority

Troubleshooting

Symptom Cause Action
A purge reports success and frees nothing Scope not set; isolation filtered every row Set scope and verify against storage metadata
Storage growing faster than forecast Volume above plan, or retention longer than assumed Re-forecast; review retention deliberately
Disk fills despite retention jobs Jobs failing silently, or the growth is in non-purgeable data Check job history first; then check which category is growing
Dead-lettered events gone before investigation Retention shorter than reaction time Raise it
Cannot reprint an older document Beyond the print horizon Adjust the horizon deliberately, understanding the space cost
Evidence unavailable for an audit Purged, or archived without a tested retrieval path Test archive retrieval before relying on it
A purge deleted something needed Statutory position not confirmed first Restore from backup. Confirm the position before any future purge
Logs filling the disk Log rotation absent, or verbose logging left on Configure rotation; return to normal log level