Skip to content

Logs and Observability

Applies to: All subscriptions

Purpose

Make Complifly's logs useful: collected, retained, searchable, and correlated — so a failure can be traced from a user's report to the exact request that caused it.

Audience

Operations teams, support engineers, infrastructure teams.

Prerequisites

  • Installation Guide complete
  • A log collection platform, or a decision to operate without one

Reference

What produces logs

Source Contains
Application service Requests, business operations, errors, startup diagnostics
Background worker Job execution, write-back dispatch, retention runs
Reverse proxy Access logs, TLS negotiation, upstream errors
Database Errors, and whatever auditing is configured
Process supervisor Restarts, crashes, resource limits

Collect all of them. A failure at the proxy leaves nothing in the application log, and a failure in the worker leaves nothing in the application log either. Investigations that only have the application log stall regularly.

Log levels

Level Use
error Something failed
warn Something is wrong but was handled
info Production default. Significant events
debug Diagnosis only. High volume

Run production at info. Raise to debug deliberately, for a bounded period, and lower it again — an environment left at debug fills disks and increases the chance of sensitive data being written.

The request identifier

Every request carries an identifier that appears in the response and in every log line for that request.

This is the single most useful thing in this section. It turns "an invoice failed yesterday afternoon" — which is an archaeology exercise — into a direct lookup.

Practice Why
Integrations store it against every call Makes any later investigation direct
Support asks for it first Fastest route to the evidence
Log searches index it Retrieve the whole request in one query

Structured logging

Logs are emitted as structured records rather than free text, so they can be filtered by field — level, request identifier, operation — instead of by substring matching.

Configure your collector to parse them as structured data. Ingesting them as plain text discards most of their value.

Collection and retention

Requirement Reason
Ship logs off the host A compromised or failed host must not take its own evidence with it
Rotate local logs Unrotated logs fill disks — a common cause of an unexplained outage
Retain long enough to investigate Compliance questions arrive weeks after the event
Make them searchable Logs nobody can query are logs nobody uses
Restrict access They contain business data

Verify: Complifly's recommended log retention period is not stated in this portal. Set it from the customer's own investigation and audit needs, and confirm with Complifly whether any longer obligation applies. Recorded as assumption E4 in the Assumptions Register.

What must never appear in a log

Never log Why
Credentials, tokens, keys A log becomes a credential store
Full personal data Data protection obligations apply to logs too
Complete document payloads at info Volume, and sensitivity

Spot-check the logs after enabling debug for any diagnosis. Verbose logging is where sensitive data most often escapes, and it escapes quietly.

Tracing a failure end to end

  User reports: "invoice INV-2026-0001 failed this morning"
        |
        v
  1. Find the document in the application. Note its status and any stored error
        |
        v
  2. If it came by API, the integration has the request identifier
        |
        v
  3. Search logs for that identifier -> the complete request
        |
        v
  4. If it reached the provider, the log shows the call and the response
        |
        v
  5. Classify: validation, platform, government, or integration
        |
        v
  6. Resolve, or escalate with the identifier and the classification

Step 2 is where investigations most often stall, because the integration did not store the identifier. Building that in is a few lines of code and saves hours repeatedly.

The audit trail is not the log

Application log Audit trail
Purpose Diagnosis Accountability
Audience Engineers Auditors, compliance
Content Technical detail Who did what, when
Retention Operational Per compliance requirement
Queried through The log platform The application

Do not use one in place of the other. An auditor will not accept a log file, and an engineer cannot diagnose from an audit trail.

Validation

Check Method Pass condition
All sources collected Review the collector's inputs Application, worker, proxy, supervisor all present
Structured parsing works Filter by a field Works, without substring matching
Request identifier searchable Search for one from a recent response Returns that request's lines
Level is correct Check the running configuration info in production
Rotation configured Inspect local log directories Bounded size
Retention adequate Search for an event from the retention boundary Present
No secrets in logs Search for credential patterns None found
Access restricted Review permissions Only those who need it
Worker logs present Trigger a background job Its execution is visible
Proxy logs present Cause a proxy-level error Visible

Troubleshooting

Symptom Cause Action
Investigation stalls with no evidence Only application logs collected Add worker, proxy and supervisor logs
A failure cannot be traced to a request Request identifier not stored by the integration Store it on every call
Logs fill the disk Rotation absent, or left at debug Configure rotation; return to info
Logs unsearchable Ingested as plain text Configure structured parsing
Evidence gone when needed Retention too short Extend it
Sensitive data found in logs Verbose logging during diagnosis Lower the level, purge the affected logs, rotate anything exposed
Nothing in the log for a failing button The failure is at the proxy, or in the front end Check proxy logs and the browser console
An auditor rejects a log file as evidence Logs are not the audit trail Use the audit trail