Skip to content

Logs and Observability

Applies to: All subscriptions

Purpose

Make Complifly's logs useful: collected, retained, searchable, and correlated — so a failure can be traced from a user's report to the exact request that caused it.

Audience

Operations teams, support engineers, infrastructure teams.

Prerequisites

  • Installation Guide complete
  • A log collection platform, or a decision to operate without one

Reference

What produces logs

Source Contains
Application service Requests, business operations, errors, startup diagnostics
Background worker Job execution, write-back dispatch, retention runs
Reverse proxy Access logs, TLS negotiation, upstream errors
Database Errors, and whatever auditing is configured
Process supervisor Restarts, crashes, resource limits

Collect all of them. A failure at the proxy leaves nothing in the application log, and a failure in the worker leaves nothing in the application log either. Investigations that only have the application log stall regularly.

Log levels

Level Use
error Something failed
warn Something is wrong but was handled
info Production default. Significant events
debug Diagnosis only. High volume

Run production at info. Raise to debug deliberately, for a bounded period, and lower it again — an environment left at debug fills disks and increases the chance of sensitive data being written.

The request identifier

Every request carries an identifier that appears in the response and in every log line for that request.

This is the single most useful thing in this section. It turns "an invoice failed yesterday afternoon" — which is an archaeology exercise — into a direct lookup.

Practice Why
Integrations store it against every call Makes any later investigation direct
Support asks for it first Fastest route to the evidence
Log searches index it Retrieve the whole request in one query

Structured logging

Logs are emitted as structured records rather than free text, so they can be filtered by field — level, request identifier, operation — instead of by substring matching.

Configure your collector to parse them as structured data. Ingesting them as plain text discards most of their value.

Collection and retention

Requirement Reason
Ship logs off the host A compromised or failed host must not take its own evidence with it
Rotate local logs Unrotated logs fill disks — a common cause of an unexplained outage
Retain long enough to investigate Compliance questions arrive weeks after the event
Make them searchable Logs nobody can query are logs nobody uses
Restrict access They contain business data

Retention has two answers. Application and diagnostic logs carry no retention obligation of their own — set the period from your own investigation and security needs. The audit trail is separate, and it is not a log: Complifly retains it for 8 years after the end of the financial year, append-only, and it is covered by Audit Log Review. A short retention setting for application logs does not shorten the audit trail.

What must never appear in a log

Never log Why
Credentials, tokens, keys A log becomes a credential store
Full personal data Data protection obligations apply to logs too
Complete document payloads at info Volume, and sensitivity

Spot-check the logs after enabling debug for any diagnosis. Verbose logging is where sensitive data most often escapes, and it escapes quietly.

Exception aggregation

Set SENTRY_DSN (see Environment Variables) and every unhandled error the application service raises — from a request, or from a background worker crash — is reported there automatically: deduplicated, with the request identifier attached, instead of existing only as one line in a log you'd otherwise have to already know to search for. Absent, this is a true no-op — no network calls, no dependency on a Sentry account existing at all.

The application never sends a request body, header, or any field carrying a GSTIN or PAN to Sentry — these are stripped before the report leaves the process, the same discipline applied to structured logs.

Platform event log and admin dashboards

Two things are easy to conflate here: what shows on a dashboard, and what triggers an email. The dashboards below show far more activity than the health sweep will ever send an email about — most of what they show is visibility only.

Independent of a log collector, Complifly maintains its own operational record: platform_event_log, populated as GST, e-Way Bill, GSP, upload, session and login events happen. It deliberately holds only business-shaped outcomes (a generation succeeded, a session was evicted at the concurrency limit) — never full payloads — so it stays small and fast to query. Only Complifly's own roles (Super Admin, Support Engineer, Platform Engineer) read it through the Observability tab; a customer's Org Admin and External Auditor do not see it. The record is not separated by customer, so every figure on those dashboards includes other customers' activity, and showing it to a customer would disclose that. Until customer-scoped dashboards exist, your own failures are in Admin & Logs > Audit Logs, the Attention Center, and the IRN and e-way bill screens, which show your registrations only. The dashboards on that Admin & Logs > Observability tab are Platform, GST, E-Invoice, E-Way Bill, Session Governance, GSP, Security Health, Security — Authentication (failed sign-ins, most-targeted accounts, top source addresses, rate-limit blocks) and five Housekeeping panels.

A separate health sweep runs every five minutes and checks only this fixed list of conditions — nothing outside this table sends an email, no matter how it looks on a dashboard:

Condition Sends an email when
SQL connectivity The database cannot be reached
Process restart loop A process restarts more than expected in a short window
GST / e-Way Bill failure rate Generation failures exceed 10% over 15 minutes and at least 5 attempts were made. Rejections caused by the customer's own data (local validation) are not counted
GSP degradation A provider's success rate falls below 50% over 10 minutes
Audit-write failures Any compliance audit-log write fails
Upload failure rate Upload job failures exceed 20% over an hour, with at least 5 jobs
Failed sign-ins 25 failed sign-ins in 15 minutes, or 10 against a single account
Housekeeping — stale An enabled retention policy has had no successful run for 48 hours
Housekeeping — failed An enabled retention policy's latest run failed

Each condition emails once when it first breaches, again at most every 30 minutes while still open, and once more when it clears — never on every 5-minute tick. See Environment Variables for the switches that turn this on and who receives it.

Attention Center checks write two event types here, visible on the Platform panel: ATTENTION_SWEEP_SUCCESS (one per completed pass, with how many findings it opened and closed) and ATTENTION_SWEEP_FAILURE (a pass in which a rule failed for at least one registration, or that took longer than its interval). ATTENTION_RULE_INVALID means a rule was skipped because its configuration could not be used. None of these sends an email; the Attention Center itself shows "checks are not running" to users when no pass has completed recently.

This is deliberately not a full APM: there is still no distributed tracing. It answers "what happened" for the events it tracks; it does not replace a log collector or Sentry for diagnosing something outside that list.

A lightweight, in-memory requests/minute, error-rate, success-rate and average-response-time view (Phase 1.1) now sits on the Platform panel alongside it — a 5-minute rolling window, reset on a process restart by design, not a substitute for a real APM's historical or cross-VM view.

Tracing a failure end to end

  User reports: "invoice INV-2026-0001 failed this morning"
        |
        v
  1. Find the document in the application. Note its status and any stored error
        |
        v
  2. If it came by API, the integration has the request identifier
        |
        v
  3. Search logs for that identifier -> the complete request
        |
        v
  4. If it reached the provider, the log shows the call and the response
        |
        v
  5. Classify: validation, platform, government, or integration
        |
        v
  6. Resolve, or escalate with the identifier and the classification

Step 2 is where investigations most often stall, because the integration did not store the identifier. Building that in is a few lines of code and saves hours repeatedly.

The audit trail is not the log

Application log Audit trail
Purpose Diagnosis Accountability
Audience Engineers Auditors, compliance
Content Technical detail Who did what, when
Retention You set it 8 years after the financial year; append-only
Queried through The log platform The application

Do not use one in place of the other. An auditor will not accept a log file, and an engineer cannot diagnose from an audit trail.

Validation

Check Method Pass condition
All sources collected Review the collector's inputs Application, worker, proxy, supervisor all present
Structured parsing works Filter by a field Works, without substring matching
Request identifier searchable Search for one from a recent response Returns that request's lines
Level is correct Check the running configuration info in production
Rotation configured Inspect local log directories Bounded size
Retention adequate Search for an event from the retention boundary Present
No secrets in logs Search for credential patterns None found
Access restricted Review permissions Only those who need it
Worker logs present Trigger a background job Its execution is visible
Proxy logs present Cause a proxy-level error Visible

Troubleshooting

Symptom Cause Action
Investigation stalls with no evidence Only application logs collected Add worker, proxy and supervisor logs
A failure cannot be traced to a request Request identifier not stored by the integration Store it on every call
Logs fill the disk Rotation absent, or left at debug Configure rotation; return to info
Logs unsearchable Ingested as plain text Configure structured parsing
Evidence gone when needed Retention too short Extend it
Sensitive data found in logs Verbose logging during diagnosis Lower the level, purge the affected logs, rotate anything exposed
Nothing in the log for a failing button The failure is at the proxy, or in the front end Check proxy logs and the browser console
An auditor rejects a log file as evidence Logs are not the audit trail Use the audit trail