personaEngine2/docs/Logging.md

Logging

Structured audit output: what is emitted, where it goes, and what may never appear in it.

The serialized contract is audit-record.md. This document covers the operational side.

Format and transport

Newline-delimited JSON (OTD-006). One record per line, UTF-8 without BOM, appended.

Every record passes through a single sink, Write-PersonaAuditRecord. Adding a transport — an approved logging platform, an event hub, a different file layout — is a change to that one function. If call sites wrote their own output, each new transport would mean auditing every call site again, and the one that got missed would be silent.

logging.destination	Behaviour
file	Appends NDJSON to logging.path
stream	Emits the record object on the PowerShell Information stream
both	Both
none	Nothing

stream uses the Information stream rather than the success stream deliberately. Audit records on the success stream would be indistinguishable from a function's return value — the run loop returns its outcome there — and mixing the two turns one object into an array of several thousand. Capture them with -InformationVariable, or redirect with 6>.

Sink failure never ends a run

A full disk or a locked file is an operational problem with the sink, not a reason to abandon a classification run mid-population and leave the directory half-reconciled. The failure surfaces as a warning, once per run, and processing continues.

Once, not once per user: a run over five thousand accounts with a locked log file should warn once, or the warning that matters is buried in the noise it generates.

Record types

Type	When	Carries
RunStart	Once, after mode is determined	Config path, target attribute, rule counts, whether tracing is on
UserEvent	Once per processed account	The full decision
Summary	Every summaryInterval accounts, and once at the end	Counters, per-rule match counts, reconciliation result
RunComplete	Once, in a finally block	Final counters, timing, exit code
EngineDefect	Reconciliation failure, or threshold breach	What went wrong and by how much

RunComplete is written even on a fatal error. A run that died at account 400 of 5,000 leaves a record saying exactly that — which is what lets an operator tell "the engine stopped early" from "the engine never started", two very different incidents that produce identical evidence if the record is written only on success.

The common envelope

Every record, every type:

timestamp · recordType · runId · engineVersion · configVersion · configurationHash · mode

These appear first in each record, so a truncated line still identifies the run that produced it.

runId comes from -CorrelationId or is generated, and is constant for the run (NFR-005). configurationHash is the SHA-256 of the configuration file bytes — two files differing only in whitespace are different configurations for audit purposes, and the hash must be reproducible from the artifact on disk.

UserEvent

100% carry runId, userPrincipalName, and accountObjectId (SC-006). 100% of Matched records carry matchedRuleId. evaluationErrorReason is non-null exactly when outcome is EvaluationError.

previousValue is present only on Updated records, captured at write time. On any other action there is nothing that was replaced, and a populated previousValue would imply otherwise to a rollback tool reading these records later. Without it, OTD-010 rollback is impossible retroactively — no future run can reconstruct what a value used to be.

What may never appear

Access tokens, Authorization headers, client secrets, certificates, credentials, and full Graph responses.

The guarantee is structural rather than filtered. New-PersonaAuditRecord accepts only named, typed values from the decision result and the counters — there is no pass-through of an arbitrary object, so there is nothing for a secret to ride in on. AuditRedaction.Tests.ps1 asserts this holds even when a caller attaches a token to the decision result, and scans every serialized record for JWT and Bearer shapes.

Approved for logs

User principal name, account object ID, matched rule ID, stored and calculated persona values, run ID, configuration version and hash, per-rule match counts, timing.

Runtime records naturally contain real UPNs and Object IDs. No such value may ever be committed to this repository (SC-013) — the sanitization scan enforces that on every build.

Condition tracing

conditionTrace is added to a UserEvent only when both gates are open: the decision result was built with tracing, and the record was asked to include it. It carries the per-rule result (True / False / Unknown) and priority.

Tracing widens what the log contains beyond the approved set, so it requires logging.acknowledgeConditionTracing in the same configuration (PE-SAF-006). Tracing never changes a decision — if it could, a debug run would stop being evidence about the real one, and ConditionTrace.Tests.ps1 asserts the outcome is identical with and without it.

Querying

# Everything from one run
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
    Where-Object runId -eq '<RUN-ID>'

# Accounts a run changed, with what it replaced
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
    Where-Object { $_.recordType -eq 'UserEvent' -and $_.action -eq 'Updated' } |
    Select-Object userPrincipalName, previousValue, calculatedPersona, matchedRuleId

# Accounts that could not be evaluated, and why
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
    Where-Object outcome -eq 'EvaluationError' |
    Select-Object userPrincipalName, evaluationErrorReason

# Which rule set produced a given decision
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
    Where-Object recordType -eq 'RunStart' |
    Select-Object runId, configVersion, configurationHash, mode

Retention

Not set by this engine. Records contain UPNs and Object IDs, so retention is governed by the organization's identity-data policy rather than by anything in this repository. Decide it before the first enforcing run, not after.