Implement Stage A: rule engine, validation, audit, and safety gates

Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).

  354 offline Pester tests      PASS
  Engine purity (Principle IV)  PASS
  Sanitization (SC-013)         PASS  (156 files)
  Graph module loaded in tests  none  (SC-008 holds)

What landed
  - Four-layer configuration validation with stable finding codes, covering
    every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
  - Run loop, audit records (NDJSON through a single sink), summaries,
    reconciliation, and exit codes 0-6
  - Persistence behind a single write-body builder whose result always has
    exactly one key
  - Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
  - Six docs, two pipelines, traceability matrix, V-5a and sanitization records

Three deviations from tasks.md, each recorded in its status block

  T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
  state and the rule engine is pure; a counter there would break Principle IV.
  It lives in New-PersonaRunCounter and is applied in the run loop.

  A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
  imports the manifest, which requires Microsoft.Graph.Authentication, so a
  loop living only inside it could not run on a machine without the Graph SDK
  and SC-004 could not be proven at all. The entry script is now a thin
  wrapper and what ships is what is tested.

  The invalid-config corpus is generated by a committed script, with the
  generated fixtures committed too, so a reviewer sees the fixture in the diff.

Defects found by running the code, not by reading it

  Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
  around a comma-returned array collapsed every membership list into one
  bogus space-joined entry. That is a silent false non-match, exactly what
  FR-013 exists to prevent.

  A 403 whose status appears only in the exception message parsed as $null,
  which the retry policy treats as a transport error - five requests per
  account against a tenant already refusing. Status extraction now falls back
  to the message text, bounded to 400-599.

  The sanitization scan walked tracked files only, so it covered 34 of 156
  files and none of this phase's code. It now scans untracked non-ignored
  files too, and a negative control confirms it catches a planted leak.

  Test-Json reports one error per violating location, not first-failure-only
  as the V-5a draft claimed. Record and pin corrected.

Enforcement remains blocked on the V-4 security sign-off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-20 21:48:19 -04:00
parent c59c85dd55
commit cdc6bb33d3
124 changed files with 16638 additions and 199 deletions
+138
View File
@@ -0,0 +1,138 @@
# Architecture
How Persona Engine is put together, and why the boundaries sit where they do.
## The one structural rule
The rule engine is pure. It takes a normalized record and a rule set, and returns a decision. It has
no knowledge of Microsoft Graph, no authentication, no console, no filesystem, and no clock.
Everything else follows from that. It is constitution Principle IV, it is enforced by
[`tests/Test-EnginePurity.ps1`](../tests/Test-EnginePurity.ps1) on every build, and it is the reason
354 tests can run on a laptop with no tenant, no credentials, and no network.
The purity check is a tokenizer pass, not a text search: it parses each file under `src/RuleEngine/`
with the PowerShell AST parser and inspects only the code tokens. An earlier text-matching version
flagged a comment that merely *mentioned* a persistence function, which is the kind of false positive
that gets a check disabled.
## Layers
Loaded in this order by [`PersonaEngine.psm1`](../PersonaEngine.psm1). Order matters only for
readability — PowerShell resolves function names at call time — but the dependency direction is real
and one-way.
| Layer | Responsibility | May depend on |
| --- | --- | --- |
| `Normalization` | Convert raw directory objects into `UserRecord` and `MembershipRecord` | nothing |
| `Configuration` | Load, validate, and resolve configuration | Normalization |
| `RuleEngine` | Evaluate conditions, groups, rules; produce a decision | Normalization only |
| `Authentication` | Acquire a Graph connection | nothing in this module |
| `DataProviders` | Retrieve users, membership, roles; retry policy | Normalization |
| `Persistence` | Compare values; build and issue the write | DataProviders |
| `Presentation` | Per-user output, summaries, reconciliation | nothing |
| `Engine` | The run loop that composes all of the above | everything |
| `Audit` | Build and emit structured records | Presentation |
The arrow never points into `RuleEngine`. Nothing in `RuleEngine` may reference anything from
`Authentication`, `DataProviders`, `Persistence`, or `Presentation`.
## The normalization boundary
`ConvertTo-PersonaUserRecord` and `ConvertTo-PersonaMembershipRecord` are the only places where a raw
Graph shape becomes an engine shape. Downstream of them, nothing knows Graph exists.
Two consequences worth stating plainly:
**Fixtures are real inputs.** A synthetic `UserRecord` built in a test is indistinguishable to the
engine from one built from a live tenant response. That is what makes the offline suite evidence
rather than a rehearsal.
**Filtering rules live in one place.** `memberOf` returns directory objects of mixed type.
Administrative units arriving on that endpoint are discarded during normalization, not by each
caller. An administrative unit ID treated as a group ID would never match — which reads as "not a
member", a false non-match, the exact outcome FR-013 exists to prevent.
## The MembershipRecord shape
Three independently-retrieved facets — direct groups, transitive groups, directory roles — each with
its own retrieval flag and failure reason.
This started as a single record with one `Mode` field, and running the code against the example
configuration is what proved it wrong: RE-007 makes membership mode a **per-condition** choice, so a
rule set may legitimately ask for transitive membership in one rule and direct in another. A record
carrying only one mode cannot answer both, and seven of nine fixtures came back as `EvaluationError`.
The engine was right; the contract was wrong.
Independent facets also contain failure. If the transitive lookup times out but the direct lookup
succeeded, only conditions needing transitive data become `Unknown`. One collapsed flag would turn
one slow endpoint into a tenant-wide outage.
Every `*Retrieved` flag defaults to `$false`. An unset flag means *unknown*, never *not a member*, so
a forgotten flag degrades to `EvaluationError` rather than silently misclassifying a privileged
account.
## Tri-state evaluation
Conditions return `'True'`, `'False'`, or `'Unknown'` — not a boolean. `Unknown` propagates through
condition groups by the table in [data-model.md](../specs/001-persona-engine/data-model.md), and an
`Unknown` reaching a rule's root becomes `EvaluationError` for that account.
An `Unknown` at priority 30 stops evaluation even though a lower-priority rule might have matched.
Continuing would risk assigning a persona from priority 900 when the account may in truth have
matched at 30 — a privilege downgrade drawn from data nobody could read. Preserving the stored value
is the only safe answer.
## The write gate has one origin
Mode is derived from `$PSCmdlet.ShouldProcess()` and nothing else. There is no `-Preview` switch, no
configuration key that suppresses writes, and no reading of `$WhatIfPreference`. Two sources of truth
for a write gate is the defect class Principle III exists to prevent: the day they disagree, one of
them is wrong and the directory finds out first.
`Invoke-PersonaEngine.ps1` owns the `ShouldProcess` call and passes the result down to
`Invoke-PersonaEngineRun` as a scriptblock. The run loop never learns what `-WhatIf` is, so it cannot
disagree with it — and it defaults to a gate that refuses, so a caller that forgets to supply one
previews rather than writes.
## Why the run loop is a module function
`Invoke-PersonaEngine.ps1` is a thin wrapper: parameter binding, module import, `ShouldProcess`,
exit code. The loop itself is `Invoke-PersonaEngineRun` in `src/Engine/`.
That split exists because SC-004 requires proof that a `-WhatIf` run issues zero writes across a full
population. A loop that only exists inside an entry script — one that imports a manifest requiring
the Graph SDK — cannot be exercised without a tenant, so the claim could not be tested. What ships
and what is tested are now the same code.
## Single emission point for audit
Every audit record passes through `Write-PersonaAuditRecord`. Adding a transport is a change to that
one function. If call sites wrote their own output, each new transport would mean auditing every call
site again, and the one that got missed would be silent.
Records go to the **Information** stream, not the success stream. Audit records on the success stream
would be indistinguishable from a function's return value — the run loop returns its outcome there,
and mixing the two turns one object into an array of several thousand.
## Failure containment
| Failure | Scope | Outcome |
| --- | --- | --- |
| Configuration invalid | Run | Exit 1, before any connection is attempted |
| Authentication fails | Run | Exit 2 |
| Enumeration truncated | Run | Exit 3 — a partial population is never processed |
| One membership lookup fails | One facet, one user | `EvaluationError`, stored value preserved |
| Too many `EvaluationError` | Run status only | Exit 4; no value was changed |
| Counters disagree | Run | Exit 5, `EngineDefect` record |
| One write fails | One user | `UpdateFailed`, run continues |
The dividing line: anything affecting the whole population ends the run; anything affecting one
account is contained and reported.
## Related
- [SecurityModel.md](SecurityModel.md) — the OTD-003 trade-off and its compensating controls
- [ConfigurationReference.md](ConfigurationReference.md) — every schema field and finding code
- [OperationsRunbook.md](OperationsRunbook.md) — kill switch and rollback
- [Logging.md](Logging.md) — record types and what they may contain
+137
View File
@@ -0,0 +1,137 @@
# Business rules
How to write, order, and change the rules that decide what an account is.
Field-by-field syntax is in [ConfigurationReference.md](ConfigurationReference.md). This document is
about judgement.
## The model
Rules are evaluated in ascending `priority`. The first rule that returns `True` wins, and evaluation
stops. An account matching no enabled rule is `Unclassified`. An account whose evaluation hits data
that could not be retrieved is `EvaluationError`, and its stored value is preserved.
Three consequences worth internalising before writing a rule:
**Order is meaning.** A rule at priority 900 only ever sees accounts that failed every rule above it.
Changing a priority silently reclassifies every account matched by more than one rule, which is why
`PE-SAF-005` blocks a reorder without a `configVersion` change.
**A rule cannot express "and not the previous ones".** It does not need to. First-match already
excludes them. Adding explicit exclusions duplicates the ordering in two places, and the day they
disagree the ordering wins silently.
**Determinism is absolute.** The same account and the same configuration always produce the same
persona. Nothing time-dependent, random, or order-dependent may enter a decision — `effectiveDate` is
metadata for exactly this reason, and the engine's per-user timing uses a monotonic stopwatch rather
than the wall clock so no clock value can reach a decision.
## Priority bands
A convention, not enforced, but it makes the intent of a rule set legible at a glance:
| Band | Purpose | Examples |
| --- | --- | --- |
| 199 | Accounts that must never be reclassified by anything | Emergency access, Tier 0 |
| 100199 | Directory facts that are definitional | Guest, external |
| 200499 | Non-human accounts | Service, shared functional, room devices |
| 500799 | Population subsets | Contractor, student, restricted |
| 800999 | Defaults | Employee |
Leave gaps. Renumbering to insert a rule is a reorder, and a reorder is a `PE-SAF-005` finding.
## Writing a rule that holds up
**Identify special accounts by Object ID, never by name.** Display names and UPNs change; Object IDs
do not. RE-009 exists because a break-glass account renamed during an incident must not silently stop
being a break-glass account.
**Require two independent signals for a consequential classification.** The example configuration's
service-account rule requires both a naming convention *and* group membership, so a person whose UPN
happens to start with `svc-` is not classified as a service account.
**Prefer group membership to string matching for anything privileged.** A group is administered,
auditable, and has an owner. A naming convention is a habit.
**Give every rule a description that says why it exists**, not what it does — the conditions already
say what it does. A rule nobody can explain cannot be safely changed, which is why `description` is
required.
## Membership mode
`direct` asks whether the account is a member of the named group itself. `transitive` asks whether it
is a member through any chain of nesting.
Mode is a per-condition choice (RE-007). The three facets — direct, transitive, roles — are retrieved
independently, so mixing modes in one rule set is fully supported. It costs one extra request per
account for each additional facet.
Use `transitive` when the group is a role-holding group that other groups nest into — which is most
Tier 0 groups. Use `direct` when membership is explicitly managed and nesting would be a mistake.
**Do not pin `dataSources.groups.membershipMode` unless you mean to restrict.** Absent means "any mode
is acceptable". Pinning it turns every per-condition override into a `PE-SEM-014` warning, which
trains people to ignore warnings.
## Unknown is not false
If a membership lookup fails, the condition is `Unknown`, not `False`. A `notMemberOf` condition
therefore does **not** become satisfied when the lookup fails.
This is the single most important behaviour in the engine. Without it, a transient Graph outage would
make every privileged account look like a non-member of its Tier 0 group, and a single run would
quietly demote the entire administrative population. `UnknownNotFalse.Tests.ps1` exists solely to
prevent that regression.
The cost is that a failed lookup produces `EvaluationError` rather than a classification. That is the
correct trade: preserving a possibly-stale value is recoverable, and writing a confidently wrong one
is not.
## Nesting
`all` and `any` groups nest to `maxConditionDepth` (default 5, ceiling 10). Beyond the limit the
engine returns `Unknown`, which becomes `EvaluationError` for every account the rule reaches — so a
too-deep rule fails safe rather than silently.
A rule needing more than three levels is usually two rules with different priorities. Depth is
expensive to read and the ordering you would express with nesting is already available for free.
## Changing a rule set
1. Edit the configuration.
2. Validate against the deployed copy so the drift checks actually run:
```bash
pwsh ./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -PreviousConfigPath ./deployed/persona-engine.json -ValidateOnly -NonInteractive
```
3. Run the rules against synthetic fixtures.
4. Preview against one account, then against the tenant with `-WhatIf`.
5. Compare the summary's per-rule match counts against the previous run. A rule whose count moved
sharply is either the change you made or a change you did not intend.
6. Raise `configVersion`.
Step 5 is the one people skip. The summary lists every rule including zero-match ones precisely so
that a rule which *stopped* firing is visible, and a rule that stopped firing is the usual signature
of an accidental reorder.
## Disabling versus deleting
Disable rather than delete. A disabled rule still appears in every summary with a zero count, so the
audit trail keeps reporting on it and an operator can see it was deliberately turned off. A deleted
rule is indistinguishable from one that never existed, which is why `PE-SAF-005` flags a deletion
without a version change.
## Testing a rule set without a tenant
```bash
pwsh ./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -TestDataPath ./tests/TestData -ValidateOnly -NonInteractive
```
This runs the real engine against the synthetic fixtures in `tests/TestData/`. The fixtures include
accounts with null and absent properties, mixed casing, a guest, a disabled account, and — most
usefully — two accounts whose membership lookups failed, so `EvaluationError` behaviour is visible
before it happens against a real directory.
Add fixtures for the cases your rule set actually cares about. A fixture that reproduces a real edge
case, sanitized, is worth more than any amount of reasoning about what the engine will probably do.
+217
View File
@@ -0,0 +1,217 @@
# Configuration reference
Every field in `persona-engine.json`, and every finding code the validator can produce.
The authoritative schema is [`config/persona-engine.schema.json`](../config/persona-engine.schema.json)
(JSON Schema draft-07). A working example is
[`config/persona-engine.example.json`](../config/persona-engine.example.json), which is validated by
CI against its own schema — if the example the documentation points at could not pass, every reader's
first run would fail.
## Top level
| Field | Required | Notes |
| --- | --- | --- |
| `configVersion` | yes | Semantic version, `major.minor.patch`. A downgrade is a safety finding. |
| `metadata` | no | `owner`, `changeReference`, `description`. Free-form; not read by the engine. |
| `engine` | yes | Engine behaviour. |
| `dataSources` | yes | Which directory data may be retrieved. |
| `logging` | no | Audit output. |
| `personas` | yes | The declared persona catalogue. |
| `rules` | yes | Ordered business rules. |
## `engine`
| Field | Required | Default | Notes |
| --- | --- | --- | --- |
| `targetAttribute` | yes | — | The single attribute the engine may write. Must be a directory extension property and must appear in `approvedWritableAttributes`. |
| `approvedWritableAttributes` | yes | — | The allow-list. Comparison is **ordinal** — extension property names are case-sensitive in Graph. |
| `maxConditionDepth` | no | `5` | RE-004. Minimum 1, hard ceiling 10. |
| `summaryInterval` | no | `25` | Interim summary frequency. `0` suppresses interim summaries; a final summary always appears. |
| `defaultMembershipMode` | no | `direct` | Mode for membership conditions that do not specify one. |
| `evaluationErrorThreshold` | no | unset | Count of `EvaluationError` results above which the run reports exit code 4. Unset means report, do not fail. |
Setting `evaluationErrorThreshold` to `0` makes a single transient lookup failure fail the run. That
is occasionally what you want; it is rarely what you meant.
## `dataSources`
| Field | Required | Notes |
| --- | --- | --- |
| `groups.enabled` | yes | When false, no membership facet is retrieved. Rules needing it become `EvaluationError`. |
| `groups.membershipMode` | no | Pins a mode globally. **Leave it out unless you mean to restrict** — absent means "any mode is acceptable", and RE-007 makes mode a per-condition choice. Pinning it makes every per-condition override a `PE-SEM-014` warning. |
| `roles.enabled` | yes | Directory role assignments. |
| `roles.includeEligible` | no | PIM-eligible assignments. **Out of scope for v1**; no provider is implemented. |
The engine retrieves only the facets enabled rules actually reference. A configuration with no role
conditions never calls the role endpoint, so a tenant where role reads are unavailable can still run
property-only rules.
## `logging`
| Field | Required | Default | Notes |
| --- | --- | --- | --- |
| `destination` | no | `both` | `file`, `stream`, `both`, or `none`. `stream` writes records to the PowerShell Information stream. |
| `path` | no | — | NDJSON output file. One record per line. |
| `traceConditionValues` | no | `false` | Writes evaluated attribute values into audit records. |
| `acknowledgeConditionTracing` | no | `false` | **Required whenever `traceConditionValues` is true** (VR-003). |
## `personas`
The declared catalogue. A rule assigning a persona absent from this list is a `PE-SEM-010` error — the
catalogue is what stops a typo from writing a new persona value into the directory.
`Unclassified` and `EvaluationError` are processing results and may never be declared or assigned.
## `rules`
| Field | Required | Notes |
| --- | --- | --- |
| `id` | yes | Unique. Appears in every audit record; this is how a decision is traced to its rule. |
| `name` | yes | Human-readable. |
| `description` | yes | Why the rule exists. Required, because a rule nobody can explain cannot be safely changed. |
| `enabled` | yes | Disabled rules are excluded from evaluation but still appear in summaries with zero matches. |
| `priority` | yes | Unique integer. **Lower evaluates first.** |
| `persona` | yes | Must appear in `personas`. |
| `match` | yes | The root condition group. |
| `tags`, `owner`, `changeReference`, `effectiveDate`, `notes`, `testCases` | no | Metadata. `effectiveDate` is **not** evaluated — a date-dependent decision would break determinism. |
Priorities must be unique among enabled rules. The engine breaks ties by rule ID so results stay
deterministic, but the resulting order is an accident rather than a decision, so `PE-SEM-002` blocks it.
## Condition groups and conditions
A group has `operator` (`all` or `any`) and a `conditions` array. Each entry is either another group
or a condition.
| Field | Applies to | Notes |
| --- | --- | --- |
| `type` | all | `property`, `membership`, or `role`. |
| `property` | `property` | One of the supported names below, or an extension property. |
| `operator` | all | See the operator table. |
| `value` | most | Single comparison value. |
| `values` | `in`, `notIn` | Comparison set. |
| `groupObjectIds` | `membership` | Group Object IDs. Names are mutable; IDs are not (RE-009). |
| `roleIds` | `role` | Role **template** IDs, which are stable across tenants. |
| `membershipMode` | `membership` | `direct` or `transitive`, per condition. |
| `caseSensitive` | — | Reserved. Not implemented in v1; the schema accepts the key so a later version needs no breaking change. |
### Supported properties
`AccountObjectId` · `UserPrincipalName` · `DisplayName` · `UserType` · `AccountEnabled` ·
`CompanyName` · `JobTitle` · `Department`
Plus any directory extension property named `extension_<32-hex-app-id>_<name>`. Anything else is
`PE-SEM-015`: unsupported properties are never retrieved, so the condition would compare against a
permanently absent value and quietly never match.
### Operators (RE-005)
| Operator | Applies to | Notes |
| --- | --- | --- |
| `equals`, `notEquals` | property | Case-insensitive (RE-006). |
| `contains`, `notContains` | property | Case-insensitive substring. |
| `startsWith`, `endsWith` | property | Case-insensitive. |
| `matchesRegex` | property | Pattern compiled at validation time. An invalid pattern is `PE-SEM-016`, not a runtime failure. |
| `in`, `notIn` | property | Requires `values`. |
| `isNull`, `isNotNull` | property | Tests presence. **Must not carry a value** — it would be silently ignored (`PE-SEM-009`). |
| `memberOf`, `notMemberOf` | membership, role | Requires `groupObjectIds` or `roleIds`. |
Null and absent properties are treated as empty for ordinary comparisons and never cause an
evaluation failure (FR-012). Intentional null matching uses `isNull` / `isNotNull`.
## Tri-state evaluation
Conditions return `True`, `False`, or `Unknown`. `Unknown` means required data could not be
retrieved, and it propagates:
| Group | Contains | Result |
| --- | --- | --- |
| `all` | any `False` | `False` |
| `all` | only `True` plus at least one `Unknown` | `Unknown` |
| `any` | any `True` | `True` |
| `any` | only `False` plus at least one `Unknown` | `Unknown` |
An `Unknown` at a rule's root makes the account `EvaluationError`: the stored persona is preserved
and no write is attempted (FR-013, FR-014).
## Validation layers
Run in order, stopping at the first that produces `Error` findings. Running semantic checks over a
structurally invalid document yields noise, not signal.
| Layer | Mechanism | Codes |
| --- | --- | --- |
| 1 Syntax | `ConvertFrom-Json` | `PE-SYN-nnn` |
| 2 Schema | `Test-Json -SchemaFile` | `PE-SCH-nnn` |
| 3 Semantic | PowerShell checks | `PE-SEM-nnn` |
| 4 Safety | PowerShell checks | `PE-SAF-nnn` |
Codes are stable. Pipelines and runbooks match on them, so a code is never reused for a different
condition and never renumbered.
### Syntax — `PE-SYN`
| Code | Condition |
| --- | --- |
| `PE-SYN-001` | Configuration file not found, or is not a file |
| `PE-SYN-002` | File exists but could not be read |
| `PE-SYN-003` | File is not valid JSON |
### Schema — `PE-SCH`
| Code | Condition |
| --- | --- |
| `PE-SCH-001` | Document violates the schema |
| `PE-SCH-002` | Schema file not found |
| `PE-SCH-003` | Schema file exists but is not valid JSON Schema |
`PE-SCH-003` exists because of V-5a: `Test-Json` returns `$true` when the schema itself cannot be
parsed. A wrapper trusting the return value would report every configuration as schema-valid against
a schema that never ran.
### Semantic — `PE-SEM` (VR-002)
| Code | Condition | Severity |
| --- | --- | --- |
| `PE-SEM-001` | Duplicate rule ID | Error |
| `PE-SEM-002` | Duplicate priority among enabled rules | Error |
| `PE-SEM-003` | No rules, or no enabled rules | Error |
| `PE-SEM-004` | Blank target attribute | Error |
| `PE-SEM-005` | Target attribute absent from the approved list | Error |
| `PE-SEM-006` | Rule references a disabled data source | Error |
| `PE-SEM-007` | `memberOf` / `notMemberOf` with no group or role IDs | Error |
| `PE-SEM-008` | `in` / `notIn` with no `values` | Error |
| `PE-SEM-009` | `isNull` / `isNotNull` carrying a comparison value | Error |
| `PE-SEM-010` | Persona not in the declared catalogue | Error |
| `PE-SEM-011` | `Unclassified` used as a rule persona | Error |
| `PE-SEM-012` | Nesting deeper than `maxConditionDepth` | Error |
| `PE-SEM-013` | `maxConditionDepth` outside 110 | Error |
| `PE-SEM-014` | Condition mode differs from an explicitly pinned global mode | Warning |
| `PE-SEM-015` | Unsupported property name | Error |
| `PE-SEM-016` | Invalid regular expression | Error |
Several of these are also enforced by the schema. The overlap is deliberate: layer 2 can be bypassed
with `-SchemaPath`, and V-5a showed an unparseable schema passes silently. Anything that can
misclassify a privileged account is checked twice.
### Safety — `PE-SAF` (VR-003)
| Code | Condition | Severity |
| --- | --- | --- |
| `PE-SAF-001` | Blank target attribute | Error enforcing, Warning in preview |
| `PE-SAF-002` | Approved list contains a non-extension attribute | Error |
| `PE-SAF-003` | Enabled rules need data the data sources do not provide | Error |
| `PE-SAF-004` | `configVersion` lower than the deployed version | Error enforcing, Warning in preview |
| `PE-SAF-005` | Rules removed or reordered with no version change | Error enforcing, Warning in preview |
| `PE-SAF-006` | Tracing enabled without acknowledgement | Error |
| `PE-SAF-007` | Save would overwrite an existing configuration with no backup | Error |
`PE-SAF-004` and `PE-SAF-005` need `-PreviousConfigPath`. Without it they are **skipped**, and an
`Information` finding says so — silence would be read as approval.
## Escalation (VR-005)
`Error` blocks execution and saving. `Warning` blocks only under `-TreatWarningsAsErrors`.
`Information` never blocks. Passing `-TreatWarningsAsErrors` does not change a finding's severity;
it changes the caller's tolerance for it.
+136
View File
@@ -0,0 +1,136 @@
# Logging
Structured audit output: what is emitted, where it goes, and what may never appear in it.
The serialized contract is [audit-record.md](../specs/001-persona-engine/contracts/audit-record.md).
This document covers the operational side.
## Format and transport
Newline-delimited JSON (OTD-006). One record per line, UTF-8 without BOM, appended.
Every record passes through a single sink, `Write-PersonaAuditRecord`. Adding a transport — an
approved logging platform, an event hub, a different file layout — is a change to that one function.
If call sites wrote their own output, each new transport would mean auditing every call site again,
and the one that got missed would be silent.
| `logging.destination` | Behaviour |
| --- | --- |
| `file` | Appends NDJSON to `logging.path` |
| `stream` | Emits the record object on the PowerShell **Information** stream |
| `both` | Both |
| `none` | Nothing |
`stream` uses the Information stream rather than the success stream deliberately. Audit records on
the success stream would be indistinguishable from a function's return value — the run loop returns
its outcome there — and mixing the two turns one object into an array of several thousand. Capture
them with `-InformationVariable`, or redirect with `6>`.
### Sink failure never ends a run
A full disk or a locked file is an operational problem with the sink, not a reason to abandon a
classification run mid-population and leave the directory half-reconciled. The failure surfaces as a
warning, **once** per run, and processing continues.
Once, not once per user: a run over five thousand accounts with a locked log file should warn once,
or the warning that matters is buried in the noise it generates.
## Record types
| Type | When | Carries |
| --- | --- | --- |
| `RunStart` | Once, after mode is determined | Config path, target attribute, rule counts, whether tracing is on |
| `UserEvent` | Once per processed account | The full decision |
| `Summary` | Every `summaryInterval` accounts, and once at the end | Counters, per-rule match counts, reconciliation result |
| `RunComplete` | Once, in a `finally` block | Final counters, timing, exit code |
| `EngineDefect` | Reconciliation failure, or threshold breach | What went wrong and by how much |
`RunComplete` is written even on a fatal error. A run that died at account 400 of 5,000 leaves a
record saying exactly that — which is what lets an operator tell "the engine stopped early" from "the
engine never started", two very different incidents that produce identical evidence if the record is
written only on success.
## The common envelope
Every record, every type:
`timestamp` · `recordType` · `runId` · `engineVersion` · `configVersion` · `configurationHash` · `mode`
These appear **first** in each record, so a truncated line still identifies the run that produced it.
`runId` comes from `-CorrelationId` or is generated, and is constant for the run (NFR-005).
`configurationHash` is the SHA-256 of the configuration file bytes — two files differing only in
whitespace are different configurations for audit purposes, and the hash must be reproducible from
the artifact on disk.
## `UserEvent`
100% carry `runId`, `userPrincipalName`, and `accountObjectId` (SC-006). 100% of `Matched` records
carry `matchedRuleId`. `evaluationErrorReason` is non-null exactly when `outcome` is
`EvaluationError`.
`previousValue` is present **only** on `Updated` records, captured at write time. On any other action
there is nothing that was replaced, and a populated `previousValue` would imply otherwise to a
rollback tool reading these records later. Without it, OTD-010 rollback is impossible retroactively —
no future run can reconstruct what a value used to be.
## What may never appear
Access tokens, `Authorization` headers, client secrets, certificates, credentials, and full Graph
responses.
The guarantee is structural rather than filtered. `New-PersonaAuditRecord` accepts only named, typed
values from the decision result and the counters — there is no pass-through of an arbitrary object,
so there is nothing for a secret to ride in on.
[`AuditRedaction.Tests.ps1`](../tests/Unit/AuditRedaction.Tests.ps1) asserts this holds even when a
caller attaches a token to the decision result, and scans every serialized record for JWT and Bearer
shapes.
## Approved for logs
User principal name, account object ID, matched rule ID, stored and calculated persona values, run
ID, configuration version and hash, per-rule match counts, timing.
Runtime records naturally contain real UPNs and Object IDs. **No such value may ever be committed to
this repository** (SC-013) — the sanitization scan enforces that on every build.
## Condition tracing
`conditionTrace` is added to a `UserEvent` only when **both** gates are open: the decision result was
built with tracing, and the record was asked to include it. It carries the per-rule result
(`True` / `False` / `Unknown`) and priority.
Tracing widens what the log contains beyond the approved set, so it requires
`logging.acknowledgeConditionTracing` in the same configuration (`PE-SAF-006`). Tracing never changes
a decision — if it could, a debug run would stop being evidence about the real one, and
[`ConditionTrace.Tests.ps1`](../tests/Unit/ConditionTrace.Tests.ps1) asserts the outcome is identical
with and without it.
## Querying
```powershell
# Everything from one run
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object runId -eq '<RUN-ID>'
# Accounts a run changed, with what it replaced
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object { $_.recordType -eq 'UserEvent' -and $_.action -eq 'Updated' } |
Select-Object userPrincipalName, previousValue, calculatedPersona, matchedRuleId
# Accounts that could not be evaluated, and why
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object outcome -eq 'EvaluationError' |
Select-Object userPrincipalName, evaluationErrorReason
# Which rule set produced a given decision
Get-Content <LOG-OUTPUT-PATH> | ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object recordType -eq 'RunStart' |
Select-Object runId, configVersion, configurationHash, mode
```
## Retention
Not set by this engine. Records contain UPNs and Object IDs, so retention is governed by the
organization's identity-data policy rather than by anything in this repository. Decide it before the
first enforcing run, not after.
+168
View File
@@ -0,0 +1,168 @@
# Operations runbook
What to do when the engine is running, and what to do when it should not be.
## Before any run
1. Validate the configuration. It costs seconds and needs no tenant.
```bash
pwsh ./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -ValidateOnly -NonInteractive
```
2. Run the rules against synthetic fixtures. This shows what the rule set *does* before it sees a
real account.
```bash
pwsh ./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -TestDataPath ./tests/TestData -ValidateOnly -NonInteractive
```
3. Preview a single user before previewing the tenant.
```bash
pwsh ./Invoke-PersonaEngine.ps1 -ConfigPath ./config/persona-engine.json -UserObjectId <ACCOUNT-OBJECT-ID> -WhatIf -Verbose
```
Never skip step 3. A rule set that behaves correctly against fixtures can still request a property
your tenant does not populate, and finding that out on one account is cheaper than on fifty thousand.
## Reading a run
Per-user lines appear immediately, one per account, colour-coded by action:
| Action | Meaning |
| --- | --- |
| `Unchanged` | Calculated value already matches the stored value. Nothing to do. |
| `WouldUpdate` | A change is proposed. Preview mode, or the per-user gate refused. |
| `Updated` | The attribute was written. |
| `UpdateFailed` | The write was attempted and rejected. Stored value is untouched. |
| `Skipped` | No write is possible: `EvaluationError`, or a blank or unapproved target. |
`Skipped` on every account almost always means the target attribute is blank or unapproved — check
the header line and `PE-SAF-001`.
A summary appears every `summaryInterval` accounts and once at the end, listing **every** rule
including disabled and zero-match ones. A rule that never fired and a rule that is not in the
configuration look identical if zero-match rules are omitted, and that distinction is usually what
you are looking for.
## Exit codes
| Code | Meaning | First thing to check |
| --- | --- | --- |
| 0 | Success | — |
| 1 | Configuration validation failed | The findings printed above it; no connection was attempted |
| 2 | Authentication or authorization failed | Scopes, consent, and whether the account can sign in |
| 3 | User enumeration failed | Graph availability. **No accounts were processed** — a partial population is never used |
| 4 | `evaluationErrorThreshold` exceeded | Group or role endpoint health. Nothing was changed |
| 5 | Reconciliation failed | **An engine defect.** Open an issue with the `EngineDefect` record |
| 6 | Unexpected fatal error | The message and `-Verbose` stack trace |
Exit 5 is never a data condition. Outcomes are assigned by the engine, exactly one per account, so if
`Processed` does not equal `Matched + Unclassified + EvaluationError` the engine lost a user or
double-counted one. Report it rather than re-running.
Exit 4 means the population was classified from data that could not be trusted. Stored values were
preserved, so nothing is damaged — but do not draw conclusions from the run.
## Kill switch
In increasing order of severity. Pick the lowest one that addresses the problem.
### 1. Stop writing — immediate, no deployment
Add `-WhatIf` to the invocation. Reads, evaluation, output, and audit records continue unchanged;
zero write requests are constructed.
### 2. Stop classifying — one configuration change
Set `enabled: false` on every rule and deploy.
> Every account becomes `Unclassified`. **In an enforcing run that proposes clearing every stored
> persona.** Combine with `-WhatIf`, or use option 1 instead, unless clearing is what you want.
### 3. Stop running — Stage B only
Disable the Automation schedule. Nothing is in flight; the next run simply does not start.
### 4. Remove the capability — the one that holds if the code is the problem
Revoke `User.ReadWrite.All` from the execution identity. The engine keeps running and every write
becomes `UpdateFailed`, which is loud, logged, and harmless.
### 5. Remove the path — for a suspected defect in the write path
Remove the write deployment stage from the release pipeline so no build can restore write capability
by accident.
Options 1 through 3 rely on the engine behaving correctly. Options 4 and 5 do not, which is why they
exist.
## Rollback (OTD-010)
Every `Updated` audit record carries `previousValue`, captured **before** the PATCH. That is what
makes rollback possible; reading the value back afterwards would return the new one.
To roll back a run:
1. Find the run's records by `runId`.
2. Select records where `recordType` is `UserEvent` and `action` is `Updated`.
3. For each, write `previousValue` back to `accountObjectId`.
```powershell
# Reads the NDJSON audit file and lists what a rollback would restore.
# Review this output before writing anything back.
Get-Content <LOG-OUTPUT-PATH> |
ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object { $_.runId -eq '<RUN-ID>' -and $_.recordType -eq 'UserEvent' -and $_.action -eq 'Updated' } |
Select-Object accountObjectId, userPrincipalName, previousValue, calculatedPersona
```
> A rollback is itself a directory write and is subject to the same V-4 gate and the same test-account
> restriction as any enforcement run. A rollback tool is **not implemented in v1** — the data needed
> to build one is captured, deliberately, because it cannot be reconstructed retroactively.
If a rollback is genuinely intended at the configuration level, publish it as a **new higher
version** rather than reusing the old number. Two different rule sets sharing one `configVersion`
makes the audit trail unable to tell them apart (`PE-SAF-004`).
## Common situations
**Every account is `EvaluationError`.** A required data source is disabled or unreachable. Check
`dataSources.groups.enabled` and `dataSources.roles.enabled` against what the rules need — `PE-SAF-003`
catches this at validation time, so a run reaching this state usually means validation was bypassed.
**Every account is `Unclassified`.** Either every rule is disabled, or no rule matches. The summary
table distinguishes these: disabled rules are dimmed, zero-match enabled rules show `0`.
**The run is slow.** Check the cache hit ratio via `-Verbose`. Membership lookups dominate: one
account needing both direct and transitive facets plus roles is three requests. Narrowing rules to a
single membership mode roughly halves that.
**A write failed with 403.** Non-retryable by design — retrying would hide a configuration or
authorization defect behind a timeout. Check that the identity holds `User.ReadWrite.All` and that
the target attribute exists on the application registration.
**Audit file sink warnings.** The sink warns once per run and processing continues. A locked or full
log file is an operational problem with the sink, not a reason to abandon a run mid-population and
leave the directory half-reconciled.
## Concurrency
Two concurrent enforcing runs against the same tenant would race on the same attributes. Until the
OTD-009 run-start concurrency check is implemented (T120, Stage B), **the schedule is the lock**: do
not start a manual enforcing run while a scheduled one may be in flight.
Preview runs are read-only and safe to run concurrently.
## What to attach to a bug report
- The exit code
- The `RunComplete` record for the run — it carries the counters, timing, and exit code even when the
run died early
- Any `EngineDefect` record
- The `configVersion` and `configurationHash` from any record
- The rule set, sanitized
Do **not** attach raw audit records containing real UPNs or Object IDs to anything that leaves the
organization. They are approved for internal logs, not for public issue trackers.
+169
View File
@@ -0,0 +1,169 @@
# Security model
What this engine is permitted to do, what it is not, and where the gap between those two is held
open by testing rather than by the platform.
## The central trade-off (OTD-003)
**Microsoft Graph application permissions have no per-property write scope.** An identity granted
`User.ReadWrite.All` can write *any* writable property on *any* user object. It cannot be narrowed to
one extension attribute.
This is not a limitation to be worked around. It is a fact about the platform, recorded here so that
nobody later assumes the directory is enforcing something it is not.
The consequence: **the only thing standing between this engine and every writable user property is
the code in this repository, and the tests that hold it to that.** Every control below exists because
the directory will not refuse a malformed request on our behalf.
## The six compensating controls
All six are mandatory. Each is testable, and each is tested.
| # | Control | Where it lives | Proof |
| --- | --- | --- | --- |
| 1 | The persistence layer accepts only the configured target attribute | `New-PersonaWriteBody` throws for any other name | `WriteBodyRejection.Tests.ps1` |
| 2 | The target must appear in `approvedWritableAttributes` | `Resolve-TargetAttribute` and `New-PersonaWriteBody`, checked twice | `WriteBodyRejection.Tests.ps1` |
| 3 | Validation rejects every other attribute | `PE-SAF-002`, layer 4 | `Safety.Tests.ps1` |
| 4 | One dedicated function builds the request body, and it is the only one | `New-PersonaWriteBody` returns a hashtable whose `Count` is exactly 1 | `WriteBody.Tests.ps1` |
| 5 | Tests inspect the captured request body | Every body issued during a full enforcing run is asserted to have one key | `WriteBody.Tests.ps1` |
| 6 | Code owners and branch policies gate persistence changes | Repository configuration, outside this codebase | Branch protection on `src/Persistence/` |
Control 4 is the load-bearing one. A single construction site makes SC-005 a property of one testable
function rather than a convention every future call site has to remember. `WriteBody.Tests.ps1`
includes a scan asserting that no other file under `src/` builds a PATCH body.
Control 2 is deliberately redundant. Validation runs once at startup against the file; the write
builder checks again on every write against the values actually in hand — so a configuration object
mutated mid-run still cannot widen the blast radius.
### Why comparison is ordinal here and case-insensitive elsewhere
Rule matching is case-insensitive (RE-006), because a rule author should not have to match directory
casing. Attribute approval is **ordinal and case-sensitive**, because extension property names are
case-sensitive in Graph: `extension_<id>_Persona` and `extension_<id>_persona` are two different
attributes, and approving one does not approve the other.
Change detection is also ordinal (FR-015). A stored `employee` against a calculated `Employee` is a
real difference worth correcting, not a formatting quirk.
## Permissions
### Stage A — local, delegated (current)
```powershell
Connect-MgGraph -Scopes 'User.Read.All','GroupMember.Read.All','RoleManagement.Read.Directory'
```
Read-only. Sufficient for every preview run and for closing V-1 (read) and V-3.
The engine requests only the scopes the enabled rules actually need: a configuration with no role
conditions never asks for `RoleManagement.Read.Directory`, and a configuration with no membership
conditions never asks for `GroupMember.Read.All`. Least privilege applies to data as well as to
permissions — properties nothing references are not even added to `$select`.
**Stage A3 (delegated write) adds `User.ReadWrite.All` and targets purpose-created test accounts
only.** Under delegated authentication the write runs *as the operator*, which makes the compensating
controls more important rather than less: the directory sees the operator's own permissions, not a
narrowed service identity.
> **Never sign in with a standing privileged account for a write run.** A Global Administrator
> session invalidates V-3 as evidence and removes every practical limit on what a defect could reach.
### Stage B — Azure Automation, application permissions (deferred)
`User.Read.All`, `GroupMember.Read.All`, `RoleManagement.Read.Directory`, and — for enforcement —
`User.ReadWrite.All`, granted to a **managed identity**. No client secret, ever, in source control or
in a runbook parameter.
Deferred, not waived: no Automation account is available. V-3b and V-5b remain open.
## The persona attribute (OTD-001)
A **directory (schema) extension property**, registered on an application registration and addressable
as `extension_<appId>_<name>`.
Two alternatives were rejected for concrete reasons:
- **`extensionAttribute1..15`** — unavailable for cloud writes on objects that are, or ever were,
synchronized from on-premises, and on Exchange-originated objects. A classification engine that
silently cannot write to a subset of the population is worse than one that cannot write at all.
- **Custom security attributes** — not exposed to the dynamic group membership engine, which defeats
the purpose: the persona exists so that Conditional Access can be targeted through dynamic groups.
`PE-SAF-002` rejects any approved attribute that is not shaped like a directory extension property.
Built-in attributes such as `department` or `jobTitle` are excluded **even when the operator holds
permission to write them** — they are authoritative in the sync source or in HR, and this engine does
not own them.
## Data handling
**Approved for logs**: user principal name, account object ID, matched rule ID, stored and calculated
persona values, run ID, configuration version and hash.
**Never logged, under any setting**: access tokens, `Authorization` headers, client secrets,
certificates, credentials, or full Graph responses.
The guarantee is structural rather than filtered. `New-PersonaAuditRecord` accepts only named, typed
values from the decision result and the counters — there is no pass-through of an arbitrary object,
so there is nothing for a secret to ride in on. `AuditRedaction.Tests.ps1` asserts this holds even
when a caller actively attaches a token to the decision result.
### Condition tracing
`logging.traceConditionValues` writes evaluated attribute values into audit records, widening what
the log contains beyond the approved set. It requires `logging.acknowledgeConditionTracing` in the
same configuration, or validation fails with `PE-SAF-006`.
The acknowledgement lives in the configuration rather than in a command-line switch on purpose: a
flag passed at a console is invisible to review, while a field in the configuration appears in the
diff of the change that enables tracing, next to the person who approved it.
## The no-write control
`-WhatIf` is the only approved no-write control.
`-Debug` does **not** imply read-only. A `-Debug` run without `-WhatIf` writes, and
`ShouldProcessGate.Tests.ps1` asserts that it does — because an operator who believed otherwise would
reach for `-Debug` as a safety measure and get an enforcing run. The same holds for `-Verbose`.
`ShouldProcessGate.Tests.ps1` also asserts that the entry script declares no `-Preview`, `-NoWrite`,
`-ReadOnly`, or `-DryRun` parameter, and never reads `$WhatIfPreference`.
## Verification gates
| Item | Status | Blocks |
| --- | --- | --- |
| V-1 read half | Open — needs a tenant | Confidence in the read path across origin types |
| V-1 write half | Open — needs test accounts | Enforcement |
| V-2 dynamic group + CA | Open | Declaring the persona useful |
| V-3 non-privileged `-WhatIf` run | Open — needs a tenant | Stage A2 sign-off |
| V-3b managed-identity scopes | Deferred | Stage B |
| **V-4 security sign-off on these controls** | **Open** | **All enforcement (T101)** |
| V-5a `Test-Json` behaviour | **Closed** — see [V-5a.md](../specs/001-persona-engine/verification/V-5a.md) | Layer 2 implementation |
| V-5b `Test-Json` in Automation | Deferred | Stage B |
**V-4 gates enforcement.** No write run against anything other than purpose-created test accounts
until it is recorded in `specs/001-persona-engine/verification/V-4.md`.
## Kill switch
In increasing order of severity — see [OperationsRunbook.md](OperationsRunbook.md) for the procedure:
1. Run with `-WhatIf`.
2. Set every rule to `enabled: false` and deploy.
3. Disable the Automation schedule (Stage B).
4. Revoke `User.ReadWrite.All` from the execution identity.
5. Remove the write deployment stage.
Steps 4 and 5 are the ones that hold if the code itself is the problem.
## Sanitization (SC-013)
No organization name, real domain, tenant or subscription ID, real UPN or Object ID, real group or
role identifier, environment-specific attribute name, or any secret may appear in any tracked file.
Placeholders only.
[`tests/Test-Sanitization.ps1`](../tests/Test-Sanitization.ps1) scans every tracked file on every
build. Runtime records naturally contain real UPNs and Object IDs — approved for logs — but no such
value is ever committed.