Files

T

dave cdc6bb33d3 Implement Stage A: rule engine, validation, audit, and safety gates

Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).

  354 offline Pester tests      PASS
  Engine purity (Principle IV)  PASS
  Sanitization (SC-013)         PASS  (156 files)
  Graph module loaded in tests  none  (SC-008 holds)

What landed
  - Four-layer configuration validation with stable finding codes, covering
    every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
  - Run loop, audit records (NDJSON through a single sink), summaries,
    reconciliation, and exit codes 0-6
  - Persistence behind a single write-body builder whose result always has
    exactly one key
  - Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
  - Six docs, two pipelines, traceability matrix, V-5a and sanitization records

Three deviations from tasks.md, each recorded in its status block

  T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
  state and the rule engine is pure; a counter there would break Principle IV.
  It lives in New-PersonaRunCounter and is applied in the run loop.

  A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
  imports the manifest, which requires Microsoft.Graph.Authentication, so a
  loop living only inside it could not run on a machine without the Graph SDK
  and SC-004 could not be proven at all. The entry script is now a thin
  wrapper and what ships is what is tested.

  The invalid-config corpus is generated by a committed script, with the
  generated fixtures committed too, so a reviewer sees the fixture in the diff.

Defects found by running the code, not by reading it

  Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
  around a comma-returned array collapsed every membership list into one
  bogus space-joined entry. That is a silent false non-match, exactly what
  FR-013 exists to prevent.

  A 403 whose status appears only in the exception message parsed as $null,
  which the retry policy treats as a transport error - five requests per
  account against a tenant already refusing. Status extraction now falls back
  to the message text, bounded to 400-599.

  The sanitization scan walked tracked files only, so it covered 34 of 156
  files and none of this phase's code. It now scans untracked non-ignored
  files too, and a negative control confirms it catches a planted leak.

  Test-Json reports one error per violating location, not first-failure-only
  as the V-5a draft claimed. Record and pin corrected.

Enforcement remains blocked on the V-4 security sign-off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-20 21:48:19 -04:00

3.5 KiB

Raw Blame History

V-5a — `Test-Json -SchemaFile` failure behaviour (local PowerShell)

Status: CLOSED Date: 2026-08-20 Environment: PowerShell 7.6.5, Windows 11, local workstation (Stage A1 — offline, no tenant) Task: T061 Pinned by: tests/Configuration/TestJsonBehaviour.Tests.ps1

Question

OTD-005 selected Test-Json -SchemaFile for layer 2 validation. The documented risk was that Test-Json reports schema failure inconsistently across PowerShell versions — returning $false, writing a non-terminating error, or throwing. Layer 2 cannot be written until the actual behaviour on the target build is observed rather than assumed.

Observed behaviour

Scenario	Return value	Error stream	Terminating?
Valid document	`$true`	empty	no
Type mismatch (`"a": 123` against `"type": "string"`)	`$false`	1 error: `The JSON is not valid with the schema: Value is "integer" but should be "string" at '/a'`	no
Missing required property	`$false`	1 error: `The JSON is not valid with the schema: Required properties ["a"] are not present at ''`	no
Schema file itself unparseable	`$true`	1 error: `Cannot parse the JSON schema.`	no

Exception type on the error record is System.Exception in every failing case — there is no distinct exception type to branch on, so the wrapper must branch on the message text or, better, on error presence alone.

Findings that shape the implementation

Non-terminating, not throwing. With the default $ErrorActionPreference = 'Continue' the cmdlet writes to the error stream and execution continues, returning $false. It does not throw. -ErrorAction SilentlyContinue -ErrorVariable is therefore sufficient to capture failures, as the editor contract requires.
An unparseable schema returns $true. This is the load-bearing observation. A wrapper that trusted the return value alone would report a configuration as schema-valid when the schema never ran. Layer 2 MUST treat "error variable is non-empty" as failure regardless of the return value, and MUST distinguish the Cannot parse the JSON schema. message so it can surface exit code 4 (schema file not found or itself invalid) rather than exit code 1 (configuration invalid).
One error per violating location, but not exhaustive. Two independent property violations yield two error records, each with its own JSON pointer. Deeper or nested subschema failures may still be reported as a single error at the outermost failing location. The wrapper therefore emits one finding per collected error rather than assuming a single one, and the author may still need more than one validation pass to see everything. That residual limitation is documented rather than worked around — full violation reporting would require replacing Test-Json with a third-party validator, which OTD-005 rejected.

Consequences recorded elsewhere

Layer 2 implementation: src/Configuration/Test-PersonaConfiguration.ps1
Regression pin: tests/Configuration/TestJsonBehaviour.Tests.ps1 fails if a future PowerShell build changes any row of the table above.
V-5b remains open: this observation is for PowerShell 7.6.5 only. The Azure Automation runtime version is unverified, and finding 2 in particular is version-sensitive. Re-run this probe there before Stage B (T116).

3.5 KiB Raw Blame History

V-5a — Test-Json -SchemaFile failure behaviour (local PowerShell)