Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
VR-001: the four layers run in order and stop at the first that produces errors.
|
||||
|
||||
The reason is signal, not speed. A document missing a required section produces a
|
||||
cascade of consequent semantic errors, and the author then has to guess which one
|
||||
is the cause. Stopping at the structural failure reports the one thing that is
|
||||
actually wrong.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:schema = Join-Path $repoRoot 'config/persona-engine.schema.json'
|
||||
$script:scratch = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-layers-{0}" -f [guid]::NewGuid().ToString('N'))
|
||||
$null = New-Item -ItemType Directory -Path $script:scratch -Force
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Layer ordering and fail-fast (VR-001)' {
|
||||
|
||||
It 'stops at layer 1 for malformed JSON' {
|
||||
$path = Join-Path $script:scratch 'malformed.json'
|
||||
Set-Content -LiteralPath $path -Value '{ "configVersion": "1.0.0", ' -Encoding utf8NoBOM
|
||||
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.StoppedAtLayer | Should -Be 'Syntax'
|
||||
$result.Findings.Code | Should -Contain 'PE-SYN-003'
|
||||
}
|
||||
|
||||
It 'reports a missing file at layer 1 without attempting to parse it' {
|
||||
$result = Test-PersonaConfiguration -Path (Join-Path $script:scratch 'absent.json') -SchemaPath $script:schema
|
||||
|
||||
$result.StoppedAtLayer | Should -Be 'Syntax'
|
||||
$result.Findings.Code | Should -Contain 'PE-SYN-001'
|
||||
}
|
||||
|
||||
It 'stops at layer 2 for a structurally invalid document, before semantic checks run' {
|
||||
# The document below also has a semantic defect - a duplicate rule ID - which
|
||||
# must NOT appear, because layer 3 never ran.
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].id = $document.rules[0].id
|
||||
$document.engine.Remove('approvedWritableAttributes')
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.StoppedAtLayer | Should -Be 'Schema'
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-001'
|
||||
$result.Findings.Code | Should -Not -Contain 'PE-SEM-001'
|
||||
}
|
||||
|
||||
It 'stops at layer 3 for a semantic error, before safety checks run' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].id = $document.rules[0].id
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.StoppedAtLayer | Should -Be 'Semantic'
|
||||
$result.Findings.Code | Should -Contain 'PE-SEM-001'
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'reaches layer 4 when layers 1 to 3 are clean' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.StoppedAtLayer | Should -BeNullOrEmpty
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -Not -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'skips layer 4 on request without claiming it passed' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema -SkipSafety
|
||||
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -BeNullOrEmpty
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'A schema that cannot be used is never reported as a pass (V-5a)' {
|
||||
|
||||
It 'flags an unparseable schema rather than trusting the $true return value' {
|
||||
# Test-Json returns $true here. Trusting it would validate every configuration
|
||||
# against a schema that never ran.
|
||||
$brokenSchema = Join-Path $script:scratch 'broken-schema.json'
|
||||
Set-Content -LiteralPath $brokenSchema -Value '{ not json' -Encoding utf8NoBOM
|
||||
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $brokenSchema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.SchemaUnusable | Should -BeTrue
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-003'
|
||||
}
|
||||
|
||||
It 'flags a missing schema file separately from an invalid configuration' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath (Join-Path $script:scratch 'no-such-schema.json')
|
||||
|
||||
$result.SchemaUnusable | Should -BeTrue
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-002'
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'The shipped example configuration passes every layer' {
|
||||
|
||||
It 'validates cleanly against the shipped schema' {
|
||||
# If the example the documentation points at cannot pass its own validator,
|
||||
# every reader's first run fails.
|
||||
$result = Test-PersonaConfiguration -Path (Join-Path $repoRoot 'config/persona-engine.example.json')
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.ErrorCount | Should -Be 0
|
||||
$result.WarningCount | Should -Be 0
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user