Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,191 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
SC-011: every engine exit code 0-6 is reachable and returned for its documented
|
||||
condition.
|
||||
|
||||
An exit code is what a scheduler, a pipeline, or an on-call runbook reacts to. A
|
||||
code that cannot be produced is a promise the contract makes and the engine does
|
||||
not keep, and it is invisible until the day someone builds an alert on it.
|
||||
|
||||
Codes 0, 3, 4, and 5 are reachable inside the run loop and are tested by fault
|
||||
injection here. Codes 1, 2, and 6 are owned by the entry script, which decides
|
||||
them before or around the loop; those are asserted against the script's own
|
||||
control flow.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:target = 'extension_<EXTENSION-APP-ID>_<PERSONA>'
|
||||
$script:entryScript = Join-Path $repoRoot 'Invoke-PersonaEngine.ps1'
|
||||
}
|
||||
|
||||
Describe 'Exit codes produced by the run loop' {
|
||||
|
||||
BeforeEach {
|
||||
Mock Write-Host { }
|
||||
Mock Write-Verbose { }
|
||||
}
|
||||
|
||||
It 'returns 0 for a clean run' {
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 10 -TargetAttribute $script:target) }
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration (New-TestRuntimeConfiguration -TargetAttribute $script:target) `
|
||||
-TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'returns 3 when user enumeration fails' {
|
||||
Mock Get-PersonaUsers { throw 'User enumeration returned an unexpected response shape on page 4.' }
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration (New-TestRuntimeConfiguration -TargetAttribute $script:target) `
|
||||
-TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.ExitCode | Should -Be 3
|
||||
$outcome.FailureReason | Should -Not -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'processes nobody when enumeration fails, rather than reporting a partial run' {
|
||||
Mock Get-PersonaUsers { throw 'truncated' }
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration (New-TestRuntimeConfiguration -TargetAttribute $script:target) `
|
||||
-TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.Counters.Processed | Should -Be 0
|
||||
}
|
||||
|
||||
It 'returns 4 when the EvaluationError count exceeds the configured threshold' {
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 10 -TargetAttribute $script:target) }
|
||||
Mock Get-PersonaGroupMembership { New-PersonaMembershipRecord -DirectFailureReason 'Graph 503 after 5 attempts' }
|
||||
|
||||
$config = New-TestRuntimeConfiguration -TargetAttribute $script:target -EvaluationErrorThreshold 3 -Rules @(
|
||||
(New-TestMembershipRule)
|
||||
[pscustomobject]@{
|
||||
id = 'RULE-0900-EMPLOYEE'; name = 'Employees'; enabled = $true; priority = 900; persona = 'Employee'
|
||||
match = [pscustomobject]@{ operator = 'all'; conditions = @([pscustomobject]@{ type = 'property'; property = 'Department'; operator = 'isNotNull' }) }
|
||||
})
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.Counters.EvaluationError | Should -BeGreaterThan 3
|
||||
$outcome.ExitCode | Should -Be 4
|
||||
}
|
||||
|
||||
It 'returns 0 when the EvaluationError count is at or below the threshold' {
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 10 -TargetAttribute $script:target) }
|
||||
Mock Get-PersonaGroupMembership { New-PersonaMembershipRecord -DirectFailureReason 'Graph 503' }
|
||||
|
||||
$config = New-TestRuntimeConfiguration -TargetAttribute $script:target -EvaluationErrorThreshold 100 -Rules @(
|
||||
(New-TestMembershipRule)
|
||||
[pscustomobject]@{
|
||||
id = 'RULE-0900-EMPLOYEE'; name = 'Employees'; enabled = $true; priority = 900; persona = 'Employee'
|
||||
match = [pscustomobject]@{ operator = 'all'; conditions = @([pscustomobject]@{ type = 'property'; property = 'Department'; operator = 'isNotNull' }) }
|
||||
})
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.Counters.EvaluationError | Should -BeGreaterThan 0
|
||||
$outcome.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'leaves the threshold inactive when it is not configured' {
|
||||
# An unset threshold means "report, do not fail". Defaulting it to 0 would turn
|
||||
# a single transient lookup failure into a failed run.
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 10 -TargetAttribute $script:target) }
|
||||
Mock Get-PersonaGroupMembership { New-PersonaMembershipRecord -DirectFailureReason 'Graph 503' }
|
||||
|
||||
$config = New-TestRuntimeConfiguration -TargetAttribute $script:target -Rules @(
|
||||
(New-TestMembershipRule)
|
||||
[pscustomobject]@{
|
||||
id = 'RULE-0900-EMPLOYEE'; name = 'Employees'; enabled = $true; priority = 900; persona = 'Employee'
|
||||
match = [pscustomobject]@{ operator = 'all'; conditions = @([pscustomobject]@{ type = 'property'; property = 'Department'; operator = 'isNotNull' }) }
|
||||
})
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.Counters.EvaluationError | Should -BeGreaterThan 0
|
||||
$outcome.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'returns 5 when reconciliation fails' {
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 6 -TargetAttribute $script:target) }
|
||||
Mock Add-PersonaRunResult { $Counters.Processed++ }
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration (New-TestRuntimeConfiguration -TargetAttribute $script:target) `
|
||||
-TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.ExitCode | Should -Be 5
|
||||
}
|
||||
|
||||
It 'prefers 5 over 4 when both conditions hold' {
|
||||
# A reconciliation failure means the counters cannot be trusted, so the
|
||||
# threshold check is reading numbers that may be wrong. The engine defect is
|
||||
# the more serious and more actionable report.
|
||||
Mock Get-PersonaUsers { @(New-TestPopulation -Count 6 -TargetAttribute $script:target) }
|
||||
Mock Get-PersonaGroupMembership { New-PersonaMembershipRecord -DirectFailureReason 'Graph 503' }
|
||||
# Processed advances twice per user while only one outcome is tallied: the
|
||||
# buckets no longer add up AND the error count clears the threshold, so both
|
||||
# conditions are genuinely true at once.
|
||||
Mock Add-PersonaRunResult { $Counters.Processed += 2; $Counters.EvaluationError++ }
|
||||
|
||||
$config = New-TestRuntimeConfiguration -TargetAttribute $script:target -EvaluationErrorThreshold 1 -Rules @(
|
||||
(New-TestMembershipRule)
|
||||
)
|
||||
|
||||
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $script:target -Context (New-TestAuditContext)
|
||||
|
||||
$outcome.ExitCode | Should -Be 5
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Exit codes owned by the entry script' {
|
||||
|
||||
BeforeAll {
|
||||
$script:entryText = Get-Content -LiteralPath $script:entryScript -Raw
|
||||
}
|
||||
|
||||
It 'declares all seven documented codes' {
|
||||
foreach ($name in @('EXIT_OK', 'EXIT_CONFIG', 'EXIT_AUTH', 'EXIT_ENUMERATION',
|
||||
'EXIT_DATA', 'EXIT_RECONCILIATION', 'EXIT_UNEXPECTED')) {
|
||||
$script:entryText | Should -Match "\`$$name\s*=\s*\d"
|
||||
}
|
||||
}
|
||||
|
||||
It 'assigns each code the value the contract documents' {
|
||||
$expected = @{
|
||||
'EXIT_OK' = 0; 'EXIT_CONFIG' = 1; 'EXIT_AUTH' = 2; 'EXIT_ENUMERATION' = 3
|
||||
'EXIT_DATA' = 4; 'EXIT_RECONCILIATION' = 5; 'EXIT_UNEXPECTED' = 6
|
||||
}
|
||||
|
||||
foreach ($name in $expected.Keys) {
|
||||
$script:entryText | Should -Match "\`$$name\s*=\s*$($expected[$name])\b"
|
||||
}
|
||||
}
|
||||
|
||||
It 'returns exit code 1 without connecting when validation fails' {
|
||||
# The ordering matters as much as the code: FR-002 requires validation to
|
||||
# complete before any connection is attempted.
|
||||
$configIndex = $script:entryText.IndexOf('exit $EXIT_CONFIG')
|
||||
$connectIndex = $script:entryText.IndexOf('Connect-PersonaGraphInteractive')
|
||||
|
||||
$configIndex | Should -BeGreaterThan 0
|
||||
$connectIndex | Should -BeGreaterThan $configIndex
|
||||
}
|
||||
|
||||
It 'maps an authentication failure to code 2' {
|
||||
$script:entryText | Should -Match '(?s)Connect-PersonaGraphInteractive.*?\$exitCode = \$EXIT_AUTH'
|
||||
}
|
||||
|
||||
It 'falls back to code 6 for an unexpected error' {
|
||||
$script:entryText | Should -Match '\$exitCode = \$EXIT_UNEXPECTED'
|
||||
}
|
||||
|
||||
It 'writes a RunComplete record even on a fatal error' {
|
||||
# A run that died at user 400 of 5000 has to leave evidence saying so.
|
||||
$script:entryText | Should -Match '(?s)finally\s*\{.*Export-PersonaRunReport'
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user