dave/personaEngine2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement Stage A: rule engine, validation, audit, and safety gates

Browse Source 
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).

  354 offline Pester tests      PASS
  Engine purity (Principle IV)  PASS
  Sanitization (SC-013)         PASS  (156 files)
  Graph module loaded in tests  none  (SC-008 holds)

What landed
  - Four-layer configuration validation with stable finding codes, covering
    every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
  - Run loop, audit records (NDJSON through a single sink), summaries,
    reconciliation, and exit codes 0-6
  - Persistence behind a single write-body builder whose result always has
    exactly one key
  - Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
  - Six docs, two pipelines, traceability matrix, V-5a and sanitization records

Three deviations from tasks.md, each recorded in its status block

  T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
  state and the rule engine is pure; a counter there would break Principle IV.
  It lives in New-PersonaRunCounter and is applied in the run loop.

  A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
  imports the manifest, which requires Microsoft.Graph.Authentication, so a
  loop living only inside it could not run on a machine without the Graph SDK
  and SC-004 could not be proven at all. The entry script is now a thin
  wrapper and what ships is what is tested.

  The invalid-config corpus is generated by a committed script, with the
  generated fixtures committed too, so a reviewer sees the fixture in the diff.

Defects found by running the code, not by reading it

  Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
  around a comma-returned array collapsed every membership list into one
  bogus space-joined entry. That is a silent false non-match, exactly what
  FR-013 exists to prevent.

  A 403 whose status appears only in the exception message parsed as $null,
  which the retry policy treats as a transport error - five requests per
  account against a tenant already refusing. Status extraction now falls back
  to the message text, bounded to 400-599.

  The sanitization scan walked tracked files only, so it covered 34 of 156
  files and none of this phase's code. It now scans untracked non-ignored
  files too, and a negative control confirms it catches a planted leak.

  Test-Json reports one error per violating location, not first-failure-only
  as the V-5a draft claimed. Record and pin corrected.

Enforcement remains blocked on the V-4 security sign-off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dave
2026-08-20 21:48:19 -04:00
parent c59c85dd55
commit cdc6bb33d3
124 changed files with 16638 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/Safety/WhatIfZeroWrites.Tests.ps1
+133
View File
@@ -0,0 +1,133 @@
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
<#
SC-004: a preview run issues zero write requests across a full population.
The assertion that matters is the call count on the write adapter, not the
absence of an error. A run that never reached the write path because it crashed
at user 3 would also record zero writes, so every test here checks the population
was fully processed as well.
#>
BeforeAll {
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
$script:target = 'extension_<EXTENSION-APP-ID>_<PERSONA>'
$script:config = [pscustomobject]@{
ConfigVersion = '1.0.0'
ConfigurationHash = ('0' * 64)
TargetAttribute = $target
ApprovedWritableAttributes = @($target)
MaxConditionDepth = 5
SummaryInterval = 0
DefaultMembershipMode = 'Direct'
EvaluationErrorThreshold = $null
Rules = @(
[pscustomobject]@{
id = 'RULE-0010-GUEST'; name = 'Guests'; enabled = $true; priority = 10; persona = 'Guest'
match = [pscustomobject]@{
operator = 'all'
conditions = @([pscustomobject]@{ type = 'property'; property = 'UserType'; operator = 'equals'; value = 'Guest' })
}
}
[pscustomobject]@{
id = 'RULE-0900-EMPLOYEE'; name = 'Employees'; enabled = $true; priority = 900; persona = 'Employee'
match = [pscustomobject]@{
operator = 'all'
conditions = @([pscustomobject]@{ type = 'property'; property = 'Department'; operator = 'isNotNull' })
}
}
)
}
$script:population = @(New-TestPopulation -Count 30 -TargetAttribute $target)
}
Describe 'Zero writes in preview mode (SC-004, FR-017)' -Tag 'Safety' {
BeforeEach {
Mock Get-PersonaUsers { $script:population }
Mock Set-UserPersonaAttribute { throw 'The write adapter must never be reached in preview mode.' }
Mock Write-Host { }
}
It 'issues no write request across the full population' {
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Preview') `
-ShouldProcessGate { param($t, $d) $false }
Should -Invoke Set-UserPersonaAttribute -Times 0 -Exactly
}
It 'still processed every account, so the zero count is meaningful' {
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Preview') `
-ShouldProcessGate { param($t, $d) $false }
$outcome.Counters.Processed | Should -Be 30
}
It 'reports the intended changes as WouldUpdate rather than hiding them' {
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Preview') `
-ShouldProcessGate { param($t, $d) $false }
# Ten Guest accounts carry a stale stored value of 'Employee'.
$outcome.Counters.WouldUpdate | Should -BeGreaterThan 0
}
It 'returns exit code 0 - a preview that changes nothing is a successful run' {
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Preview') `
-ShouldProcessGate { param($t, $d) $false }
$outcome.ExitCode | Should -Be 0
}
It 'defaults to refusing writes when no gate is supplied' {
# A caller that forgets the gate must preview, not write. The default is the
# safe answer rather than the convenient one.
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Preview')
Should -Invoke Set-UserPersonaAttribute -Times 0 -Exactly
$outcome.Counters.Updated | Should -Be 0
}
}
Describe 'Writes do occur when the gate allows them' -Tag 'Safety' {
BeforeEach {
Mock Get-PersonaUsers { $script:population }
Mock Write-Host { }
Mock Set-UserPersonaAttribute {
[pscustomobject]@{ Succeeded = $true; AccountObjectId = $UserObjectId; Value = $Value; PreviousValue = $PreviousValue }
}
}
It 'writes exactly the accounts whose calculated value differs' {
# The counterpart to the zero-write test. Without this, a run loop that never
# writes under any circumstances would pass every assertion above.
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Enforce') `
-IsEnforcing `
-ShouldProcessGate { param($t, $d) $true }
$outcome.Counters.Updated | Should -BeGreaterThan 0
Should -Invoke Set-UserPersonaAttribute -Times $outcome.Counters.Updated -Exactly
}
It 'never writes an account whose stored value already matches' {
$outcome = Invoke-PersonaEngineRun -Configuration $config -TargetAttribute $target `
-Context (New-TestAuditContext -Mode 'Enforce') `
-IsEnforcing `
-ShouldProcessGate { param($t, $d) $true }
$outcome.Counters.Unchanged | Should -BeGreaterThan 0
($outcome.Counters.Updated + $outcome.Counters.Unchanged + $outcome.Counters.Skipped + $outcome.Counters.UpdateFailed) |
Should -Be $outcome.Counters.Processed
}
}