Files
personaEngine2/tests/Test-EnginePurity.ps1
T
dave cdc6bb33d3 Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).

  354 offline Pester tests      PASS
  Engine purity (Principle IV)  PASS
  Sanitization (SC-013)         PASS  (156 files)
  Graph module loaded in tests  none  (SC-008 holds)

What landed
  - Four-layer configuration validation with stable finding codes, covering
    every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
  - Run loop, audit records (NDJSON through a single sink), summaries,
    reconciliation, and exit codes 0-6
  - Persistence behind a single write-body builder whose result always has
    exactly one key
  - Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
  - Six docs, two pipelines, traceability matrix, V-5a and sanitization records

Three deviations from tasks.md, each recorded in its status block

  T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
  state and the rule engine is pure; a counter there would break Principle IV.
  It lives in New-PersonaRunCounter and is applied in the run loop.

  A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
  imports the manifest, which requires Microsoft.Graph.Authentication, so a
  loop living only inside it could not run on a machine without the Graph SDK
  and SC-004 could not be proven at all. The entry script is now a thin
  wrapper and what ships is what is tested.

  The invalid-config corpus is generated by a committed script, with the
  generated fixtures committed too, so a reviewer sees the fixture in the diff.

Defects found by running the code, not by reading it

  Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
  around a comma-returned array collapsed every membership list into one
  bogus space-joined entry. That is a silent false non-match, exactly what
  FR-013 exists to prevent.

  A 403 whose status appears only in the exception message parsed as $null,
  which the retry policy treats as a transport error - five requests per
  account against a tenant already refusing. Status extraction now falls back
  to the message text, bounded to 400-599.

  The sanitization scan walked tracked files only, so it covered 34 of 156
  files and none of this phase's code. It now scans untracked non-ignored
  files too, and a negative control confirms it catches a planted leak.

  Test-Json reports one error per violating location, not first-failure-only
  as the V-5a draft claimed. Record and pin corrected.

Enforcement remains blocked on the V-4 security sign-off.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 21:48:19 -04:00

103 lines
4.0 KiB
PowerShell

<#
.SYNOPSIS
Fails if the rule engine has acquired a dependency it must not have.
.DESCRIPTION
Constitution Principle IV: the pure rule engine must not depend on Microsoft
Graph, authentication, Azure Automation, or console rendering. Directory
structure alone does not enforce that — one convenient call is all it takes to
make the engine untestable offline, and the failure is silent until someone
tries to run the tests without a tenant.
This check is the enforcement. It runs in CI (pipelines/validate.yml) and is
cheap enough to run locally on every change.
.EXAMPLE
./tests/Test-EnginePurity.ps1
#>
[CmdletBinding()]
param(
[string] $EnginePath = (Join-Path (Split-Path $PSScriptRoot -Parent) 'src/RuleEngine'),
[switch] $PassThru
)
$ErrorActionPreference = 'Stop'
$forbidden = @(
@{ Name = 'Microsoft Graph call'; Regex = 'Invoke-MgGraphRequest|Connect-MgGraph|graph\.microsoft\.com|Invoke-PersonaGraphRequest' }
@{ Name = 'Authentication layer'; Regex = 'Connect-Persona\w+' }
@{ Name = 'Data provider layer'; Regex = 'Get-Persona(Users|GroupMembership|DirectoryRoles)' }
@{ Name = 'Persistence layer'; Regex = 'Set-UserPersonaAttribute|New-PersonaWriteBody|Compare-PersonaValue' }
@{ Name = 'Console rendering'; Regex = 'Write-Host|Write-UserPersonaResult|Write-PersonaSummary' }
@{ Name = 'Direct HTTP'; Regex = 'Invoke-RestMethod|Invoke-WebRequest|System\.Net\.Http' }
@{ Name = 'Filesystem access'; Regex = 'Get-Content|Set-Content|Out-File|Export-Csv' }
@{ Name = 'Non-deterministic input'; Regex = 'Get-Random|Get-Date|\[datetime\]::(Now|UtcNow|Today)|New-Guid' }
)
if (-not (Test-Path $EnginePath)) {
Write-Host "Engine path '$EnginePath' does not exist yet - nothing to check." -ForegroundColor Yellow
if ($PassThru) { return @() }
exit 0
}
$findings = [System.Collections.Generic.List[object]]::new()
foreach ($file in Get-ChildItem -Path $EnginePath -Filter '*.ps1' -File -Recurse) {
# Tokenize rather than scan raw text. Comments in this codebase legitimately
# name the forbidden functions when explaining why the engine does not call
# them, and a text scan cannot tell a trailing comment from code. The parser
# can, exactly.
$tokens = $null
$parseErrors = $null
$null = [System.Management.Automation.Language.Parser]::ParseFile(
$file.FullName, [ref]$tokens, [ref]$parseErrors)
if ($parseErrors.Count -gt 0) {
$findings.Add([pscustomobject]@{
File = $file.Name
Line = $parseErrors[0].Extent.StartLineNumber
Dependency = 'Parse error'
Text = $parseErrors[0].Message
})
continue
}
# Rebuild each line from its non-comment tokens.
$codeByLine = @{}
foreach ($token in $tokens) {
if ($token.Kind -eq 'Comment') { continue }
$line = $token.Extent.StartLineNumber
if (-not $codeByLine.ContainsKey($line)) { $codeByLine[$line] = [System.Text.StringBuilder]::new() }
$null = $codeByLine[$line].Append($token.Text).Append(' ')
}
foreach ($line in ($codeByLine.Keys | Sort-Object)) {
$code = $codeByLine[$line].ToString()
foreach ($rule in $forbidden) {
if ($code -match $rule.Regex) {
$findings.Add([pscustomobject]@{
File = $file.Name
Line = $line
Dependency = $rule.Name
Text = $code.Trim()
})
}
}
}
}
if ($findings.Count -gt 0) {
Write-Host "Engine purity check FAILED - $($findings.Count) violation(s) of Principle IV:" -ForegroundColor Red
$findings | Format-Table -AutoSize | Out-String | Write-Host
Write-Host 'The rule engine must remain testable offline with synthetic data.' -ForegroundColor Red
if ($PassThru) { return $findings }
exit 1
}
Write-Host 'Engine purity check passed: the rule engine has no forbidden dependencies.' -ForegroundColor Green
if ($PassThru) { return @() }
exit 0