Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,442 @@
|
||||
#Requires -Version 7.2
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Validates, tests, and interactively edits a Persona Engine configuration.
|
||||
|
||||
.DESCRIPTION
|
||||
Two jobs in one tool, deliberately: the thing that checks a configuration and the
|
||||
thing that edits it must agree about what "valid" means, and the surest way to
|
||||
guarantee that is to make them the same code path.
|
||||
|
||||
-ValidateOnly report findings and exit. Never enters the editor.
|
||||
-NonInteractive pipeline mode. Never prompts, never hangs (SC-010).
|
||||
(neither) interactive editor, re-validating before every save.
|
||||
|
||||
-NonInteractive is not a convenience flag. A build agent runs with stdin closed;
|
||||
a tool that prompts there does not fail, it hangs until the job times out, and
|
||||
the pipeline reports an infrastructure problem instead of a bad configuration.
|
||||
Every prompting call in this script sits behind a check for it.
|
||||
|
||||
Rule testing against synthetic users (-TestDataPath) runs the real rule engine
|
||||
with no tenant connectivity (FR-025, SC-008). The engine is pure, so the answer
|
||||
it gives offline is the answer it gives in production.
|
||||
|
||||
.PARAMETER ConfigPath
|
||||
Configuration to validate or edit.
|
||||
|
||||
.PARAMETER ValidateOnly
|
||||
Validate and report; do not enter the editor.
|
||||
|
||||
.PARAMETER NonInteractive
|
||||
Pipeline mode: no prompts, exit code only.
|
||||
|
||||
.PARAMETER SchemaPath
|
||||
Override the shipped schema.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Save-As target. Leaves the input file untouched.
|
||||
|
||||
.PARAMETER TreatWarningsAsErrors
|
||||
Escalate Warning findings to blocking (VR-005).
|
||||
|
||||
.PARAMETER TestDataPath
|
||||
Synthetic sample users for offline rule testing (FR-025).
|
||||
|
||||
.PARAMETER PreviousConfigPath
|
||||
Currently deployed configuration, enabling the VR-003 drift checks.
|
||||
|
||||
.PARAMETER EnforcementEnabled
|
||||
Validate as though this configuration will drive an enforcing run, which raises
|
||||
the severity of several safety findings.
|
||||
|
||||
.EXAMPLE
|
||||
./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -ValidateOnly -NonInteractive
|
||||
|
||||
The pipeline invocation. Returns 0 when the configuration is clean.
|
||||
|
||||
.EXAMPLE
|
||||
./Edit-PersonaEngineConfig.ps1 -ConfigPath ./config/persona-engine.json -TestDataPath ./tests/TestData
|
||||
|
||||
Validate, then evaluate the rules against synthetic users with no tenant.
|
||||
|
||||
.NOTES
|
||||
Exit codes
|
||||
0 valid; no blocking findings
|
||||
1 one or more Error findings
|
||||
2 Warning findings present with -TreatWarningsAsErrors
|
||||
3 configuration file not found or unreadable
|
||||
4 schema file not found or itself invalid
|
||||
#>
|
||||
[CmdletBinding(SupportsShouldProcess = $true)]
|
||||
param(
|
||||
[Parameter(Mandatory)]
|
||||
[ValidateNotNullOrEmpty()]
|
||||
[string] $ConfigPath,
|
||||
|
||||
[switch] $ValidateOnly,
|
||||
|
||||
[switch] $NonInteractive,
|
||||
|
||||
[string] $SchemaPath,
|
||||
|
||||
[string] $OutputPath,
|
||||
|
||||
[switch] $TreatWarningsAsErrors,
|
||||
|
||||
[string] $TestDataPath,
|
||||
|
||||
[string] $PreviousConfigPath,
|
||||
|
||||
[switch] $EnforcementEnabled
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# The .psm1 directly, not the manifest. The manifest declares
|
||||
# Microsoft.Graph.Authentication as a required module, and this tool never touches
|
||||
# Graph - SC-008 requires full validation and synthetic rule testing to complete with
|
||||
# no network access. Importing the manifest would make a build agent that only
|
||||
# validates configuration install a Graph SDK it will never call.
|
||||
Import-Module (Join-Path $PSScriptRoot 'PersonaEngine.psm1') -Force -ErrorAction Stop
|
||||
|
||||
$EXIT_OK = 0
|
||||
$EXIT_FINDINGS = 1
|
||||
$EXIT_WARNINGS_AS_ERRORS = 2
|
||||
$EXIT_CONFIG_UNREADABLE = 3
|
||||
$EXIT_SCHEMA_UNUSABLE = 4
|
||||
|
||||
function Invoke-ConfigurationValidation {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Runs all four layers and maps the result onto an exit code.
|
||||
|
||||
.DESCRIPTION
|
||||
One function so the interactive save path and the pipeline path cannot
|
||||
disagree. The exit-code mapping lives here rather than at the call sites for
|
||||
the same reason: an exit code is the pipeline's only view of what happened.
|
||||
#>
|
||||
param([string] $Path)
|
||||
|
||||
$params = @{ Path = $Path; EnforcementEnabled = $EnforcementEnabled }
|
||||
if ($SchemaPath) { $params['SchemaPath'] = $SchemaPath }
|
||||
if ($PreviousConfigPath) { $params['PreviousConfigPath'] = $PreviousConfigPath }
|
||||
|
||||
$result = Test-PersonaConfiguration @params
|
||||
|
||||
$code = $EXIT_OK
|
||||
|
||||
# Order matters. An unreadable file and an unusable schema are environment
|
||||
# faults, not authoring faults, and a pipeline needs to tell them apart from a
|
||||
# genuinely invalid configuration - otherwise a missing schema file is reported
|
||||
# to the author as "your rules are wrong".
|
||||
if (@($result.Findings | Where-Object Code -In @('PE-SYN-001', 'PE-SYN-002')).Count -gt 0) {
|
||||
$code = $EXIT_CONFIG_UNREADABLE
|
||||
}
|
||||
elseif ($result.SchemaUnusable -or @($result.Findings | Where-Object Code -EQ 'PE-SCH-002').Count -gt 0) {
|
||||
$code = $EXIT_SCHEMA_UNUSABLE
|
||||
}
|
||||
elseif (-not $result.IsValid) {
|
||||
$code = $EXIT_FINDINGS
|
||||
}
|
||||
elseif ($TreatWarningsAsErrors -and $result.WarningCount -gt 0) {
|
||||
# VR-005: warnings block only here. The finding severity is unchanged - the
|
||||
# configuration is not retroactively more broken because a switch was passed;
|
||||
# what changed is the caller's tolerance for it.
|
||||
$code = $EXIT_WARNINGS_AS_ERRORS
|
||||
}
|
||||
|
||||
[pscustomobject]@{ Result = $result; ExitCode = $code }
|
||||
}
|
||||
|
||||
function Invoke-SyntheticRuleTest {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Evaluates the configuration's rules against synthetic users (FR-025).
|
||||
|
||||
.DESCRIPTION
|
||||
No tenant, no credentials, no network. Reads the fixture files, builds
|
||||
normalized records, and runs the real engine over them.
|
||||
|
||||
Fixtures declare their membership facets explicitly, including failed ones.
|
||||
A fixture with an unretrieved facet is the only way to see EvaluationError
|
||||
behaviour before it happens against a live directory.
|
||||
#>
|
||||
param(
|
||||
[Parameter(Mandatory)] [object] $Configuration,
|
||||
[Parameter(Mandatory)] [string] $DataPath
|
||||
)
|
||||
|
||||
$usersFile = (Test-Path -LiteralPath $DataPath -PathType Container) `
|
||||
? (Join-Path $DataPath 'Users/users.json') `
|
||||
: $DataPath
|
||||
|
||||
if (-not (Test-Path -LiteralPath $usersFile -PathType Leaf)) {
|
||||
Write-Host "No synthetic user fixtures found at '$usersFile'." -ForegroundColor Yellow
|
||||
return
|
||||
}
|
||||
|
||||
$fixtures = (Get-Content -LiteralPath $usersFile -Raw | ConvertFrom-Json -Depth 32).users
|
||||
|
||||
$membershipFile = Join-Path (Split-Path (Split-Path $usersFile -Parent) -Parent) 'Memberships/memberships.json'
|
||||
$membershipByFixture = @{}
|
||||
|
||||
if (Test-Path -LiteralPath $membershipFile -PathType Leaf) {
|
||||
foreach ($entry in (Get-Content -LiteralPath $membershipFile -Raw | ConvertFrom-Json -Depth 32).memberships) {
|
||||
$params = @{
|
||||
DirectGroupObjectIds = @($entry.directGroupObjectIds)
|
||||
TransitiveGroupObjectIds = @($entry.transitiveGroupObjectIds)
|
||||
DirectoryRoleIds = @($entry.directoryRoleIds)
|
||||
}
|
||||
foreach ($facet in @('allRetrieved', 'directRetrieved', 'transitiveRetrieved', 'rolesRetrieved')) {
|
||||
if ($entry.PSObject.Properties[$facet] -and $entry.$facet) {
|
||||
$params[(Get-Culture).TextInfo.ToTitleCase($facet)] = $true
|
||||
}
|
||||
}
|
||||
foreach ($facet in @('directFailureReason', 'transitiveFailureReason', 'rolesFailureReason')) {
|
||||
if ($entry.PSObject.Properties[$facet] -and $entry.$facet) {
|
||||
$params[(Get-Culture).TextInfo.ToTitleCase($facet)] = [string]$entry.$facet
|
||||
}
|
||||
}
|
||||
$membershipByFixture[[string]$entry.fixtureId] = New-PersonaMembershipRecord @params
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host ''
|
||||
Write-Host 'Synthetic rule test - no tenant connectivity (FR-025, SC-008)' -ForegroundColor Cyan
|
||||
Write-Host ('{0,-24} {1,-45} {2,-20} {3}' -f 'Fixture', 'UPN', 'Outcome', 'Persona / rule') -ForegroundColor DarkGray
|
||||
|
||||
foreach ($fixture in $fixtures) {
|
||||
$properties = @{}
|
||||
if ($fixture.properties) {
|
||||
foreach ($p in $fixture.properties.PSObject.Properties) { $properties[$p.Name] = $p.Value }
|
||||
}
|
||||
|
||||
$record = New-PersonaUserRecord `
|
||||
-AccountObjectId ([string]$fixture.accountObjectId) `
|
||||
-UserPrincipalName ([string]$fixture.userPrincipalName) `
|
||||
-DisplayName ([string]$fixture.displayName) `
|
||||
-UserType ([string]$fixture.userType) `
|
||||
-AccountEnabled ([bool]$fixture.accountEnabled) `
|
||||
-Properties $properties `
|
||||
-StoredPersona ([string]$fixture.storedPersona) `
|
||||
-Membership $membershipByFixture[[string]$fixture.fixtureId]
|
||||
|
||||
$result = Resolve-UserPersona -UserRecord $record -Rules $Configuration.Rules `
|
||||
-MaxDepth $Configuration.MaxConditionDepth `
|
||||
-DefaultMembershipMode $Configuration.DefaultMembershipMode
|
||||
|
||||
$colour = switch ($result.Outcome) {
|
||||
'Matched' { 'Green' }
|
||||
'Unclassified' { 'Gray' }
|
||||
'EvaluationError' { 'Yellow' }
|
||||
}
|
||||
|
||||
$detail = $result.Outcome -eq 'Matched' `
|
||||
? "$($result.CalculatedPersona) [$($result.MatchedRuleId)]" `
|
||||
: [string]$result.EvaluationErrorReason
|
||||
|
||||
Write-Host ('{0,-24} {1,-45} {2,-20} {3}' -f
|
||||
$fixture.fixtureId, $result.UserPrincipalName, $result.Outcome, $detail) -ForegroundColor $colour
|
||||
}
|
||||
|
||||
Write-Host ''
|
||||
}
|
||||
|
||||
function Save-PersonaConfiguration {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Saves an edited configuration, backing up first (FR-026).
|
||||
|
||||
.DESCRIPTION
|
||||
The save order is the contract, and it is deliberate:
|
||||
|
||||
1. Re-validate in full.
|
||||
2. Refuse on any Error finding.
|
||||
3. Back up the existing file, or write to -OutputPath instead.
|
||||
4. Only then replace.
|
||||
|
||||
Backing up before replacing rather than after is what makes step 4 safe to
|
||||
interrupt. A crash between backup and write leaves the original and a copy;
|
||||
a crash the other way round leaves neither.
|
||||
#>
|
||||
param(
|
||||
[Parameter(Mandatory)] [object] $Document,
|
||||
[Parameter(Mandatory)] [string] $Path,
|
||||
[string] $SaveAs
|
||||
)
|
||||
|
||||
$json = $Document | ConvertTo-Json -Depth 32
|
||||
|
||||
$temp = [System.IO.Path]::GetTempFileName()
|
||||
try {
|
||||
Set-Content -LiteralPath $temp -Value $json -Encoding utf8NoBOM
|
||||
|
||||
$check = Invoke-ConfigurationValidation -Path $temp
|
||||
if (-not $check.Result.IsValid) {
|
||||
Write-Host 'Save blocked: the edited configuration has Error findings.' -ForegroundColor Red
|
||||
Write-PersonaValidationFinding -Findings $check.Result.Findings
|
||||
return $false
|
||||
}
|
||||
}
|
||||
finally {
|
||||
Remove-Item -LiteralPath $temp -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
$destination = $SaveAs ? $SaveAs : $Path
|
||||
|
||||
if ((Test-Path -LiteralPath $destination -PathType Leaf) -and -not $SaveAs) {
|
||||
$backup = '{0}.{1}.bak' -f $destination, [DateTime]::UtcNow.ToString('yyyyMMddTHHmmssZ')
|
||||
|
||||
if (-not $PSCmdlet.ShouldProcess($destination, "Back up to '$backup' and replace")) {
|
||||
Write-Host 'Save cancelled.' -ForegroundColor Yellow
|
||||
return $false
|
||||
}
|
||||
|
||||
Copy-Item -LiteralPath $destination -Destination $backup -Force
|
||||
Write-Host "Backup written: $backup" -ForegroundColor DarkGray
|
||||
}
|
||||
elseif (-not $PSCmdlet.ShouldProcess($destination, 'Write configuration')) {
|
||||
Write-Host 'Save cancelled.' -ForegroundColor Yellow
|
||||
return $false
|
||||
}
|
||||
|
||||
Set-Content -LiteralPath $destination -Value $json -Encoding utf8NoBOM
|
||||
Write-Host "Saved: $destination" -ForegroundColor Green
|
||||
$true
|
||||
}
|
||||
|
||||
function Invoke-InteractiveEditor {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
The interactive loop (FR-023).
|
||||
|
||||
.DESCRIPTION
|
||||
Deliberately small. It toggles rules, adjusts priorities, re-validates, tests
|
||||
against fixtures, and saves. It does not attempt to be a JSON editor - rule
|
||||
authoring belongs in a text editor with schema completion, and a
|
||||
half-featured structural editor would only add ways to corrupt a file that
|
||||
is already under version control.
|
||||
#>
|
||||
param(
|
||||
[Parameter(Mandatory)] [object] $Document,
|
||||
[Parameter(Mandatory)] [string] $Path
|
||||
)
|
||||
|
||||
$dirty = $false
|
||||
|
||||
while ($true) {
|
||||
Write-Host ''
|
||||
Write-Host 'Persona Engine configuration editor' -ForegroundColor Cyan
|
||||
Write-Host (" file {0}{1}" -f $Path, ($dirty ? ' [modified]' : ''))
|
||||
Write-Host (" version {0}" -f $Document.configVersion)
|
||||
Write-Host (" rules {0} ({1} enabled)" -f @($Document.rules).Count, @($Document.rules | Where-Object { $_.enabled }).Count)
|
||||
Write-Host ''
|
||||
Write-Host ' [L] list rules [T] toggle a rule [P] change a priority'
|
||||
Write-Host ' [V] re-validate [R] run rule test [S] save'
|
||||
Write-Host ' [Q] quit'
|
||||
Write-Host ''
|
||||
|
||||
$choice = (Read-Host 'Choice').Trim().ToUpperInvariant()
|
||||
|
||||
switch ($choice) {
|
||||
|
||||
'L' {
|
||||
Write-Host ''
|
||||
foreach ($rule in ($Document.rules | Sort-Object { [int]$_.priority })) {
|
||||
Write-Host (' {0,-6} {1,-28} {2,-30} {3}' -f
|
||||
$rule.priority, $rule.id, $rule.persona,
|
||||
($rule.enabled ? 'enabled' : 'DISABLED')) -ForegroundColor ($rule.enabled ? 'Gray' : 'DarkGray')
|
||||
}
|
||||
}
|
||||
|
||||
'T' {
|
||||
$id = (Read-Host 'Rule ID to toggle').Trim()
|
||||
$rule = $Document.rules | Where-Object { [string]$_.id -eq $id } | Select-Object -First 1
|
||||
|
||||
if (-not $rule) { Write-Host "No rule with ID '$id'." -ForegroundColor Yellow; break }
|
||||
|
||||
$rule.enabled = -not $rule.enabled
|
||||
$dirty = $true
|
||||
Write-Host ("Rule {0} is now {1}." -f $id, ($rule.enabled ? 'enabled' : 'disabled')) -ForegroundColor Green
|
||||
}
|
||||
|
||||
'P' {
|
||||
$id = (Read-Host 'Rule ID').Trim()
|
||||
$rule = $Document.rules | Where-Object { [string]$_.id -eq $id } | Select-Object -First 1
|
||||
|
||||
if (-not $rule) { Write-Host "No rule with ID '$id'." -ForegroundColor Yellow; break }
|
||||
|
||||
$value = 0
|
||||
if (-not [int]::TryParse((Read-Host 'New priority'), [ref] $value)) {
|
||||
Write-Host 'Priority must be an integer.' -ForegroundColor Yellow
|
||||
break
|
||||
}
|
||||
|
||||
$rule.priority = $value
|
||||
$dirty = $true
|
||||
Write-Host "Rule $id priority is now $value. Re-validate before saving - a reorder changes which rule wins." -ForegroundColor Green
|
||||
}
|
||||
|
||||
'V' {
|
||||
$temp = [System.IO.Path]::GetTempFileName()
|
||||
try {
|
||||
Set-Content -LiteralPath $temp -Value ($Document | ConvertTo-Json -Depth 32) -Encoding utf8NoBOM
|
||||
$check = Invoke-ConfigurationValidation -Path $temp
|
||||
Write-PersonaValidationFinding -Findings $check.Result.Findings
|
||||
}
|
||||
finally { Remove-Item -LiteralPath $temp -Force -ErrorAction SilentlyContinue }
|
||||
}
|
||||
|
||||
'R' {
|
||||
$dataPath = $TestDataPath ? $TestDataPath : (Join-Path $PSScriptRoot 'tests/TestData')
|
||||
$temp = [System.IO.Path]::GetTempFileName()
|
||||
try {
|
||||
Set-Content -LiteralPath $temp -Value ($Document | ConvertTo-Json -Depth 32) -Encoding utf8NoBOM
|
||||
Invoke-SyntheticRuleTest -Configuration (Import-PersonaConfiguration -Path $temp) -DataPath $dataPath
|
||||
}
|
||||
finally { Remove-Item -LiteralPath $temp -Force -ErrorAction SilentlyContinue }
|
||||
}
|
||||
|
||||
'S' {
|
||||
if (Save-PersonaConfiguration -Document $Document -Path $Path -SaveAs $OutputPath) { $dirty = $false }
|
||||
}
|
||||
|
||||
'Q' {
|
||||
if ($dirty) {
|
||||
$confirm = (Read-Host 'Unsaved changes will be lost. Quit anyway? (y/N)').Trim()
|
||||
if ($confirm -ne 'y') { break }
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
default { Write-Host 'Unrecognized choice.' -ForegroundColor Yellow }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================================ main
|
||||
|
||||
$validation = Invoke-ConfigurationValidation -Path $ConfigPath
|
||||
|
||||
Write-PersonaValidationFinding -Findings $validation.Result.Findings
|
||||
|
||||
if ($validation.ExitCode -ne $EXIT_OK) {
|
||||
Write-Host ("Validation failed with exit code {0}." -f $validation.ExitCode) -ForegroundColor Red
|
||||
exit $validation.ExitCode
|
||||
}
|
||||
|
||||
if ($TestDataPath) {
|
||||
Invoke-SyntheticRuleTest -Configuration (Import-PersonaConfiguration -Path $ConfigPath) -DataPath $TestDataPath
|
||||
}
|
||||
|
||||
# -ValidateOnly and -NonInteractive both short-circuit before any prompting call.
|
||||
# The check is here, once, at the single point where the editor could be entered.
|
||||
if ($ValidateOnly -or $NonInteractive) {
|
||||
Write-Host 'Configuration is valid.' -ForegroundColor Green
|
||||
exit $EXIT_OK
|
||||
}
|
||||
|
||||
Invoke-InteractiveEditor -Document $validation.Result.Document -Path $ConfigPath
|
||||
|
||||
exit $EXIT_OK
|
||||
Reference in New Issue
Block a user