Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,261 @@
|
||||
function Test-PersonaConfiguration {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Runs all four validation layers over a configuration file (VR-001, FR-002).
|
||||
|
||||
.DESCRIPTION
|
||||
Layers run in order and fail fast between them:
|
||||
|
||||
1. Syntax ConvertFrom-Json PE-SYN-nnn
|
||||
2. Schema Test-Json -SchemaFile PE-SCH-nnn
|
||||
3. Semantic Test-PersonaConfigurationSemantic PE-SEM-nnn
|
||||
4. Safety Test-PersonaConfigurationSafety PE-SAF-nnn
|
||||
|
||||
A layer that produces Error findings stops the sequence. Running semantic
|
||||
checks over a structurally invalid document produces noise, not signal: every
|
||||
missing field yields a cascade of consequent errors, and the author has to
|
||||
guess which one is the actual cause.
|
||||
|
||||
Layer 2 error handling is driven by the V-5a observation
|
||||
(specs/001-persona-engine/verification/V-5a.md), which matters more than it
|
||||
looks. On PowerShell 7.6.5, Test-Json returns $true when the SCHEMA ITSELF is
|
||||
unparseable, writing the failure to the error stream instead. A wrapper that
|
||||
trusted the return value would report a configuration as schema-valid when
|
||||
the schema never ran. So this function treats a non-empty error variable as
|
||||
failure regardless of what was returned, and separates the "schema is broken"
|
||||
case from the "configuration is invalid" case, because they need different
|
||||
exit codes.
|
||||
|
||||
.PARAMETER Path
|
||||
Configuration file to validate.
|
||||
|
||||
.PARAMETER SchemaPath
|
||||
Schema override. Defaults to the shipped config/persona-engine.schema.json.
|
||||
|
||||
.PARAMETER PreviousConfigPath
|
||||
A previously deployed configuration, enabling the VR-003 comparison checks
|
||||
(version downgrade, undeclared rule deletion or reorder).
|
||||
|
||||
.PARAMETER EnforcementEnabled
|
||||
Whether this configuration will be used for an enforcing run. Raises the
|
||||
severity of several safety findings.
|
||||
|
||||
.PARAMETER SkipSafety
|
||||
Runs layers 1-3 only. Used by the editor while a document is mid-edit.
|
||||
|
||||
.OUTPUTS
|
||||
PersonaEngine.ValidationResult
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
[OutputType([pscustomobject])]
|
||||
param(
|
||||
[Parameter(Mandatory)]
|
||||
[ValidateNotNullOrEmpty()]
|
||||
[string] $Path,
|
||||
|
||||
[string] $SchemaPath,
|
||||
|
||||
[string] $PreviousConfigPath,
|
||||
|
||||
[switch] $EnforcementEnabled,
|
||||
|
||||
[switch] $SkipSafety
|
||||
)
|
||||
|
||||
$findings = [System.Collections.Generic.List[object]]::new()
|
||||
$document = $null
|
||||
$stoppedAtLayer = $null
|
||||
$schemaUnusable = $false
|
||||
|
||||
# ---------------------------------------------------------------- Layer 1
|
||||
|
||||
if (-not (Test-Path -LiteralPath $Path -PathType Leaf)) {
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SYN-001' `
|
||||
-Location $Path `
|
||||
-Description "Configuration file not found or is not a file: '$Path'." `
|
||||
-SuggestedResolution 'Check the path. In a pipeline, confirm the file was checked out and the working directory is what you expect.'))
|
||||
|
||||
return New-PersonaValidationResult -Findings $findings -Document $null -StoppedAtLayer 'Syntax' -SchemaUnusable $false
|
||||
}
|
||||
|
||||
$raw = $null
|
||||
try {
|
||||
$raw = Get-Content -LiteralPath $Path -Raw -ErrorAction Stop
|
||||
}
|
||||
catch {
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SYN-002' `
|
||||
-Location $Path `
|
||||
-Description "Configuration file could not be read: $($_.Exception.Message)" `
|
||||
-SuggestedResolution 'Check file permissions and that no other process holds an exclusive lock.'))
|
||||
|
||||
return New-PersonaValidationResult -Findings $findings -Document $null -StoppedAtLayer 'Syntax' -SchemaUnusable $false
|
||||
}
|
||||
|
||||
try {
|
||||
$document = $raw | ConvertFrom-Json -Depth 32 -ErrorAction Stop
|
||||
}
|
||||
catch {
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SYN-003' `
|
||||
-Location $Path `
|
||||
-Description "Configuration is not valid JSON: $($_.Exception.Message)" `
|
||||
-SuggestedResolution 'Fix the JSON syntax. A trailing comma or an unquoted key is the usual cause.'))
|
||||
|
||||
return New-PersonaValidationResult -Findings $findings -Document $null -StoppedAtLayer 'Syntax' -SchemaUnusable $false
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- Layer 2
|
||||
|
||||
if (-not $SchemaPath) {
|
||||
$moduleRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
$SchemaPath = Join-Path $moduleRoot 'config/persona-engine.schema.json'
|
||||
}
|
||||
|
||||
if (-not (Test-Path -LiteralPath $SchemaPath -PathType Leaf)) {
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SCH-002' `
|
||||
-Location $SchemaPath `
|
||||
-Description "Schema file not found: '$SchemaPath'." `
|
||||
-SuggestedResolution 'Supply -SchemaPath, or restore config/persona-engine.schema.json.'))
|
||||
|
||||
return New-PersonaValidationResult -Findings $findings -Document $document -StoppedAtLayer 'Schema' -SchemaUnusable $true
|
||||
}
|
||||
|
||||
$schemaErrors = $null
|
||||
$schemaOk = $raw | Test-Json -SchemaFile $SchemaPath -ErrorAction SilentlyContinue -ErrorVariable schemaErrors
|
||||
|
||||
foreach ($schemaError in @($schemaErrors)) {
|
||||
$message = [string]$schemaError.Exception.Message
|
||||
|
||||
# V-5a: this message arrives with a $true return value. Treating it as a
|
||||
# pass would validate every configuration against a schema that never ran.
|
||||
if ($message -match 'Cannot parse the JSON schema') {
|
||||
$schemaUnusable = $true
|
||||
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SCH-003' `
|
||||
-Location $SchemaPath `
|
||||
-Description "The schema file itself is not valid JSON Schema and could not be used: $message" `
|
||||
-SuggestedResolution 'Repair the schema file. Until it parses, no configuration can be schema-validated, and a passing result would be meaningless.'))
|
||||
continue
|
||||
}
|
||||
|
||||
$findings.Add((New-PersonaValidationFinding -Severity Error -Code 'PE-SCH-001' `
|
||||
-Location (Get-PersonaSchemaErrorLocation -Message $message) `
|
||||
-Description $message `
|
||||
-SuggestedResolution 'Correct the document to match config/persona-engine.schema.json. Test-Json reports only the first violation per run, so re-validate after each fix.'))
|
||||
}
|
||||
|
||||
if ($schemaUnusable -or -not $schemaOk -or @($schemaErrors).Count -gt 0) {
|
||||
return New-PersonaValidationResult -Findings $findings -Document $document -StoppedAtLayer 'Schema' -SchemaUnusable $schemaUnusable
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- Layer 3
|
||||
|
||||
foreach ($finding in (Test-PersonaConfigurationSemantic -Document $document)) { $findings.Add($finding) }
|
||||
|
||||
if (@($findings | Where-Object Severity -EQ 'Error').Count -gt 0) {
|
||||
return New-PersonaValidationResult -Findings $findings -Document $document -StoppedAtLayer 'Semantic' -SchemaUnusable $false
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- Layer 4
|
||||
|
||||
if (-not $SkipSafety) {
|
||||
$safetyParams = @{ Document = $document; EnforcementEnabled = $EnforcementEnabled }
|
||||
if ($PreviousConfigPath) { $safetyParams['PreviousConfigPath'] = $PreviousConfigPath }
|
||||
|
||||
foreach ($finding in (Test-PersonaConfigurationSafety @safetyParams)) { $findings.Add($finding) }
|
||||
}
|
||||
|
||||
New-PersonaValidationResult -Findings $findings -Document $document -StoppedAtLayer $null -SchemaUnusable $false
|
||||
}
|
||||
|
||||
function New-PersonaValidationResult {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Wraps a finding collection into the shape every caller consumes.
|
||||
|
||||
.DESCRIPTION
|
||||
IsValid is computed here rather than by each caller, so console, pipeline,
|
||||
and editor cannot disagree about what "valid" means. Warnings never affect
|
||||
IsValid; escalation under -TreatWarningsAsErrors is the caller's decision
|
||||
(VR-005) and belongs where the exit code is chosen.
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
[OutputType([pscustomobject])]
|
||||
param(
|
||||
[Parameter(Mandatory)] [AllowEmptyCollection()] [object] $Findings,
|
||||
[AllowNull()] [object] $Document,
|
||||
[AllowNull()] [string] $StoppedAtLayer,
|
||||
[bool] $SchemaUnusable
|
||||
)
|
||||
|
||||
$all = @($Findings)
|
||||
|
||||
[pscustomobject]@{
|
||||
PSTypeName = 'PersonaEngine.ValidationResult'
|
||||
IsValid = (@($all | Where-Object Severity -EQ 'Error').Count -eq 0)
|
||||
Findings = $all
|
||||
ErrorCount = @($all | Where-Object Severity -EQ 'Error').Count
|
||||
WarningCount = @($all | Where-Object Severity -EQ 'Warning').Count
|
||||
Document = $Document
|
||||
StoppedAtLayer = $StoppedAtLayer
|
||||
SchemaUnusable = $SchemaUnusable
|
||||
}
|
||||
}
|
||||
|
||||
function Get-PersonaSchemaErrorLocation {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Extracts the JSON pointer from a Test-Json error message.
|
||||
|
||||
.DESCRIPTION
|
||||
Test-Json embeds the failing location in prose - "... at '/rules/3/priority'".
|
||||
VR-004 requires a location on every finding, so it is lifted out here rather
|
||||
than leaving the caller to read it out of the description. When no pointer is
|
||||
present the document root is reported, which is honest: the violation is
|
||||
somewhere in the document and the message says where in words.
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
[OutputType([string])]
|
||||
param([Parameter(Mandatory)] [AllowEmptyString()] [string] $Message)
|
||||
|
||||
if ($Message -match "at '([^']*)'") {
|
||||
$pointer = $Matches[1]
|
||||
return $pointer ? $pointer : '/'
|
||||
}
|
||||
|
||||
'/'
|
||||
}
|
||||
|
||||
function Write-PersonaValidationFinding {
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Renders validation findings for a human reader.
|
||||
|
||||
.DESCRIPTION
|
||||
Grouped by severity, most serious first, with the suggested resolution on
|
||||
its own line. A finding without a visible resolution just tells someone they
|
||||
are wrong, which is why VR-004 makes the field mandatory and why it is
|
||||
printed rather than hidden behind a verbose switch.
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory)] [AllowEmptyCollection()] [object[]] $Findings
|
||||
)
|
||||
|
||||
if (@($Findings).Count -eq 0) {
|
||||
Write-Host 'Validation passed with no findings.' -ForegroundColor Green
|
||||
return
|
||||
}
|
||||
|
||||
foreach ($severity in @('Error', 'Warning', 'Information')) {
|
||||
$set = @($Findings | Where-Object Severity -EQ $severity)
|
||||
if ($set.Count -eq 0) { continue }
|
||||
|
||||
$colour = switch ($severity) { 'Error' { 'Red' } 'Warning' { 'Yellow' } default { 'Gray' } }
|
||||
|
||||
foreach ($finding in $set) {
|
||||
Write-Host ("[{0}] {1} {2}" -f $finding.Severity.ToUpperInvariant(), $finding.Code, $finding.Location) -ForegroundColor $colour
|
||||
Write-Host (" {0}" -f $finding.Description)
|
||||
Write-Host (" -> {0}" -f $finding.SuggestedResolution) -ForegroundColor DarkGray
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user