Implement Stage A: rule engine, validation, audit, and safety gates
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,130 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
Editor exit codes 0-4 (cli-edit-persona-engine-config.md).
|
||||
|
||||
An exit code is the only thing a pipeline sees. Every code has to be reachable and
|
||||
has to mean what the contract says - in particular, code 3 (file unreadable) and
|
||||
code 4 (schema unusable) must not collapse into code 1, or a missing schema file
|
||||
gets reported to a rule author as "your configuration is invalid".
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
|
||||
$script:editor = Join-Path $repoRoot 'Edit-PersonaEngineConfig.ps1'
|
||||
$script:scratch = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-ec-{0}" -f [guid]::NewGuid().ToString('N'))
|
||||
$null = New-Item -ItemType Directory -Path $script:scratch -Force
|
||||
|
||||
function Invoke-Editor {
|
||||
param([string[]] $ArgumentList)
|
||||
|
||||
$exe = (Get-Process -Id $PID).Path
|
||||
$null = & $exe -NoProfile -NonInteractive -File $script:editor @ArgumentList 2>&1
|
||||
$LASTEXITCODE
|
||||
}
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Editor exit codes' {
|
||||
|
||||
It 'returns 0 for a valid configuration' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive') | Should -Be 0
|
||||
}
|
||||
|
||||
It 'returns 1 when Error findings are present' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].persona = 'Undeclared-Persona'
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive') | Should -Be 1
|
||||
}
|
||||
|
||||
It 'returns 2 for Warning findings under -TreatWarningsAsErrors' {
|
||||
# A pinned global mode plus a per-condition override: a Warning, not an Error.
|
||||
# VR-005 escalates it only when the caller asks.
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.dataSources.groups.membershipMode = 'direct'
|
||||
$document.rules += @{
|
||||
id = 'RULE-0030-TIER0'; name = 'Tier 0'; description = 'Tier 0 group members.'
|
||||
enabled = $true; priority = 30; persona = 'Tier0-Admin'
|
||||
match = @{
|
||||
operator = 'all'
|
||||
conditions = @(@{
|
||||
type = 'membership'; operator = 'memberOf'; membershipMode = 'transitive'
|
||||
groupObjectIds = @('00000000-0000-0000-0000-0000000000a0')
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive') | Should -Be 0
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive', '-TreatWarningsAsErrors') | Should -Be 2
|
||||
}
|
||||
|
||||
It 'returns 3 when the configuration file is absent' {
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', (Join-Path $script:scratch 'absent.json'), '-ValidateOnly', '-NonInteractive') | Should -Be 3
|
||||
}
|
||||
|
||||
It 'returns 4 when the schema file is absent' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
Invoke-Editor -ArgumentList @(
|
||||
'-ConfigPath', $path, '-ValidateOnly', '-NonInteractive',
|
||||
'-SchemaPath', (Join-Path $script:scratch 'no-schema.json')) | Should -Be 4
|
||||
}
|
||||
|
||||
It 'returns 4 when the schema file exists but cannot be parsed' {
|
||||
# Distinct from code 1 on purpose. The configuration was never actually
|
||||
# checked, so calling it invalid would be a guess.
|
||||
$brokenSchema = Join-Path $script:scratch 'broken.json'
|
||||
Set-Content -LiteralPath $brokenSchema -Value '{ not json' -Encoding utf8NoBOM
|
||||
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive', '-SchemaPath', $brokenSchema) | Should -Be 4
|
||||
}
|
||||
|
||||
It 'prefers code 3 over code 1 when the file cannot be read at all' {
|
||||
# An unreadable file produces a PE-SYN Error finding too. Reporting code 1
|
||||
# would tell the author their rules are wrong when the file never opened.
|
||||
Invoke-Editor -ArgumentList @('-ConfigPath', $script:scratch, '-ValidateOnly', '-NonInteractive') | Should -Be 3
|
||||
}
|
||||
|
||||
It 'reaches every documented code across the corpus' {
|
||||
$reached = [System.Collections.Generic.HashSet[int]]::new()
|
||||
|
||||
$valid = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$null = $reached.Add((Invoke-Editor -ArgumentList @('-ConfigPath', $valid, '-ValidateOnly', '-NonInteractive')))
|
||||
|
||||
$invalidDoc = New-TestConfigurationDocument
|
||||
$invalidDoc.rules[1].persona = 'Undeclared-Persona'
|
||||
$invalid = Save-TestConfiguration -Document $invalidDoc -Directory $script:scratch
|
||||
$null = $reached.Add((Invoke-Editor -ArgumentList @('-ConfigPath', $invalid, '-ValidateOnly', '-NonInteractive')))
|
||||
|
||||
$warnDoc = New-TestConfigurationDocument
|
||||
$warnDoc.dataSources.groups.membershipMode = 'direct'
|
||||
$warnDoc.rules += @{
|
||||
id = 'RULE-0030-TIER0'; name = 'Tier 0'; description = 'Tier 0 group members.'
|
||||
enabled = $true; priority = 30; persona = 'Tier0-Admin'
|
||||
match = @{ operator = 'all'; conditions = @(@{
|
||||
type = 'membership'; operator = 'memberOf'; membershipMode = 'transitive'
|
||||
groupObjectIds = @('00000000-0000-0000-0000-0000000000a0') }) }
|
||||
}
|
||||
$warn = Save-TestConfiguration -Document $warnDoc -Directory $script:scratch
|
||||
$null = $reached.Add((Invoke-Editor -ArgumentList @('-ConfigPath', $warn, '-ValidateOnly', '-NonInteractive', '-TreatWarningsAsErrors')))
|
||||
|
||||
$null = $reached.Add((Invoke-Editor -ArgumentList @('-ConfigPath', (Join-Path $script:scratch 'absent.json'), '-ValidateOnly', '-NonInteractive')))
|
||||
$null = $reached.Add((Invoke-Editor -ArgumentList @('-ConfigPath', $valid, '-ValidateOnly', '-NonInteractive', '-SchemaPath', (Join-Path $script:scratch 'no-schema.json'))))
|
||||
|
||||
0..4 | ForEach-Object { $reached | Should -Contain $_ }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
VR-001: the four layers run in order and stop at the first that produces errors.
|
||||
|
||||
The reason is signal, not speed. A document missing a required section produces a
|
||||
cascade of consequent semantic errors, and the author then has to guess which one
|
||||
is the cause. Stopping at the structural failure reports the one thing that is
|
||||
actually wrong.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:schema = Join-Path $repoRoot 'config/persona-engine.schema.json'
|
||||
$script:scratch = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-layers-{0}" -f [guid]::NewGuid().ToString('N'))
|
||||
$null = New-Item -ItemType Directory -Path $script:scratch -Force
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Layer ordering and fail-fast (VR-001)' {
|
||||
|
||||
It 'stops at layer 1 for malformed JSON' {
|
||||
$path = Join-Path $script:scratch 'malformed.json'
|
||||
Set-Content -LiteralPath $path -Value '{ "configVersion": "1.0.0", ' -Encoding utf8NoBOM
|
||||
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.StoppedAtLayer | Should -Be 'Syntax'
|
||||
$result.Findings.Code | Should -Contain 'PE-SYN-003'
|
||||
}
|
||||
|
||||
It 'reports a missing file at layer 1 without attempting to parse it' {
|
||||
$result = Test-PersonaConfiguration -Path (Join-Path $script:scratch 'absent.json') -SchemaPath $script:schema
|
||||
|
||||
$result.StoppedAtLayer | Should -Be 'Syntax'
|
||||
$result.Findings.Code | Should -Contain 'PE-SYN-001'
|
||||
}
|
||||
|
||||
It 'stops at layer 2 for a structurally invalid document, before semantic checks run' {
|
||||
# The document below also has a semantic defect - a duplicate rule ID - which
|
||||
# must NOT appear, because layer 3 never ran.
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].id = $document.rules[0].id
|
||||
$document.engine.Remove('approvedWritableAttributes')
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.StoppedAtLayer | Should -Be 'Schema'
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-001'
|
||||
$result.Findings.Code | Should -Not -Contain 'PE-SEM-001'
|
||||
}
|
||||
|
||||
It 'stops at layer 3 for a semantic error, before safety checks run' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].id = $document.rules[0].id
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.StoppedAtLayer | Should -Be 'Semantic'
|
||||
$result.Findings.Code | Should -Contain 'PE-SEM-001'
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'reaches layer 4 when layers 1 to 3 are clean' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.StoppedAtLayer | Should -BeNullOrEmpty
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -Not -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'skips layer 4 on request without claiming it passed' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema -SkipSafety
|
||||
|
||||
@($result.Findings | Where-Object Layer -EQ 'Safety') | Should -BeNullOrEmpty
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'A schema that cannot be used is never reported as a pass (V-5a)' {
|
||||
|
||||
It 'flags an unparseable schema rather than trusting the $true return value' {
|
||||
# Test-Json returns $true here. Trusting it would validate every configuration
|
||||
# against a schema that never ran.
|
||||
$brokenSchema = Join-Path $script:scratch 'broken-schema.json'
|
||||
Set-Content -LiteralPath $brokenSchema -Value '{ not json' -Encoding utf8NoBOM
|
||||
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $brokenSchema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.SchemaUnusable | Should -BeTrue
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-003'
|
||||
}
|
||||
|
||||
It 'flags a missing schema file separately from an invalid configuration' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath (Join-Path $script:scratch 'no-such-schema.json')
|
||||
|
||||
$result.SchemaUnusable | Should -BeTrue
|
||||
$result.Findings.Code | Should -Contain 'PE-SCH-002'
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'The shipped example configuration passes every layer' {
|
||||
|
||||
It 'validates cleanly against the shipped schema' {
|
||||
# If the example the documentation points at cannot pass its own validator,
|
||||
# every reader's first run fails.
|
||||
$result = Test-PersonaConfiguration -Path (Join-Path $repoRoot 'config/persona-engine.example.json')
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.ErrorCount | Should -Be 0
|
||||
$result.WarningCount | Should -Be 0
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
SC-010: -NonInteractive never prompts and never hangs.
|
||||
|
||||
Each case runs the editor in a child pwsh with stdin redirected from an empty
|
||||
file, which is what a build agent gives it. A tool that prompts there does not
|
||||
fail - it blocks until the job times out, and the pipeline reports an
|
||||
infrastructure problem rather than a bad configuration.
|
||||
|
||||
A wall-clock timeout is the assertion. That makes these the slowest tests in the
|
||||
suite, and there is no cheaper way to prove the absence of a hang: inspecting the
|
||||
source for Read-Host would only prove that one spelling of prompting is absent.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
|
||||
$script:editor = Join-Path $repoRoot 'Edit-PersonaEngineConfig.ps1'
|
||||
$script:scratch = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-ni-{0}" -f [guid]::NewGuid().ToString('N'))
|
||||
$null = New-Item -ItemType Directory -Path $script:scratch -Force
|
||||
|
||||
$script:emptyStdin = Join-Path $script:scratch 'empty.txt'
|
||||
Set-Content -LiteralPath $script:emptyStdin -Value '' -NoNewline
|
||||
|
||||
function Invoke-EditorWithClosedStdin {
|
||||
<#
|
||||
Runs the editor with stdin from an empty file and a hard timeout.
|
||||
Returns the exit code, or -1 if it had to be killed.
|
||||
#>
|
||||
param(
|
||||
[string[]] $ArgumentList,
|
||||
[int] $TimeoutSeconds = 60
|
||||
)
|
||||
|
||||
$stdout = Join-Path $script:scratch ("out-{0}.txt" -f [guid]::NewGuid().ToString('N'))
|
||||
$stderr = Join-Path $script:scratch ("err-{0}.txt" -f [guid]::NewGuid().ToString('N'))
|
||||
|
||||
$process = Start-Process -FilePath (Get-Process -Id $PID).Path `
|
||||
-ArgumentList (@('-NoProfile', '-NonInteractive', '-File', $script:editor) + $ArgumentList) `
|
||||
-RedirectStandardInput $script:emptyStdin `
|
||||
-RedirectStandardOutput $stdout `
|
||||
-RedirectStandardError $stderr `
|
||||
-PassThru -WindowStyle Hidden
|
||||
|
||||
if (-not $process.WaitForExit($TimeoutSeconds * 1000)) {
|
||||
$process.Kill($true)
|
||||
return [pscustomobject]@{ ExitCode = -1; Output = 'TIMED OUT'; TimedOut = $true }
|
||||
}
|
||||
|
||||
[pscustomobject]@{
|
||||
ExitCode = $process.ExitCode
|
||||
Output = (Get-Content -LiteralPath $stdout -Raw -ErrorAction SilentlyContinue)
|
||||
Error = (Get-Content -LiteralPath $stderr -Raw -ErrorAction SilentlyContinue)
|
||||
TimedOut = $false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Non-interactive mode never prompts or hangs (SC-010)' {
|
||||
|
||||
It 'completes on a valid configuration with stdin closed' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive')
|
||||
|
||||
$run.TimedOut | Should -BeFalse
|
||||
$run.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'completes on an invalid configuration with stdin closed' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.rules[1].id = $document.rules[0].id
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @('-ConfigPath', $path, '-ValidateOnly', '-NonInteractive')
|
||||
|
||||
$run.TimedOut | Should -BeFalse
|
||||
$run.ExitCode | Should -Be 1
|
||||
}
|
||||
|
||||
It 'completes with -NonInteractive alone, without -ValidateOnly' {
|
||||
# The case most likely to regress: -NonInteractive must short-circuit before
|
||||
# the editor loop even when the caller did not also ask for validate-only.
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @('-ConfigPath', $path, '-NonInteractive')
|
||||
|
||||
$run.TimedOut | Should -BeFalse
|
||||
$run.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'completes when the configuration file does not exist' {
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @('-ConfigPath', (Join-Path $script:scratch 'absent.json'), '-NonInteractive')
|
||||
|
||||
$run.TimedOut | Should -BeFalse
|
||||
$run.ExitCode | Should -Be 3
|
||||
}
|
||||
|
||||
It 'completes when running synthetic rule tests' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @(
|
||||
'-ConfigPath', $path, '-NonInteractive', '-TestDataPath', (Join-Path $repoRoot 'tests/TestData'))
|
||||
|
||||
$run.TimedOut | Should -BeFalse
|
||||
$run.ExitCode | Should -Be 0
|
||||
}
|
||||
|
||||
It 'emits no prompt text on the output stream' {
|
||||
$path = Save-TestConfiguration -Document (New-TestConfigurationDocument) -Directory $script:scratch
|
||||
|
||||
$run = Invoke-EditorWithClosedStdin -ArgumentList @('-ConfigPath', $path, '-NonInteractive')
|
||||
|
||||
$run.Output | Should -Not -Match 'Choice'
|
||||
$run.Output | Should -Not -Match 'configuration editor'
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
SC-009, VR-003 half: every safety condition produces a finding.
|
||||
|
||||
Layer 3 asks whether a configuration makes sense. These tests are about layer 4,
|
||||
which asks what happens to the directory if it runs - a coherent configuration can
|
||||
still be a silent no-op or a change nobody declared.
|
||||
|
||||
Several findings change severity with -EnforcementEnabled. Both cases are asserted
|
||||
for each: the same configuration carries very different risk in preview and in
|
||||
enforcement, and a validator that ignored the difference would either block
|
||||
harmless previews or wave through real writes.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:corpus = Join-Path $repoRoot 'tests/TestData/InvalidConfigs'
|
||||
$script:baseline = Join-Path $script:corpus 'baseline-deployed.json'
|
||||
|
||||
function Get-SafetyFinding {
|
||||
param(
|
||||
[string] $Fixture,
|
||||
[string] $Code,
|
||||
[switch] $EnforcementEnabled,
|
||||
[string] $PreviousConfigPath
|
||||
)
|
||||
|
||||
$document = Get-Content -LiteralPath (Join-Path $script:corpus "$Fixture.json") -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
$params = @{ Document = $document; EnforcementEnabled = $EnforcementEnabled }
|
||||
if ($PreviousConfigPath) { $params['PreviousConfigPath'] = $PreviousConfigPath }
|
||||
|
||||
@(Test-PersonaConfigurationSafety @params | Where-Object Code -EQ $Code)
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Safety validation, VR-003 conditions (SC-009)' {
|
||||
|
||||
It 'detects a blank target attribute and blocks it under enforcement' {
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-001-blank-target-production' -Code 'PE-SAF-001' -EnforcementEnabled
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'reports the same blank target attribute as a Warning in preview' {
|
||||
# In preview there is nothing to write, so the configuration is merely
|
||||
# pointless rather than dangerous.
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-001-blank-target-production' -Code 'PE-SAF-001'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Warning'
|
||||
}
|
||||
|
||||
It 'detects an unsupported writable attribute' {
|
||||
# 'department' is authoritative in the sync source. Holding permission to
|
||||
# write it is not the same as owning it.
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-002-unsupported-writable-attribute' -Code 'PE-SAF-002'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Description | Should -Match 'department'
|
||||
}
|
||||
|
||||
It 'detects enabled group rules while group retrieval is disabled' {
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-003-group-rules-without-group-retrieval' -Code 'PE-SAF-003'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Location | Should -Be 'dataSources.groups.enabled'
|
||||
}
|
||||
|
||||
It 'detects a configuration version downgrade against the deployed baseline' {
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-004-version-downgrade' -Code 'PE-SAF-004' `
|
||||
-PreviousConfigPath $script:baseline -EnforcementEnabled
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Location | Should -Be 'configVersion'
|
||||
}
|
||||
|
||||
It 'detects a rule removed without a version change' {
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-005-rule-removed-without-version-change' -Code 'PE-SAF-005' `
|
||||
-PreviousConfigPath $script:baseline -EnforcementEnabled
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Description | Should -Match 'RULE-0900-EMPLOYEE'
|
||||
}
|
||||
|
||||
It 'accepts a rule removal when the version was raised to declare it' {
|
||||
$document = Get-Content -LiteralPath (Join-Path $script:corpus 'PE-SAF-005-rule-removed-without-version-change.json') -Raw | ConvertFrom-Json -Depth 32
|
||||
$document.configVersion = '1.1.0'
|
||||
|
||||
$findings = @(Test-PersonaConfigurationSafety -Document $document -PreviousConfigPath $script:baseline -EnforcementEnabled |
|
||||
Where-Object Code -EQ 'PE-SAF-005')
|
||||
|
||||
$findings.Count | Should -Be 0
|
||||
}
|
||||
|
||||
It 'detects condition tracing enabled without acknowledgement' {
|
||||
$findings = Get-SafetyFinding -Fixture 'PE-SAF-006-tracing-without-acknowledgement' -Code 'PE-SAF-006'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Location | Should -Be 'logging.traceConditionValues'
|
||||
}
|
||||
|
||||
It 'accepts tracing when it is acknowledged in the same configuration' {
|
||||
$document = Get-Content -LiteralPath (Join-Path $script:corpus 'PE-SAF-006-tracing-without-acknowledgement.json') -Raw | ConvertFrom-Json -Depth 32
|
||||
$document.logging | Add-Member -NotePropertyName 'acknowledgeConditionTracing' -NotePropertyValue $true
|
||||
|
||||
@(Test-PersonaConfigurationSafety -Document $document | Where-Object Code -EQ 'PE-SAF-006') | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'detects a save that would overwrite an existing configuration with no backup' {
|
||||
$document = Get-Content -LiteralPath $script:baseline -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
$findings = @(Test-PersonaConfigurationSafety -Document $document -SavePath $script:baseline |
|
||||
Where-Object Code -EQ 'PE-SAF-007')
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'accepts the same save when a backup is planned' {
|
||||
$document = Get-Content -LiteralPath $script:baseline -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
@(Test-PersonaConfigurationSafety -Document $document -SavePath $script:baseline -BackupPlanned |
|
||||
Where-Object Code -EQ 'PE-SAF-007') | Should -BeNullOrEmpty
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'A skipped check is reported as skipped, never as a pass' {
|
||||
|
||||
It 'reports an Information finding when no baseline is supplied' {
|
||||
# Silence would be read as approval. The comparison checks did not run, and
|
||||
# the output has to say so.
|
||||
$document = Get-Content -LiteralPath $script:baseline -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
$findings = @(Test-PersonaConfigurationSafety -Document $document | Where-Object Code -EQ 'PE-SAF-004')
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Information'
|
||||
$findings[0].Description | Should -Match 'not a pass'
|
||||
}
|
||||
|
||||
It 'does not block on the skipped-check notice' {
|
||||
$document = Get-Content -LiteralPath $script:baseline -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
@(Test-PersonaConfigurationSafety -Document $document | Where-Object Severity -EQ 'Error') | Should -BeNullOrEmpty
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Safety findings carry everything VR-004 requires' {
|
||||
|
||||
It 'gives every finding a severity, code, location, description, resolution, and layer' {
|
||||
foreach ($fixture in (Get-ChildItem -Path $script:corpus -Filter 'PE-SAF-*.json')) {
|
||||
$document = Get-Content -LiteralPath $fixture.FullName -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
foreach ($finding in (Test-PersonaConfigurationSafety -Document $document -PreviousConfigPath $script:baseline)) {
|
||||
$finding.Severity | Should -BeIn @('Error', 'Warning', 'Information')
|
||||
$finding.Code | Should -Match '^PE-SAF-\d{3}$'
|
||||
$finding.Location | Should -Not -BeNullOrEmpty
|
||||
$finding.Description | Should -Not -BeNullOrEmpty
|
||||
$finding.SuggestedResolution | Should -Not -BeNullOrEmpty
|
||||
$finding.Layer | Should -Be 'Safety'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
SC-009: every VR-002 condition produces a finding with a code, a severity, and a
|
||||
location.
|
||||
|
||||
Layer 3 is exercised directly rather than through Test-PersonaConfiguration. Most
|
||||
of these defects are also caught by the JSON Schema, so a full-pipeline test would
|
||||
stop at layer 2 and never reach the code under test - it would be asserting that
|
||||
the schema works, which LayerOrdering.Tests.ps1 already does.
|
||||
|
||||
The overlap is deliberate (see the note in Test-PersonaConfigurationSemantic):
|
||||
layer 2 can be bypassed with -SchemaPath, and V-5a showed an unparseable schema
|
||||
passes silently on this build. Anything that can misclassify a privileged account
|
||||
is checked twice.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:corpus = Join-Path $repoRoot 'tests/TestData/InvalidConfigs'
|
||||
|
||||
function Get-Finding {
|
||||
param([string] $Fixture, [string] $Code)
|
||||
|
||||
$document = Get-Content -LiteralPath (Join-Path $script:corpus "$Fixture.json") -Raw | ConvertFrom-Json -Depth 32
|
||||
@(Test-PersonaConfigurationSemantic -Document $document | Where-Object Code -EQ $Code)
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Semantic validation, VR-002 conditions (SC-009)' {
|
||||
|
||||
It 'detects duplicate rule IDs' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-001-duplicate-rule-id' -Code 'PE-SEM-001'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Location | Should -Match 'RULE-0010-GUEST'
|
||||
}
|
||||
|
||||
It 'detects duplicate priorities among enabled rules' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-002-duplicate-priority' -Code 'PE-SEM-002'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Description | Should -Match 'RULE-0010-GUEST'
|
||||
$findings[0].Description | Should -Match 'RULE-0900-EMPLOYEE'
|
||||
}
|
||||
|
||||
It 'detects a configuration with no enabled rules' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-003-no-enabled-rules' -Code 'PE-SEM-003'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects a blank target attribute' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-004-blank-target-attribute' -Code 'PE-SEM-004'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Location | Should -Be 'engine.targetAttribute'
|
||||
}
|
||||
|
||||
It 'detects a target attribute absent from the approved list' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-005-target-not-approved' -Code 'PE-SEM-005'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects a reference to a disabled data source' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-006-unavailable-data-source' -Code 'PE-SEM-006'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Description | Should -Match 'EvaluationError'
|
||||
}
|
||||
|
||||
It 'detects memberOf with no group Object IDs' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-007-memberof-without-groups' -Code 'PE-SEM-007'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects in without a values array' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-008-in-without-values' -Code 'PE-SEM-008'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects isNotNull carrying a comparison value' {
|
||||
# The value is silently ignored at evaluation time, so the rule does not do
|
||||
# what the author plainly intended it to do.
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-009-isnull-with-value' -Code 'PE-SEM-009'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects a persona absent from the declared catalogue' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-010-undeclared-persona' -Code 'PE-SEM-010'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Description | Should -Match 'Undeclared-Persona'
|
||||
}
|
||||
|
||||
It 'detects Unclassified used as an ordinary rule persona' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-011-unclassified-as-persona' -Code 'PE-SEM-011'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'detects nesting deeper than the configured maximum' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-012-depth-over-configured-maximum' -Code 'PE-SEM-012'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
$findings[0].Location | Should -Match 'conditions'
|
||||
}
|
||||
|
||||
It 'detects a configured maximum above the hard ceiling of 10' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-013-depth-over-hard-ceiling' -Code 'PE-SEM-013'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Location | Should -Be 'engine.maxConditionDepth'
|
||||
}
|
||||
|
||||
It 'warns when a condition mode differs from an explicitly pinned global mode' {
|
||||
# A Warning, not an Error: the three facets are retrieved independently, so
|
||||
# the condition is answered correctly. The cost is an extra call per account,
|
||||
# which is worth surfacing but not worth blocking.
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-014-mode-not-enabled-globally' -Code 'PE-SEM-014'
|
||||
|
||||
$findings.Count | Should -BeGreaterThan 0
|
||||
$findings[0].Severity | Should -Be 'Warning'
|
||||
}
|
||||
|
||||
It 'detects an unsupported property name' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-015-unsupported-property' -Code 'PE-SEM-015'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Description | Should -Match 'employeeHireDate'
|
||||
}
|
||||
|
||||
It 'detects an invalid regular expression' {
|
||||
$findings = Get-Finding -Fixture 'PE-SEM-016-invalid-regex' -Code 'PE-SEM-016'
|
||||
|
||||
$findings.Count | Should -Be 1
|
||||
$findings[0].Severity | Should -Be 'Error'
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'Semantic findings carry everything VR-004 requires' {
|
||||
|
||||
It 'gives every finding a severity, code, location, description, resolution, and layer' {
|
||||
$fixtures = Get-ChildItem -Path $script:corpus -Filter 'PE-SEM-*.json'
|
||||
$fixtures.Count | Should -BeGreaterThan 0
|
||||
|
||||
foreach ($fixture in $fixtures) {
|
||||
$document = Get-Content -LiteralPath $fixture.FullName -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
foreach ($finding in (Test-PersonaConfigurationSemantic -Document $document)) {
|
||||
$finding.Severity | Should -BeIn @('Error', 'Warning', 'Information')
|
||||
$finding.Code | Should -Match '^PE-SEM-\d{3}$'
|
||||
$finding.Location | Should -Not -BeNullOrEmpty
|
||||
$finding.Description | Should -Not -BeNullOrEmpty
|
||||
$finding.SuggestedResolution | Should -Not -BeNullOrEmpty
|
||||
$finding.Layer | Should -Be 'Semantic'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
It 'produces no findings for the valid baseline' {
|
||||
# Without this, a validator that flagged everything would pass every test above.
|
||||
$document = Get-Content -LiteralPath (Join-Path $script:corpus 'baseline-deployed.json') -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
@(Test-PersonaConfigurationSemantic -Document $document) | Should -BeNullOrEmpty
|
||||
}
|
||||
|
||||
It 'produces no findings for the shipped example configuration' {
|
||||
$document = Get-Content -LiteralPath (Join-Path $repoRoot 'config/persona-engine.example.json') -Raw | ConvertFrom-Json -Depth 32
|
||||
|
||||
@(Test-PersonaConfigurationSemantic -Document $document) | Should -BeNullOrEmpty
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
Pins the V-5a observation about Test-Json -SchemaFile.
|
||||
|
||||
OTD-005 chose a cmdlet whose failure reporting is version-dependent, and
|
||||
validation layer 2 is written against the behaviour observed on PowerShell 7.6.5
|
||||
(specs/001-persona-engine/verification/V-5a.md). If a future build changes any row
|
||||
of that table, the wrapper's assumptions change with it - and the failure mode is
|
||||
silent: configurations start passing validation that should not.
|
||||
|
||||
These tests fail loudly at that moment instead.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
|
||||
$script:schemaFile = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-pin-{0}.json" -f [guid]::NewGuid().ToString('N'))
|
||||
Set-Content -LiteralPath $script:schemaFile -Encoding utf8NoBOM -Value @'
|
||||
{ "type": "object", "required": ["a"], "properties": { "a": { "type": "string" } } }
|
||||
'@
|
||||
|
||||
$script:brokenSchemaFile = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-pin-broken-{0}.json" -f [guid]::NewGuid().ToString('N'))
|
||||
Set-Content -LiteralPath $script:brokenSchemaFile -Encoding utf8NoBOM -Value '{ not json'
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:schemaFile, $script:brokenSchemaFile -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Test-Json -SchemaFile failure behaviour (V-5a)' {
|
||||
|
||||
It 'returns $true and writes nothing for a valid document' {
|
||||
$errors = $null
|
||||
$result = '{ "a": "ok" }' | Test-Json -SchemaFile $script:schemaFile -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
$result | Should -BeTrue
|
||||
@($errors).Count | Should -Be 0
|
||||
}
|
||||
|
||||
It 'returns $false and writes one error for a type mismatch' {
|
||||
$errors = $null
|
||||
$result = '{ "a": 123 }' | Test-Json -SchemaFile $script:schemaFile -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
$result | Should -BeFalse
|
||||
@($errors).Count | Should -Be 1
|
||||
}
|
||||
|
||||
It 'returns $false and writes one error for a missing required property' {
|
||||
$errors = $null
|
||||
$result = '{}' | Test-Json -SchemaFile $script:schemaFile -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
$result | Should -BeFalse
|
||||
@($errors).Count | Should -Be 1
|
||||
}
|
||||
|
||||
It 'reports failure without throwing, so -ErrorVariable is sufficient' {
|
||||
# If a future build made this terminating, layer 2 would abort the run instead
|
||||
# of returning findings, and the editor would report an exception rather than
|
||||
# a PE-SCH finding.
|
||||
{ '{ "a": 123 }' | Test-Json -SchemaFile $script:schemaFile -ErrorAction SilentlyContinue } | Should -Not -Throw
|
||||
}
|
||||
|
||||
It 'returns $TRUE when the schema itself cannot be parsed - the trap layer 2 is built around' {
|
||||
# The load-bearing observation. A wrapper trusting the return value alone
|
||||
# would report every configuration as schema-valid against a schema that
|
||||
# never ran.
|
||||
$errors = $null
|
||||
$result = '{}' | Test-Json -SchemaFile $script:brokenSchemaFile -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
$result | Should -BeTrue
|
||||
@($errors).Count | Should -Be 1
|
||||
$errors[0].Exception.Message | Should -Match 'Cannot parse the JSON schema'
|
||||
}
|
||||
|
||||
It 'embeds a JSON pointer in the failure message, which VR-004 needs for Location' {
|
||||
$errors = $null
|
||||
$null = '{ "a": 123 }' | Test-Json -SchemaFile $script:schemaFile -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
$errors[0].Exception.Message | Should -Match "at '/a'"
|
||||
}
|
||||
|
||||
It 'reports one error per violating location when several properties fail' {
|
||||
# Not exhaustive across nested subschemas, but not first-failure-only either.
|
||||
# Layer 2 therefore emits one finding per collected error rather than assuming
|
||||
# a single one.
|
||||
$multiSchema = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-pin-multi-{0}.json" -f [guid]::NewGuid().ToString('N'))
|
||||
Set-Content -LiteralPath $multiSchema -Encoding utf8NoBOM -Value @'
|
||||
{ "type": "object", "required": ["a", "b"], "properties": { "a": { "type": "string" }, "b": { "type": "string" } } }
|
||||
'@
|
||||
try {
|
||||
$errors = $null
|
||||
$null = '{ "a": 1, "b": 2 }' | Test-Json -SchemaFile $multiSchema -ErrorAction SilentlyContinue -ErrorVariable errors
|
||||
|
||||
@($errors).Count | Should -Be 2
|
||||
($errors | ForEach-Object { $_.Exception.Message }) -join ' ' | Should -Match "at '/a'"
|
||||
($errors | ForEach-Object { $_.Exception.Message }) -join ' ' | Should -Match "at '/b'"
|
||||
}
|
||||
finally { Remove-Item -LiteralPath $multiSchema -Force -ErrorAction SilentlyContinue }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
#Requires -Modules @{ ModuleName = 'Pester'; ModuleVersion = '5.0.0' }
|
||||
|
||||
<#
|
||||
VR-003: traceConditionValues without explicit acknowledgement is a safety finding.
|
||||
|
||||
The acknowledgement lives in the configuration rather than in a command-line
|
||||
switch, and that placement is the point. A flag passed at the console is invisible
|
||||
to review; a field in the configuration appears in the diff of the change that
|
||||
enables tracing, next to the person who approved it.
|
||||
#>
|
||||
|
||||
BeforeAll {
|
||||
$repoRoot = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
|
||||
. (Join-Path $repoRoot 'tests/TestHelpers.ps1')
|
||||
foreach ($file in (Get-PersonaSourceFile -RepoRoot $repoRoot)) { . $file }
|
||||
|
||||
$script:schema = Join-Path $repoRoot 'config/persona-engine.schema.json'
|
||||
$script:scratch = Join-Path ([System.IO.Path]::GetTempPath()) ("pe-trace-{0}" -f [guid]::NewGuid().ToString('N'))
|
||||
$null = New-Item -ItemType Directory -Path $script:scratch -Force
|
||||
}
|
||||
|
||||
AfterAll {
|
||||
Remove-Item -LiteralPath $script:scratch -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
Describe 'Tracing acknowledgement (VR-003)' {
|
||||
|
||||
It 'produces PE-SAF-006 when tracing is on and acknowledgement is absent' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $true }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeFalse
|
||||
$result.Findings.Code | Should -Contain 'PE-SAF-006'
|
||||
}
|
||||
|
||||
It 'produces PE-SAF-006 when acknowledgement is present but false' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $true; acknowledgeConditionTracing = $false }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.Findings.Code | Should -Contain 'PE-SAF-006'
|
||||
}
|
||||
|
||||
It 'accepts tracing when acknowledgement is true' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $true; acknowledgeConditionTracing = $true }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.Findings.Code | Should -Not -Contain 'PE-SAF-006'
|
||||
}
|
||||
|
||||
It 'does not require acknowledgement when tracing is off' {
|
||||
# Acknowledging something that is not happening would train people to set the
|
||||
# field reflexively, which is how an acknowledgement stops meaning anything.
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $false }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.IsValid | Should -BeTrue
|
||||
$result.Findings.Code | Should -Not -Contain 'PE-SAF-006'
|
||||
}
|
||||
|
||||
It 'blocks the run: the finding is an Error, not a Warning' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $true }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
($result.Findings | Where-Object Code -EQ 'PE-SAF-006').Severity | Should -Be 'Error'
|
||||
}
|
||||
|
||||
It 'explains what tracing actually widens, not merely that it is enabled' {
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'stream'; traceConditionValues = $true }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$finding = (Test-PersonaConfiguration -Path $path -SchemaPath $script:schema).Findings |
|
||||
Where-Object Code -EQ 'PE-SAF-006'
|
||||
|
||||
$finding.Description | Should -Match 'attribute values'
|
||||
$finding.SuggestedResolution | Should -Match 'acknowledgeConditionTracing'
|
||||
}
|
||||
}
|
||||
|
||||
Describe 'The schema accepts the acknowledgement field' {
|
||||
|
||||
It 'validates a configuration carrying acknowledgeConditionTracing' {
|
||||
# additionalProperties is false on the logging block, so the field has to be
|
||||
# declared in the schema or the acknowledgement itself becomes a schema error.
|
||||
$document = New-TestConfigurationDocument
|
||||
$document.logging = @{ destination = 'both'; path = '<LOG-OUTPUT-PATH>'; traceConditionValues = $true; acknowledgeConditionTracing = $true }
|
||||
|
||||
$path = Save-TestConfiguration -Document $document -Directory $script:scratch
|
||||
$result = Test-PersonaConfiguration -Path $path -SchemaPath $script:schema
|
||||
|
||||
$result.Findings.Code | Should -Not -Contain 'PE-SCH-001'
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user