dave
cdc6bb33d3
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8.6 KiB
8.6 KiB
Requirements traceability
Every functional requirement, non-functional requirement, and success criterion in spec.md, mapped to the code that implements it and the test that holds it there.
A row with no test is a requirement nobody is checking. Those are listed explicitly at the bottom rather than left out, because an incomplete matrix that looks complete is worse than no matrix.
Status as at 2026-08-20: 354 offline tests passing; engine purity and sanitization gates passing; no tenant-dependent item verified.
Functional requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| FR-001 | Load JSON configuration | Import-PersonaConfiguration |
LayerOrdering.Tests.ps1 |
| FR-002 | Validate before connecting | Test-PersonaConfiguration; entry script exits 1 before Connect- |
LayerOrdering.Tests.ps1, ExitCodes.Tests.ps1 |
| FR-003 | Adapter-isolated authentication | Connect-PersonaGraphInteractive |
Purity gate; ShouldProcessGate.Tests.ps1 |
| FR-004 | Enumerate all users with pagination | Get-PersonaUsers |
Pagination.Tests.ps1 |
| FR-005 | Select only required properties | Get-PersonaRequiredProperties |
Pagination.Tests.ps1 |
| FR-006 | Retrieve and cache related data | Get-PersonaGroupMembership, Get-PersonaCachedMembership |
Pagination.Tests.ps1, OutcomeExclusivity.Tests.ps1 |
| FR-007 | Normalize before evaluation | ConvertTo-PersonaUserRecord, ConvertTo-PersonaMembershipRecord |
RecordContracts.Tests.ps1 |
| FR-008 | Evaluate rules in priority order | Resolve-UserPersona |
Ordering.Tests.ps1 |
| FR-009 | Stop at first match | Resolve-UserPersona |
Ordering.Tests.ps1 |
| FR-010 | Unclassified when nothing matches |
Resolve-UserPersona |
Unclassified.Tests.ps1 |
| FR-011 | Disabled accounts stay in scope | New-PersonaUserRecord exposes AccountEnabled |
Operators.Tests.ps1 |
| FR-012 | Null treated as empty | Test-PersonaCondition |
Operators.Tests.ps1 |
| FR-013 | Unretrievable group data yields EvaluationError |
Tri-state evaluation; facet retrieval flags | UnknownNotFalse.Tests.ps1, UnknownPropagation.Tests.ps1 |
| FR-014 | Preserve stored value on failure | Compare-PersonaValue sets Skipped first |
EvaluationError.Tests.ps1, WriteGate.Tests.ps1 |
| FR-015 | Compare stored and calculated | Compare-PersonaValue, ordinal |
WriteGate.Tests.ps1 |
| FR-016 | Write only changed values, four conditions | Compare-PersonaValue + run-loop gate |
WriteGate.Tests.ps1 |
| FR-017 | Preview issues no write request | Invoke-PersonaEngineRun gate |
WhatIfZeroWrites.Tests.ps1 |
| FR-018 | Immediate per-user output | Write-UserPersonaResult |
Exercised by every run-loop suite |
| FR-019 | Periodic summary | Write-PersonaSummary, interval check |
SummaryInterval.Tests.ps1 |
| FR-020 | Interval semantics, final always shown | Invoke-PersonaEngineRun |
SummaryInterval.Tests.ps1 |
| FR-021 | Reconciliation at every summary | Test-PersonaReconciliation |
Reconciliation.Tests.ps1 |
| FR-022 | Structured audit records | New-PersonaAuditRecord, Write-PersonaAuditRecord |
AuditRecordShape.Tests.ps1 |
| FR-023 | Configuration editor | Edit-PersonaEngineConfig.ps1 |
ExitCodes.Tests.ps1 (editor) |
| FR-024 | Non-interactive validation with exit codes | -NonInteractive short-circuit |
NonInteractive.Tests.ps1 |
| FR-025 | Synthetic rule testing, no tenant | Invoke-SyntheticRuleTest |
NonInteractive.Tests.ps1 |
| FR-026 | Validate and back up before save | Save-PersonaConfiguration |
Safety.Tests.ps1 (PE-SAF-007) |
Rule engine requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| RE-001 | Required rule fields | Schema definitions/rule |
LayerOrdering.Tests.ps1 |
| RE-002 | Unique priorities, lower first | Resolve-UserPersona; PE-SEM-002 |
Ordering.Tests.ps1, Semantic.Tests.ps1 |
| RE-003 | all / any with nesting |
Test-PersonaConditionGroup |
Composition.Tests.ps1 |
| RE-004 | Depth limit and hard ceiling | Test-PersonaConditionGroup; PE-SEM-012, PE-SEM-013 |
Composition.Tests.ps1, Semantic.Tests.ps1 |
| RE-005 | Thirteen operators | Test-PersonaCondition |
Operators.Tests.ps1 |
| RE-006 | Case-insensitive; regex validated first | Test-PersonaCondition; PE-SEM-016 |
Operators.Tests.ps1, Semantic.Tests.ps1 |
| RE-007 | Per-condition membership mode | Three-facet MembershipRecord |
RecordContracts.Tests.ps1, UnknownPropagation.Tests.ps1 |
| RE-008 | Combined identity sources | Get-PersonaRequiredFacets |
OutcomeExclusivity.Tests.ps1 |
| RE-009 | Special accounts by Object ID | Example configuration; no hard-coded path | Purity gate |
Validation requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| VR-001 | Four ordered layers, fail-fast | Test-PersonaConfiguration |
LayerOrdering.Tests.ps1 |
| VR-002 | Sixteen semantic conditions | Test-PersonaConfigurationSemantic |
Semantic.Tests.ps1 — one test per code |
| VR-003 | Seven safety conditions | Test-PersonaConfigurationSafety |
Safety.Tests.ps1, TraceAcknowledgement.Tests.ps1 |
| VR-004 | Finding shape | New-PersonaValidationFinding |
Semantic.Tests.ps1, Safety.Tests.ps1, RecordContracts.Tests.ps1 |
| VR-005 | Warnings block only on request | Editor exit-code mapping | ExitCodes.Tests.ps1 (editor) |
Non-functional requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| NFR-001 | PowerShell 7 | #Requires -Version 7.2; manifest floor |
Runs on 7.6.5 |
| NFR-002 | Caching, per-user and total duration | New-PersonaDataCache; stopwatch in Resolve-UserPersona; RunComplete.durationMs |
AuditCompleteness.Tests.ps1 — no target set |
| NFR-003 | Pagination, bounded retry, backoff | Invoke-PersonaGraphRequest |
RetryPolicy.Tests.ps1, Pagination.Tests.ps1 |
| NFR-004 | Comment-based help on public functions | Every function in src/ |
Manual review |
| NFR-005 | Run ID, UPN, Object ID, config hash | New-PersonaAuditContext |
AuditCompleteness.Tests.ps1 |
| NFR-006 | Least privilege, single attribute, no secrets | Six OTD-003 controls | WriteBody.Tests.ps1, WriteBodyRejection.Tests.ps1, sanitization gate |
| NFR-007 | Engine and validation run without the platform | Purity; layer dot-sourcing | Purity gate; whole offline suite |
| NFR-008 | No Windows PowerShell-only dependencies | Microsoft.Graph.Authentication only |
Unverified in Automation (V-5b) |
Success criteria
| ID | Criterion | Test | Status |
|---|---|---|---|
| SC-001 | Exactly one outcome per user | OutcomeExclusivity.Tests.ps1 |
Passing |
| SC-002 | Second run proposes zero changes | Idempotence.Tests.ps1 |
Passing |
| SC-003 | Determinism across shuffled input | Determinism.Tests.ps1 |
Passing |
| SC-004 | Zero writes under -WhatIf |
WhatIfZeroWrites.Tests.ps1 |
Passing |
| SC-005 | Single-attribute request body | WriteBody.Tests.ps1 |
Passing |
| SC-006 | Audit completeness | AuditCompleteness.Tests.ps1 |
Passing |
| SC-007 | Reconciliation, and its failure path | Reconciliation.Tests.ps1 |
Passing |
| SC-008 | Everything runs offline | Whole offline suite; validate.yml gate 7 |
Passing |
| SC-009 | Every VR-002 and VR-003 condition detected | Semantic.Tests.ps1, Safety.Tests.ps1 |
Passing |
| SC-010 | Non-interactive never prompts or hangs | NonInteractive.Tests.ps1 |
Passing |
| SC-011 | Every exit code reachable | ExitCodes.Tests.ps1 (both) |
Passing |
| SC-012 | Per-user output is immediate | Write-UserPersonaResult emits in process |
Structural, not timed |
| SC-013 | No tenant data committed | Test-Sanitization.ps1 |
Passing |
Gaps, stated plainly
| Item | Why it is not covered | What would close it |
|---|---|---|
| NFR-002 performance | No target exists until representative tenant testing | A timed run against a real population |
| NFR-004 help coverage | Reviewed by eye, not asserted | A test parsing every exported function for a help block |
| NFR-008 Automation compatibility | No Automation account available | V-5b (T116) |
| SC-012 timing | Asserted structurally, not measured | A timed harness — low value against the cost |
| V-1, V-2, V-3 | Require a tenant | Stage A2 and A3 runs |
| V-4 | Requires a person | Written security sign-off |
| Phase 13 (T115–T121) | Requires an Automation account | Stage B |
How to keep this honest
When a requirement's implementation moves, this table moves with it. When a test is deleted, the row it backed becomes a gap and belongs in the gaps table, not silently in the main one. A matrix that is allowed to drift is worse than none, because it converts "we do not know" into "we checked".