9.5 KiB
9.5 KiB
Requirements traceability
Every functional requirement, non-functional requirement, and success criterion in spec.md, mapped to the code that implements it and the test that holds it there.
A row with no test is a requirement nobody is checking. Those are listed explicitly at the bottom rather than left out, because an incomplete matrix that looks complete is worse than no matrix.
Status as at 2026-08-21: 400 offline tests passing; engine purity and sanitization gates passing; no tenant-dependent item verified. The increase from 354 is FR-027–FR-030 (rule add/edit/delete in the interactive editor), added to spec.md and implemented in the same change (T122–T129).
Functional requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| FR-001 | Load JSON configuration | Import-PersonaConfiguration |
LayerOrdering.Tests.ps1 |
| FR-002 | Validate before connecting | Test-PersonaConfiguration; entry script exits 1 before Connect- |
LayerOrdering.Tests.ps1, ExitCodes.Tests.ps1 |
| FR-003 | Adapter-isolated authentication | Connect-PersonaGraphInteractive |
Purity gate; ShouldProcessGate.Tests.ps1 |
| FR-004 | Enumerate all users with pagination | Get-PersonaUsers |
Pagination.Tests.ps1 |
| FR-005 | Select only required properties | Get-PersonaRequiredProperties |
Pagination.Tests.ps1 |
| FR-006 | Retrieve and cache related data | Get-PersonaGroupMembership, Get-PersonaCachedMembership |
Pagination.Tests.ps1, OutcomeExclusivity.Tests.ps1 |
| FR-007 | Normalize before evaluation | ConvertTo-PersonaUserRecord, ConvertTo-PersonaMembershipRecord |
RecordContracts.Tests.ps1 |
| FR-008 | Evaluate rules in priority order | Resolve-UserPersona |
Ordering.Tests.ps1 |
| FR-009 | Stop at first match | Resolve-UserPersona |
Ordering.Tests.ps1 |
| FR-010 | Unclassified when nothing matches |
Resolve-UserPersona |
Unclassified.Tests.ps1 |
| FR-011 | Disabled accounts stay in scope | New-PersonaUserRecord exposes AccountEnabled |
Operators.Tests.ps1 |
| FR-012 | Null treated as empty | Test-PersonaCondition |
Operators.Tests.ps1 |
| FR-013 | Unretrievable group data yields EvaluationError |
Tri-state evaluation; facet retrieval flags | UnknownNotFalse.Tests.ps1, UnknownPropagation.Tests.ps1 |
| FR-014 | Preserve stored value on failure | Compare-PersonaValue sets Skipped first |
EvaluationError.Tests.ps1, WriteGate.Tests.ps1 |
| FR-015 | Compare stored and calculated | Compare-PersonaValue, ordinal |
WriteGate.Tests.ps1 |
| FR-016 | Write only changed values, four conditions | Compare-PersonaValue + run-loop gate |
WriteGate.Tests.ps1 |
| FR-017 | Preview issues no write request | Invoke-PersonaEngineRun gate |
WhatIfZeroWrites.Tests.ps1 |
| FR-018 | Immediate per-user output | Write-UserPersonaResult |
Exercised by every run-loop suite |
| FR-019 | Periodic summary | Write-PersonaSummary, interval check |
SummaryInterval.Tests.ps1 |
| FR-020 | Interval semantics, final always shown | Invoke-PersonaEngineRun |
SummaryInterval.Tests.ps1 |
| FR-021 | Reconciliation at every summary | Test-PersonaReconciliation |
Reconciliation.Tests.ps1 |
| FR-022 | Structured audit records | New-PersonaAuditRecord, Write-PersonaAuditRecord |
AuditRecordShape.Tests.ps1 |
| FR-023 | Configuration editor | Edit-PersonaEngineConfig.ps1 |
ExitCodes.Tests.ps1 (editor) |
| FR-024 | Non-interactive validation with exit codes | -NonInteractive short-circuit |
NonInteractive.Tests.ps1 |
| FR-025 | Synthetic rule testing, no tenant | Invoke-SyntheticRuleTest |
NonInteractive.Tests.ps1 |
| FR-026 | Validate and back up before save | Save-PersonaConfiguration |
Safety.Tests.ps1 (PE-SAF-007) |
| FR-027 | Add a rule interactively | Add-PersonaConfigRule; [A] command in Edit-PersonaEngineConfig.ps1 |
EditorAddRule.Tests.ps1, EditorStructuralEdits.Tests.ps1 |
| FR-028 | Edit a rule interactively, including its condition tree | Get/Add/Remove-PersonaConditionNode, Set-PersonaConditionLeaf; [E] command |
EditorEditRule.Tests.ps1, EditorStructuralEdits.Tests.ps1 |
| FR-029 | Delete a rule interactively, with confirmation | Remove-PersonaConfigRule; [D] command |
EditorDeleteRule.Tests.ps1, EditorStructuralEdits.Tests.ps1 |
| FR-030 | Structural edits validated like a hand-edited file | Test-PersonaCandidateEdit in Edit-PersonaEngineConfig.ps1 |
EditorStructuralEdits.Tests.ps1 |
Rule engine requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| RE-001 | Required rule fields | Schema definitions/rule |
LayerOrdering.Tests.ps1 |
| RE-002 | Unique priorities, lower first | Resolve-UserPersona; PE-SEM-002 |
Ordering.Tests.ps1, Semantic.Tests.ps1 |
| RE-003 | all / any with nesting |
Test-PersonaConditionGroup |
Composition.Tests.ps1 |
| RE-004 | Depth limit and hard ceiling | Test-PersonaConditionGroup; PE-SEM-012, PE-SEM-013 |
Composition.Tests.ps1, Semantic.Tests.ps1 |
| RE-005 | Thirteen operators | Test-PersonaCondition |
Operators.Tests.ps1 |
| RE-006 | Case-insensitive; regex validated first | Test-PersonaCondition; PE-SEM-016 |
Operators.Tests.ps1, Semantic.Tests.ps1 |
| RE-007 | Per-condition membership mode | Three-facet MembershipRecord |
RecordContracts.Tests.ps1, UnknownPropagation.Tests.ps1 |
| RE-008 | Combined identity sources | Get-PersonaRequiredFacets |
OutcomeExclusivity.Tests.ps1 |
| RE-009 | Special accounts by Object ID | Example configuration; no hard-coded path | Purity gate |
Validation requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| VR-001 | Four ordered layers, fail-fast | Test-PersonaConfiguration |
LayerOrdering.Tests.ps1 |
| VR-002 | Sixteen semantic conditions | Test-PersonaConfigurationSemantic |
Semantic.Tests.ps1 — one test per code |
| VR-003 | Seven safety conditions | Test-PersonaConfigurationSafety |
Safety.Tests.ps1, TraceAcknowledgement.Tests.ps1 |
| VR-004 | Finding shape | New-PersonaValidationFinding |
Semantic.Tests.ps1, Safety.Tests.ps1, RecordContracts.Tests.ps1 |
| VR-005 | Warnings block only on request | Editor exit-code mapping | ExitCodes.Tests.ps1 (editor) |
Non-functional requirements
| ID | Requirement | Implementation | Test |
|---|---|---|---|
| NFR-001 | PowerShell 7 | #Requires -Version 7.2; manifest floor |
Runs on 7.6.5 |
| NFR-002 | Caching, per-user and total duration | New-PersonaDataCache; stopwatch in Resolve-UserPersona; RunComplete.durationMs |
AuditCompleteness.Tests.ps1 — no target set |
| NFR-003 | Pagination, bounded retry, backoff | Invoke-PersonaGraphRequest |
RetryPolicy.Tests.ps1, Pagination.Tests.ps1 |
| NFR-004 | Comment-based help on public functions | Every function in src/ |
Manual review |
| NFR-005 | Run ID, UPN, Object ID, config hash | New-PersonaAuditContext |
AuditCompleteness.Tests.ps1 |
| NFR-006 | Least privilege, single attribute, no secrets | Six OTD-003 controls | WriteBody.Tests.ps1, WriteBodyRejection.Tests.ps1, sanitization gate |
| NFR-007 | Engine and validation run without the platform | Purity; layer dot-sourcing | Purity gate; whole offline suite |
| NFR-008 | No Windows PowerShell-only dependencies | Microsoft.Graph.Authentication only |
Unverified in Automation (V-5b) |
Success criteria
| ID | Criterion | Test | Status |
|---|---|---|---|
| SC-001 | Exactly one outcome per user | OutcomeExclusivity.Tests.ps1 |
Passing |
| SC-002 | Second run proposes zero changes | Idempotence.Tests.ps1 |
Passing |
| SC-003 | Determinism across shuffled input | Determinism.Tests.ps1 |
Passing |
| SC-004 | Zero writes under -WhatIf |
WhatIfZeroWrites.Tests.ps1 |
Passing |
| SC-005 | Single-attribute request body | WriteBody.Tests.ps1 |
Passing |
| SC-006 | Audit completeness | AuditCompleteness.Tests.ps1 |
Passing |
| SC-007 | Reconciliation, and its failure path | Reconciliation.Tests.ps1 |
Passing |
| SC-008 | Everything runs offline | Whole offline suite; validate.yml gate 7 |
Passing |
| SC-009 | Every VR-002 and VR-003 condition detected | Semantic.Tests.ps1, Safety.Tests.ps1 |
Passing |
| SC-010 | Non-interactive never prompts or hangs | NonInteractive.Tests.ps1 |
Passing |
| SC-011 | Every exit code reachable | ExitCodes.Tests.ps1 (both) |
Passing |
| SC-012 | Per-user output is immediate | Write-UserPersonaResult emits in process |
Structural, not timed |
| SC-013 | No tenant data committed | Test-Sanitization.ps1 |
Passing |
Gaps, stated plainly
| Item | Why it is not covered | What would close it |
|---|---|---|
| NFR-002 performance | No target exists until representative tenant testing | A timed run against a real population |
| NFR-004 help coverage | Reviewed by eye, not asserted | A test parsing every exported function for a help block |
| NFR-008 Automation compatibility | No Automation account available | V-5b (T116) |
| SC-012 timing | Asserted structurally, not measured | A timed harness — low value against the cost |
| V-1, V-2, V-3 | Require a tenant | Stage A2 and A3 runs |
| V-4 | Requires a person | Written security sign-off |
| Phase 13 (T115–T121) | Requires an Automation account | Stage B |
How to keep this honest
When a requirement's implementation moves, this table moves with it. When a test is deleted, the row it backed becomes a gap and belongs in the gaps table, not silently in the main one. A matrix that is allowed to drift is worse than none, because it converts "we do not know" into "we checked".