dave
cdc6bb33d3
Completes 109 of 121 tasks. Every remaining task needs a tenant connection
(T055, T056, T101-T103) or an Azure Automation account (T115-T121).
354 offline Pester tests PASS
Engine purity (Principle IV) PASS
Sanitization (SC-013) PASS (156 files)
Graph module loaded in tests none (SC-008 holds)
What landed
- Four-layer configuration validation with stable finding codes, covering
every VR-002 and VR-003 condition, plus a 23-fixture invalid-config corpus
- Run loop, audit records (NDJSON through a single sink), summaries,
reconciliation, and exit codes 0-6
- Persistence behind a single write-body builder whose result always has
exactly one key
- Invoke-PersonaEngine.ps1 and Edit-PersonaEngineConfig.ps1
- Six docs, two pipelines, traceability matrix, V-5a and sanitization records
Three deviations from tasks.md, each recorded in its status block
T033 is not in Resolve-UserPersona. evaluationErrorThreshold is run-level
state and the rule engine is pure; a counter there would break Principle IV.
It lives in New-PersonaRunCounter and is applied in the run loop.
A new src/Engine/ layer holds Invoke-PersonaEngineRun. The entry script
imports the manifest, which requires Microsoft.Graph.Authentication, so a
loop living only inside it could not run on a machine without the Graph SDK
and SC-004 could not be proven at all. The entry script is now a thin
wrapper and what ships is what is tested.
The invalid-config corpus is generated by a committed script, with the
generated fixtures committed too, so a reviewer sees the fixture in the diff.
Defects found by running the code, not by reading it
Group and role ID lists were double-wrapped: @(Get-PersonaGroupIdPage ...)
around a comma-returned array collapsed every membership list into one
bogus space-joined entry. That is a silent false non-match, exactly what
FR-013 exists to prevent.
A 403 whose status appears only in the exception message parsed as $null,
which the retry policy treats as a transport error - five requests per
account against a tenant already refusing. Status extraction now falls back
to the message text, bounded to 400-599.
The sanitization scan walked tracked files only, so it covered 34 of 156
files and none of this phase's code. It now scans untracked non-ignored
files too, and a negative control confirms it catches a planted leak.
Test-Json reports one error per violating location, not first-failure-only
as the V-5a draft claimed. Record and pin corrected.
Enforcement remains blocked on the V-4 security sign-off.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
115 lines
8.6 KiB
Markdown
115 lines
8.6 KiB
Markdown
# Requirements traceability
|
||
|
||
Every functional requirement, non-functional requirement, and success criterion in
|
||
[spec.md](spec.md), mapped to the code that implements it and the test that holds it there.
|
||
|
||
A row with no test is a requirement nobody is checking. Those are listed explicitly at the bottom
|
||
rather than left out, because an incomplete matrix that looks complete is worse than no matrix.
|
||
|
||
**Status as at 2026-08-20**: 354 offline tests passing; engine purity and sanitization gates passing;
|
||
no tenant-dependent item verified.
|
||
|
||
## Functional requirements
|
||
|
||
| ID | Requirement | Implementation | Test |
|
||
| --- | --- | --- | --- |
|
||
| FR-001 | Load JSON configuration | `Import-PersonaConfiguration` | `LayerOrdering.Tests.ps1` |
|
||
| FR-002 | Validate before connecting | `Test-PersonaConfiguration`; entry script exits 1 before `Connect-` | `LayerOrdering.Tests.ps1`, `ExitCodes.Tests.ps1` |
|
||
| FR-003 | Adapter-isolated authentication | `Connect-PersonaGraphInteractive` | Purity gate; `ShouldProcessGate.Tests.ps1` |
|
||
| FR-004 | Enumerate all users with pagination | `Get-PersonaUsers` | `Pagination.Tests.ps1` |
|
||
| FR-005 | Select only required properties | `Get-PersonaRequiredProperties` | `Pagination.Tests.ps1` |
|
||
| FR-006 | Retrieve and cache related data | `Get-PersonaGroupMembership`, `Get-PersonaCachedMembership` | `Pagination.Tests.ps1`, `OutcomeExclusivity.Tests.ps1` |
|
||
| FR-007 | Normalize before evaluation | `ConvertTo-PersonaUserRecord`, `ConvertTo-PersonaMembershipRecord` | `RecordContracts.Tests.ps1` |
|
||
| FR-008 | Evaluate rules in priority order | `Resolve-UserPersona` | `Ordering.Tests.ps1` |
|
||
| FR-009 | Stop at first match | `Resolve-UserPersona` | `Ordering.Tests.ps1` |
|
||
| FR-010 | `Unclassified` when nothing matches | `Resolve-UserPersona` | `Unclassified.Tests.ps1` |
|
||
| FR-011 | Disabled accounts stay in scope | `New-PersonaUserRecord` exposes `AccountEnabled` | `Operators.Tests.ps1` |
|
||
| FR-012 | Null treated as empty | `Test-PersonaCondition` | `Operators.Tests.ps1` |
|
||
| FR-013 | Unretrievable group data yields `EvaluationError` | Tri-state evaluation; facet retrieval flags | `UnknownNotFalse.Tests.ps1`, `UnknownPropagation.Tests.ps1` |
|
||
| FR-014 | Preserve stored value on failure | `Compare-PersonaValue` sets `Skipped` first | `EvaluationError.Tests.ps1`, `WriteGate.Tests.ps1` |
|
||
| FR-015 | Compare stored and calculated | `Compare-PersonaValue`, ordinal | `WriteGate.Tests.ps1` |
|
||
| FR-016 | Write only changed values, four conditions | `Compare-PersonaValue` + run-loop gate | `WriteGate.Tests.ps1` |
|
||
| FR-017 | Preview issues no write request | `Invoke-PersonaEngineRun` gate | `WhatIfZeroWrites.Tests.ps1` |
|
||
| FR-018 | Immediate per-user output | `Write-UserPersonaResult` | Exercised by every run-loop suite |
|
||
| FR-019 | Periodic summary | `Write-PersonaSummary`, interval check | `SummaryInterval.Tests.ps1` |
|
||
| FR-020 | Interval semantics, final always shown | `Invoke-PersonaEngineRun` | `SummaryInterval.Tests.ps1` |
|
||
| FR-021 | Reconciliation at every summary | `Test-PersonaReconciliation` | `Reconciliation.Tests.ps1` |
|
||
| FR-022 | Structured audit records | `New-PersonaAuditRecord`, `Write-PersonaAuditRecord` | `AuditRecordShape.Tests.ps1` |
|
||
| FR-023 | Configuration editor | `Edit-PersonaEngineConfig.ps1` | `ExitCodes.Tests.ps1` (editor) |
|
||
| FR-024 | Non-interactive validation with exit codes | `-NonInteractive` short-circuit | `NonInteractive.Tests.ps1` |
|
||
| FR-025 | Synthetic rule testing, no tenant | `Invoke-SyntheticRuleTest` | `NonInteractive.Tests.ps1` |
|
||
| FR-026 | Validate and back up before save | `Save-PersonaConfiguration` | `Safety.Tests.ps1` (`PE-SAF-007`) |
|
||
|
||
## Rule engine requirements
|
||
|
||
| ID | Requirement | Implementation | Test |
|
||
| --- | --- | --- | --- |
|
||
| RE-001 | Required rule fields | Schema `definitions/rule` | `LayerOrdering.Tests.ps1` |
|
||
| RE-002 | Unique priorities, lower first | `Resolve-UserPersona`; `PE-SEM-002` | `Ordering.Tests.ps1`, `Semantic.Tests.ps1` |
|
||
| RE-003 | `all` / `any` with nesting | `Test-PersonaConditionGroup` | `Composition.Tests.ps1` |
|
||
| RE-004 | Depth limit and hard ceiling | `Test-PersonaConditionGroup`; `PE-SEM-012`, `PE-SEM-013` | `Composition.Tests.ps1`, `Semantic.Tests.ps1` |
|
||
| RE-005 | Thirteen operators | `Test-PersonaCondition` | `Operators.Tests.ps1` |
|
||
| RE-006 | Case-insensitive; regex validated first | `Test-PersonaCondition`; `PE-SEM-016` | `Operators.Tests.ps1`, `Semantic.Tests.ps1` |
|
||
| RE-007 | Per-condition membership mode | Three-facet `MembershipRecord` | `RecordContracts.Tests.ps1`, `UnknownPropagation.Tests.ps1` |
|
||
| RE-008 | Combined identity sources | `Get-PersonaRequiredFacets` | `OutcomeExclusivity.Tests.ps1` |
|
||
| RE-009 | Special accounts by Object ID | Example configuration; no hard-coded path | Purity gate |
|
||
|
||
## Validation requirements
|
||
|
||
| ID | Requirement | Implementation | Test |
|
||
| --- | --- | --- | --- |
|
||
| VR-001 | Four ordered layers, fail-fast | `Test-PersonaConfiguration` | `LayerOrdering.Tests.ps1` |
|
||
| VR-002 | Sixteen semantic conditions | `Test-PersonaConfigurationSemantic` | `Semantic.Tests.ps1` — one test per code |
|
||
| VR-003 | Seven safety conditions | `Test-PersonaConfigurationSafety` | `Safety.Tests.ps1`, `TraceAcknowledgement.Tests.ps1` |
|
||
| VR-004 | Finding shape | `New-PersonaValidationFinding` | `Semantic.Tests.ps1`, `Safety.Tests.ps1`, `RecordContracts.Tests.ps1` |
|
||
| VR-005 | Warnings block only on request | Editor exit-code mapping | `ExitCodes.Tests.ps1` (editor) |
|
||
|
||
## Non-functional requirements
|
||
|
||
| ID | Requirement | Implementation | Test |
|
||
| --- | --- | --- | --- |
|
||
| NFR-001 | PowerShell 7 | `#Requires -Version 7.2`; manifest floor | Runs on 7.6.5 |
|
||
| NFR-002 | Caching, per-user and total duration | `New-PersonaDataCache`; stopwatch in `Resolve-UserPersona`; `RunComplete.durationMs` | `AuditCompleteness.Tests.ps1` — **no target set** |
|
||
| NFR-003 | Pagination, bounded retry, backoff | `Invoke-PersonaGraphRequest` | `RetryPolicy.Tests.ps1`, `Pagination.Tests.ps1` |
|
||
| NFR-004 | Comment-based help on public functions | Every function in `src/` | Manual review |
|
||
| NFR-005 | Run ID, UPN, Object ID, config hash | `New-PersonaAuditContext` | `AuditCompleteness.Tests.ps1` |
|
||
| NFR-006 | Least privilege, single attribute, no secrets | Six OTD-003 controls | `WriteBody.Tests.ps1`, `WriteBodyRejection.Tests.ps1`, sanitization gate |
|
||
| NFR-007 | Engine and validation run without the platform | Purity; layer dot-sourcing | Purity gate; whole offline suite |
|
||
| NFR-008 | No Windows PowerShell-only dependencies | `Microsoft.Graph.Authentication` only | **Unverified in Automation (V-5b)** |
|
||
|
||
## Success criteria
|
||
|
||
| ID | Criterion | Test | Status |
|
||
| --- | --- | --- | --- |
|
||
| SC-001 | Exactly one outcome per user | `OutcomeExclusivity.Tests.ps1` | Passing |
|
||
| SC-002 | Second run proposes zero changes | `Idempotence.Tests.ps1` | Passing |
|
||
| SC-003 | Determinism across shuffled input | `Determinism.Tests.ps1` | Passing |
|
||
| SC-004 | Zero writes under `-WhatIf` | `WhatIfZeroWrites.Tests.ps1` | Passing |
|
||
| SC-005 | Single-attribute request body | `WriteBody.Tests.ps1` | Passing |
|
||
| SC-006 | Audit completeness | `AuditCompleteness.Tests.ps1` | Passing |
|
||
| SC-007 | Reconciliation, and its failure path | `Reconciliation.Tests.ps1` | Passing |
|
||
| SC-008 | Everything runs offline | Whole offline suite; `validate.yml` gate 7 | Passing |
|
||
| SC-009 | Every VR-002 and VR-003 condition detected | `Semantic.Tests.ps1`, `Safety.Tests.ps1` | Passing |
|
||
| SC-010 | Non-interactive never prompts or hangs | `NonInteractive.Tests.ps1` | Passing |
|
||
| SC-011 | Every exit code reachable | `ExitCodes.Tests.ps1` (both) | Passing |
|
||
| SC-012 | Per-user output is immediate | `Write-UserPersonaResult` emits in `process` | Structural, not timed |
|
||
| SC-013 | No tenant data committed | `Test-Sanitization.ps1` | Passing |
|
||
|
||
## Gaps, stated plainly
|
||
|
||
| Item | Why it is not covered | What would close it |
|
||
| --- | --- | --- |
|
||
| NFR-002 performance | No target exists until representative tenant testing | A timed run against a real population |
|
||
| NFR-004 help coverage | Reviewed by eye, not asserted | A test parsing every exported function for a help block |
|
||
| NFR-008 Automation compatibility | No Automation account available | V-5b (T116) |
|
||
| SC-012 timing | Asserted structurally, not measured | A timed harness — low value against the cost |
|
||
| V-1, V-2, V-3 | Require a tenant | Stage A2 and A3 runs |
|
||
| V-4 | Requires a person | Written security sign-off |
|
||
| Phase 13 (T115–T121) | Requires an Automation account | Stage B |
|
||
|
||
## How to keep this honest
|
||
|
||
When a requirement's implementation moves, this table moves with it. When a test is deleted, the row
|
||
it backed becomes a gap and belongs in the gaps table, not silently in the main one. A matrix that is
|
||
allowed to drift is worse than none, because it converts "we do not know" into "we checked".
|