personaEngine2/specs/001-persona-engine/research.md

# Phase 0 Research: Persona Engine

**Date**: 2026-08-20 | **Spec**: [spec.md](spec.md) | **Plan**: [plan.md](plan.md)

Resolves the Clarification Register in [spec.md](spec.md). OTD-001 through OTD-005 are
**persistence-blocking** and are decided here. OTD-006 through OTD-010 receive provisional decisions
sufficient to plan implementation.

Every decision below that depends on a tenant-specific fact carries a **verification item (V-n)**.
Per the constitution, attribute-level write authorization "MUST be verified, never assumed" — the
decisions state what the product documentation says, and the V-items state what the team must prove
in its own tenant before enforcement is enabled.

---

## OTD-001 — Persona attribute mechanism

**Decision**: Store the persona in a **directory (schema) extension** single-valued string property
on the `user` resource, registered against a dedicated application registration in the tenant. The
property is referenced as `extension_<EXTENSION-APP-ID>_<APPROVED-PERSONA-ATTRIBUTE-NAME>`.

**Rationale**:

- **Writable via Graph for cloud-mastered users** with an ordinary `PATCH /users/{id}`, and readable
  via `$select` on the extension property name.
- **Consumable by Conditional Access.** CA assigns policy by user and group, not by user attribute,
  so the consumption path is: persona attribute → dynamic membership group rule → CA assignment.
  Dynamic membership rules support custom extension properties in the form
  `user.extension_<appId>_<propertyName>`, provided the property is **single-valued** and the
  extension belongs to an application in the tenant. Both conditions hold here.
- **Not population-dependent.** Unlike `extensionAttributeN`, it does not fail on accounts with an
  external origin (see rejected alternative A).

**Alternatives considered**:

| Alternative | Verdict | Reason |
| --- | --- | --- |
| **A. `onPremisesExtensionAttributes.extensionAttributeN`** (extensionAttribute1–15) | **Rejected** | Updates via Graph succeed only for objects that have always been mastered in Entra. Accounts that were ever synced from on-premises AD — or that originated in Exchange Online — fail with *"Unable to update the specified properties for objects that have originated within an external service."* In a hybrid or formerly-hybrid tenant this produces write failures determined by an account's history rather than by its rule match, which is a direct hazard to Principle I (deterministic) and Principle III (fail-safe). Remediation would require an Exchange Online PowerShell write path — a second persistence mechanism and a second permission surface. |
| **B. Directory (schema) extension** | **Selected** | See rationale above. |
| **C. Custom security attribute** | **Rejected as primary; retained as the security-first alternative** | This is the *only* mechanism offering genuine attribute-scoped authorization (see OTD-003), which makes it attractive. But custom security attributes are **not exposed to the dynamic group evaluation engine** and cannot be referenced in dynamic membership rules, so they cannot drive the CA consumption path that motivates the persona value. They are also not returned by default and require a separate permission and role. Choosing C trades the feature's primary downstream use for a stronger write boundary. |

**Consequences**: the compensating controls in OTD-003 become mandatory, because alternative B has
no attribute-scoped authorization.

**Verification items**:

- **V-1** — Register the extension application and property in a non-production tenant or an
  isolated attribute name; confirm read via `$select` and write via `PATCH` for: a cloud-only user, a
  currently-synced user, and a formerly-synced user. Directory extensions are not on-premises-mastered
  properties, so all three are expected to succeed — but this must be proven, not assumed, because the
  whole reason A was rejected is an origin-dependent write restriction.
- **V-2** — Confirm a dynamic membership group rule referencing the extension property evaluates and
  populates as expected, and that a CA policy assigned to that group applies.

---

## OTD-002 — Least-privilege Microsoft Graph permissions

**Decision**: application (managed identity) permissions, granted only as each capability is enabled:

| Capability | Permission | Notes |
| --- | --- | --- |
| Read users and the persona extension | `User.Read.All` | Extension property returned via `$select`. |
| Read group membership (`memberOf` conditions) | `GroupMember.Read.All` | Sufficient for `/users/{id}/memberOf` and `/transitiveMemberOf`. Narrower than `Group.Read.All`. |
| Read directory role assignments | `RoleManagement.Read.Directory` | For role-based conditions. |
| Write the persona attribute (enforcement only) | `User.ReadWrite.All` | **Supersedes** `User.Read.All`; grant only to the enforcement identity, and only after security approval of the OTD-003 controls. |

Application permissions are the Stage B (Automation, managed identity) form. Stage A uses the same
four as **delegated** scopes on an interactive connection.

**Rationale**: each permission maps to exactly one enabled capability, so a tenant that disables
group or role conditions grants strictly less. `Directory.Read.All` is deliberately **rejected** — it
is materially broader than the three read permissions combined and would grant visibility well
outside the enumerated data sources.

**Local (Stage A) equivalent — delegated scopes.** With no Automation account, the same three
capabilities are requested as delegated scopes on an interactive connection:

```powershell
Connect-MgGraph -Scopes 'User.Read.All','GroupMember.Read.All','RoleManagement.Read.Directory'
```

These delegated scopes require one-time admin consent for the app registration used locally; after
that, an ordinary user account can hold them. **Effective access is the intersection of the granted
scope and the signed-in user's directory roles** — which is precisely why V-3 must be run as a
non-privileged account (see below).

**Alternatives considered**: `Directory.ReadWrite.All` (rejected — grossly over-broad);
`User.ManageIdentities.All` (not applicable); delegated-only operation (rejected — unattended
Automation requires application permissions; delegated remains the local development path per FR-003).

**Verification items**:

- **V-3** — During the read-only pilot, grant only the three read permissions and confirm every
  enabled rule evaluates without a permission error. Any `EvaluationError` attributable to
  authorization identifies a missing-but-required permission and must be resolved before enforcement.
  **Stage A method**: connect with the three delegated scopes above while signed in as an **ordinary,
  non-privileged user account**. Running this as a Global Administrator invalidates the test — the
  scope narrows the token, but the account's directory roles still grant broad read access, so the
  run would succeed regardless of whether the three permissions are actually sufficient. This item is
  closeable now and does not need Automation.
- **V-3b** *(Stage B, deferred)* — Repeat as **application** permissions on the managed identity.
  Delegated and application authorization are evaluated differently, so a passing V-3 is strong
  evidence but not proof for the unattended path.

---

## OTD-003 — Can write authorization be restricted to the single target attribute?

**Decision**: **No — not for the mechanism selected in OTD-001.** Microsoft Graph application
permissions have no per-property scope: `User.ReadWrite.All` authorizes writes to every writable
property of every user in the tenant. There is no supported way to grant "write only
`extension_<app>_<persona>`".

Therefore the compensating controls are **mandatory and testable**, not advisory:

1. `Set-UserPersonaAttribute` accepts only the configured target attribute; any other name is a
   terminating error.
2. The target attribute MUST appear in `approvedWritableAttributes`; validation rejects all others
   (VR-002).
3. A single dedicated function, `New-PersonaWriteBody`, constructs the request body, and it emits a
   hashtable containing exactly one key.
4. Unit and integration tests assert on the **request body**, not on observed behaviour (SC-005).
5. Code owners gate every change to persistence, `approvedWritableAttributes`, and the target
   attribute.
6. Directory audit logs are monitored for property writes by the engine's service principal other
   than the target attribute.

**The one mechanism that *would* satisfy attribute-level authorization**: custom security attributes
(OTD-001 alternative C). Their assignment is governed by attribute sets: a principal is granted
`Attribute Assignment Administrator` **scoped to a specific attribute set**, plus the separate
`CustomSecAttributeAssignment.ReadWrite.All` permission — and notably, Global Administrator does not
hold this access by default. That is a real, enforced boundary rather than a compensating control.
It was rejected only because custom security attributes cannot feed dynamic groups (OTD-001).

**Stage A makes these controls *more* important, not less.** Local delegated writes run as the
signed-in operator, whose directory roles are typically far broader than the eventual service
principal's. During Stage A3 the compensating controls are the **only** thing standing between the
engine and an unintended property write, because the authorization boundary is effectively "whatever
the operator can do." Two additional Stage A rules follow:

- Stage A3 writes target **purpose-created test accounts only**. The baseline's read-only-during-
  early-development assumption continues to hold for the general population.
- Never sign in with a standing privileged account for a write run. Elevate for the session, and
  expect the directory audit log to attribute the write to the operator rather than to a service
  principal — which is exactly why Stage A3 evidence does not substitute for Stage B evidence.

**This trade-off requires explicit security sign-off.** The decision record for security review is:
*accept tenant-wide user-write permission plus six compensating controls, in exchange for a persona
value that Conditional Access can actually consume.*

**Verification items**:

- **V-4** — Confirm with the security owner, in writing, that the compensating-control set is
  accepted in place of attribute-scoped authorization. This is a **gate before enforcement**, per the
  constitution's Definition of Done.
- **V-4a** — Investigate whether an Administrative Unit-scoped role assignment can narrow the
  enforcement identity's write scope to a subset of the user population. This narrows *which users*,
  never *which attribute*, so it is a partial mitigation at best; do not present it as closing OTD-003.

---

## OTD-004 — Directory access approach

**Decision**: **Direct REST via `Invoke-MgGraphRequest`**, with `Microsoft.Graph.Authentication` as
the only runtime module. No resource-specific SDK modules (`Microsoft.Graph.Users`,
`Microsoft.Graph.Groups`, etc.).

**Rationale**:

- **Explicit request bodies.** The constitution requires the write payload to contain exactly one
  attribute and requires tests to inspect that body. `Invoke-MgGraphRequest -Method PATCH -Body` makes
  the body a first-class, assertable value. SDK cmdlets construct bodies internally from parameter
  binding, which makes SC-005 far harder to prove.
- **Dynamic extension properties.** The persona property name is configuration-driven and unknown at
  authoring time. Passing an arbitrary `extension_<appId>_<name>` key is natural in a hashtable body
  and awkward through typed cmdlet parameters.
- **Automation footprint.** One small module to import instead of the SDK's large module set, which
  reduces cold-start time, import failures, and version drift in the Automation PS7 environment
  (NFR-008).
- Managed-identity and interactive token acquisition are still handled by `Connect-MgGraph`, so
  nothing is reimplemented.

**Alternatives considered**: full Graph SDK cmdlets (rejected — heavy, opaque bodies, version drift);
raw `Invoke-RestMethod` with hand-rolled token acquisition (rejected — reimplements managed-identity
token handling and refresh for no benefit).

**Consequence**: pagination (`@odata.nextLink`), throttling, and error shaping are the engine's
responsibility. They are handled once, in the data-provider layer — see OTD-007 and
[contracts/graph-data-provider.md](contracts/graph-data-provider.md).

---

## OTD-005 — JSON Schema validation approach

**Decision**: the built-in **`Test-Json -SchemaFile`** cmdlet, with the schema authored to
**JSON Schema draft-07**.

**Rationale**: `Test-Json` ships with PowerShell 6.1+ and therefore needs no module import in either
the local or Automation PS7 environment — the strongest possible answer to NFR-008 and OTD-005's
"compatible locally and in automation" requirement. Its underlying validator is the Newtonsoft JSON
Schema implementation, whose reliable coverage is draft-04/06/07; **draft 2019-09 and 2020-12
constructs must not be used** in the schema.

**Implementation notes**:

- `Test-Json` signals failure by writing errors rather than simply returning `$false` in several
  PowerShell versions. `Test-PersonaConfiguration` MUST wrap it with
  `-ErrorAction SilentlyContinue -ErrorVariable` and translate the collected errors into
  `Validation Finding` objects (VR-004), so that layer 2 produces structured findings like every other
  layer.
- Schema validation is layer 2 of four. It cannot express the semantic rules in VR-002 (duplicate
  priorities, depth limits, cross-field constraints), which is why layers 3 and 4 exist as PowerShell
  checks. Do not attempt to push semantic rules into the schema.

**Alternatives considered**: bundling a third-party schema library (rejected — an extra Automation
dependency for capability the platform already provides); hand-written structural validation only
(rejected — VR-001 mandates a schema layer, and a schema is also the editor's contract).

**Verification items**:

- **V-5a** *(Stage A, closeable now)* — Execute `Test-Json -SchemaFile` against the draft-07 schema
  on the local PowerShell 7.4 workstation. Record the exact behaviour on failure: whether it returns
  `$false`, writes a non-terminating error, or throws. The layer-2 wrapper is built against **this
  observed behaviour**, and the observation is pinned in a unit test so a runtime change is caught
  rather than discovered.
- **V-5b** *(Stage B, deferred)* — Repeat inside the Azure Automation PowerShell 7 runtime and record
  its exact PowerShell version. If the behaviour differs from V-5a, the wrapper handles both — do not
  assume parity. This is the single highest-value item to run on day one of Stage B.

---

## Non-blocking decisions (OTD-006 – OTD-010)

These do not block persistence. They are decided far enough to implement v1 without rework.

### OTD-006 — Structured log destination and transport

**Decision**: newline-delimited JSON (one audit record per line) written to a configurable file path,
plus the Automation output stream. Emission goes through a single `Write-PersonaAuditRecord` sink
function so a Log Analytics or Event Hub transport can be added later without touching call sites.
Log Analytics ingestion is **deferred**, not designed out.

**Rationale**: NDJSON is append-safe, streamable, trivially ingestible later, and needs no
dependency. The sink indirection is what keeps the deferral cheap.

### OTD-007 — Retry policy

**Decision**: bounded exponential backoff with full jitter in the data-provider layer.

- **Retryable**: HTTP 429, 500, 502, 503, 504, and transport-level timeouts.
- **Never retried**: 400, 401, 403, 404, 409 — these are configuration, authorization, or logic
  defects and retrying masks them.
- **`Retry-After` honoured** whenever present; it overrides the computed backoff.
- **Max 5 attempts**, base delay 1s, exponential with full jitter, per-delay cap 60s.
- Every retry logs attempt number, status code, and delay. Exhausting retries on **required** data
  yields `EvaluationError` for the affected user (FR-013) — never a silent non-match.

**Rationale**: satisfies NFR-003 while keeping Principle III intact: the failure mode of exhausted
retries is preserve-and-report, not assume-false.

### OTD-008 — Full versus incremental processing

**Decision**: v1 performs **full enumeration** with pagination. Delta query is deferred and out of
scope for v1 (already recorded in spec Out of Scope). Revisit only when pilot timings justify it.

### OTD-009 — Schedule and concurrency lock

**Decision**: hourly is the candidate cadence, deployment-configurable, and the schedule ships
**disabled** (per the release pipeline). Concurrency control for v1 is the Automation account's own
job behaviour plus a run-start check that fails fast if another job for the same runbook is running.
A durable distributed lock is deferred.

**Stage A status**: not applicable. With no Automation account there is no schedule and no
concurrency surface — runs are manual and serial by construction. Nothing here needs building until
Stage B, and nothing in Stages A1–A3 depends on it.

**Rationale**: overlapping runs are idempotent by construction (Principle III) — the harm is wasted
Graph quota and confusing audit output, not incorrect data — so a lightweight check is proportionate
for v1.

### OTD-010 — Rollback

**Decision**: rollback is driven by the audit trail. Every write record carries the **previous
value**, the calculated value, the matched rule ID, and the run ID (NFR-005), which makes a run
reversible by replaying prior values from its audit output. A replay utility is deferred to v1.1; the
**record shape that makes it possible is v1** and is fixed in
[contracts/audit-record.md](contracts/audit-record.md).

**Rationale**: the cheap, decisive part is capturing the previous value at write time. Miss that in
v1 and rollback becomes impossible retroactively.

---

## Verification checklist

| ID | Item | Stage | Blocks |
| --- | --- | --- | --- |
| V-1 | Extension property read/write proven for cloud-only, synced, and formerly-synced users | A2 (read) / A3 (write) | Persistence implementation |
| V-2 | Dynamic group rule on the extension property populates; CA assignment applies | A3 | Downstream value of the feature |
| V-3 | Delegated pilot completes with only the three scopes, signed in as a **non-privileged** account | A2 | Enforcement |
| V-3b | Same, as application permissions on the managed identity | **B — deferred** | Unattended enforcement |
| V-4 | Written security sign-off on compensating controls in place of attribute-scoped write | Out-of-band | **Enforcement (constitution gate)** |
| V-4a | Administrative Unit scoping investigated as partial mitigation | Any | Nothing (informational) |
| V-5a | `Test-Json -SchemaFile` failure behaviour observed and pinned locally | A1 | Layer-2 wrapper implementation |
| V-5b | Same, confirmed in the Automation runtime, with PS version recorded | **B — deferred** | Configuration validation sign-off |

**Closeable in the current stage**: V-1 (read half), V-3, V-4a, V-5a — and V-4, which needs a
conversation rather than a tenant. **Deferred with Automation**: V-3b, V-5b.

## Sources

- [Manage rules for dynamic membership groups in Microsoft Entra ID](https://learn.microsoft.com/en-us/entra/identity/users/groups-dynamic-membership)
- [Creating dynamic groups using custom security attributes](https://learn.microsoft.com/en-us/answers/questions/5763638/creating-dynamic-groups-using-custom-security-attr)
- [Conditional Access: Users, Groups, Agents, and Workload Identities](https://learn.microsoft.com/en-us/entra/identity/conditional-access/concept-conditional-access-users-groups)
- [onPremisesExtensionAttributes resource type](https://learn.microsoft.com/en-us/graph/api/resources/onpremisesextensionattributes?view=graph-rest-1.0)
- [Update user — Microsoft Graph v1.0](https://learn.microsoft.com/en-us/graph/api/user-update?view=graph-rest-1.0)
- [Why is it not possible to update extension attributes of former hybrid users via Graph API?](https://learn.microsoft.com/en-us/answers/questions/1850101/why-is-it-not-possible-to-update-extension-attribu)
- [Add custom data to resources using extensions](https://learn.microsoft.com/en-us/graph/extensibility-overview)
- [What are custom security attributes in Microsoft Entra ID?](https://learn.microsoft.com/en-us/entra/fundamentals/custom-security-attributes-overview)
- [Manage access to custom security attributes in Microsoft Entra ID](https://learn.microsoft.com/en-us/entra/fundamentals/custom-security-attributes-manage)
- [Assign, update, list, or remove custom security attributes for a user](https://learn.microsoft.com/en-us/entra/identity/users/users-custom-security-attributes)