# AutoQA Decision Policy Reference

**Reference version:** 0.1.0

## 1. Purpose

The decision policy converts evidence-backed criterion assessments into operational actions. It is deliberately separate from semantic evaluation so that:

- project owners can change operational thresholds without rerunning all semantic analysis;
- evidence and criterion verdicts remain inspectable;
- severity does not silently become action;
- attempt, annotation, and composite-case decisions remain independent;
- every action has a deterministic policy trace.

## 2. Action semantics

### Accept

The target is ready for its intended use. All hard requirements are affirmatively satisfied at the required calibrated confidence, and no unresolved major issue remains.

### Accept with notes

The target is usable, but one or more advisory or minor findings should be retained. Notes must not hide a major unresolved defect.

### Rework

The target is not ready, but the defect is specific and repairable. Feedback should identify the minimal sufficient correction.

### Reject

The target has a disqualifying defect under an explicit project rule. Rejection should require strong evidence and calibrated confidence unless the failure is fully deterministic.

### Escalate

The system should not decide autonomously. Reasons include:

- uncertain applicability;
- ambiguous instruction interpretation;
- insufficient evidence;
- required domain expertise;
- conflicting high-quality evidence;
- uncalibrated semantic judgment;
- unsupported input type or context range;
- security or integrity anomaly;
- disagreement near a high-cost threshold.

## 3. Target semantics

### Attempt target

Uses attempt-quality criteria. An annotation error must not lower the attempt verdict.

### Annotation target

Uses annotation-fidelity criteria. A good attempt can have a bad annotation, and vice versa.

### Composite-case target

Represents whether the attempt-plus-annotation record is ready as annotated data. Its policy may require both targets to be acceptable.

Typical mapping:

| Attempt | Annotation | Composite |
|---|---|---|
| accept | accept | accept |
| accept | rework | rework |
| rework | accept | rework |
| reject | any | reject or rework according to retention policy |
| escalate | any | escalate |
| any | escalate | escalate |

The exact mapping belongs in the project contract.

## 4. Inputs

The policy engine consumes only recorded state:

- criterion assessments;
- criterion metadata and tags;
- confirmed findings;
- unresolved findings;
- calibrated confidence;
- annotation disagreement types;
- deterministic validator outcomes;
- human-resolution response where present;
- policy rules and priorities.

It must not independently reinterpret the raw attempt.

## 5. Rule match semantics

Each rule targets one of:

```text
attempt
annotation
composite_case
batch
```

A rule may select assessments by:

- criterion identifiers;
- criterion tags;
- verdicts;
- severities;
- default decision effects;
- minimum calibrated confidence;
- presence of unresolved findings;
- annotation disagreement type.

### 5.1 Multiple selectors

Selectors within one rule use logical AND unless the contract explicitly defines otherwise.

Example:

```yaml
criterionTags: [hard-quality]
verdicts: [not_met]
severities: [critical]
minimumCalibratedConfidence: 0.95
```

This matches a criterion only when all four conditions are satisfied.

### 5.2 Multiple criteria

A project must declare whether a rule over multiple criteria means:

- **any matching criterion**;
- **all listed criteria**;
- **at least N matching criteria**.

Version 0.1.0 recommends the following default:

- negative and escalation rules use **any**;
- clean-accept rules use **all**.

The human-readable rule description must make this explicit until a later schema version adds a formal quantifier.

### 5.3 Rule priority

Rules are evaluated from lowest numeric priority to highest. A terminal matched rule ends evaluation for that target. Nonterminal matches remain in the policy trace and may be superseded by a later terminal rule according to project logic.

### 5.4 Tie-break order

If multiple nonterminal actions remain, use the contract’s `tieBreakOrder`. A conservative default is:

```text
reject > escalate > rework > accept_with_notes > accept
```

This is not universally appropriate. For repairable work, a project may prefer escalation or rework over rejection.

## 6. Required preconditions

Before applying semantic action rules, the policy engine must check:

1. input and contract hashes resolve;
2. required assessment targets exist;
3. no cross-document referential error exists;
4. decisive findings are confirmed;
5. the calibration profile is compatible;
6. required deterministic checks completed;
7. any human response is valid for its request;
8. no unresolved security anomaly invalidates the run.

A failed precondition normally produces escalation or a system failure, not a negative judgment against the attempter.

## 7. Reference algorithm

```typescript
function decideTarget(
  target: DecisionTarget,
  contract: ProjectContract,
  evaluation: EvaluationState,
): Decision {
  const applicableRules = contract.decisionPolicy.rules
    .filter((rule) => rule.target === target)
    .sort((a, b) => a.priority - b.priority);

  const trace: PolicyTraceEntry[] = [];
  const matchedActions: OperationalAction[] = [];
  const triggeredRuleIds: string[] = [];

  assertReferentialIntegrity(contract, evaluation);
  assertCalibrationCompatibility(contract, evaluation, target);

  for (const rule of applicableRules) {
    const match = evaluateRuleMatch(rule.match, contract, evaluation, target);

    trace.push({
      ruleId: rule.ruleId,
      matched: match.matched,
      inputs: match.traceInputs,
      result: match.explanation,
    });

    if (!match.matched) continue;

    matchedActions.push(rule.action);
    triggeredRuleIds.push(rule.ruleId);

    if (rule.terminal) {
      return buildDecision(target, rule.action, triggeredRuleIds, trace, evaluation);
    }
  }

  const action = matchedActions.length
    ? resolveByTieBreak(matchedActions, contract.decisionPolicy.tieBreakOrder)
    : contract.decisionPolicy.defaultAction;

  return buildDecision(target, action, triggeredRuleIds, trace, evaluation);
}
```

The production implementation must also enforce the safeguards below.

## 8. Safeguards

### 8.1 Positive-proof safeguard

An attempt or annotation cannot be automatically accepted merely because no failure rule matched.

For every required-presence, completeness, grounding, or positive-quality criterion needed for acceptance:

- applicability must be `applies`;
- verdict must be `met`;
- at least one confirmed positive finding must be referenced;
- required expected elements must be complete;
- calibrated confidence must meet the action gate.

### 8.2 Unresolved-finding safeguard

A decision-changing unresolved finding blocks automatic acceptance or rejection unless the policy explicitly routes it to a qualified human resolution.

### 8.3 Calibration safeguard

If the chosen action is listed in `requiresCalibrationForActions`, every decisive semantic assessment must have a compatible calibrated score or band.

Otherwise use `uncalibratedAction`.

A deterministic validator may be exempt when its correctness has been separately verified and the contract explicitly permits autonomous action from it.

### 8.4 Counterevidence safeguard

Before autonomous reject or accept, decisive findings should have undergone a counterevidence challenge. A confirmed counterfinding that materially weakens the decision should lower confidence or trigger escalation.

### 8.5 Noncompensation safeguard

A criterion with `allowCompensation = false` cannot be offset by strengths on other criteria.

### 8.6 Annotation isolation safeguard

Annotation-fidelity failures cannot change the attempt target unless the project defines the annotation as part of the attempt itself. They may change the annotation and composite targets.

### 8.7 Ambiguity safeguard

A criterion marked `ambiguous` must not be treated as `not_met`. A criterion marked `unknown` must not be treated as `met`.

### 8.8 System-error safeguard

Provider failures, truncation, malformed output, unsupported media, or retrieval failure must never be counted as attempter defects.

## 9. Human-resolution policy

### 9.1 Candidate unresolved issues

The engine identifies unresolved findings that are:

- decision-changing;
- answerable by one qualified human;
- expressible as a bounded choice;
- within the project’s allowed question types.

### 9.2 Select one question

When multiple issues are eligible, prioritize by expected reduction in decision loss:

```text
expected value = loss before answer
                 − expected loss after answer
                 − interaction cost
```

Practical ranking factors:

- severity of affected criterion;
- probability the answer changes the target action;
- confidence that the human can resolve it;
- human effort;
- recurrence value for contract improvement.

### 9.3 Apply response

A response must map to an explicit update such as:

- applicability becomes applies or not applicable;
- one instruction gains precedence;
- one evidence class becomes authoritative;
- a specified inference is permitted or forbidden;
- severity becomes major or minor;
- an alternative is accepted;
- the contract is declared ambiguous.

The policy engine reruns only after the update is recorded in the evaluation state and audit trail.

### 9.4 Project ambiguity response

When the responder chooses `project instruction is ambiguous`:

- final target action should normally be escalate;
- the case should not count as reviewer error;
- a contract issue should be created;
- recurring affected cases should be grouped for re-adjudication after clarification.

## 10. Example policy

### 10.1 Attempt rules

```yaml
- target: attempt
  ruleId: reject-critical-grounding
  priority: 1
  match:
    criterionTags: [hard-quality]
    verdicts: [not_met]
    severities: [critical]
    minimumCalibratedConfidence: 0.95
  action: reject
  terminal: true

- target: attempt
  ruleId: rework-major
  priority: 2
  match:
    verdicts: [not_met]
    severities: [major]
    minimumCalibratedConfidence: 0.85
  action: rework
  terminal: false

- target: attempt
  ruleId: escalate-unresolved
  priority: 3
  match:
    hasUnresolvedFindings: true
  action: escalate
  terminal: true

- target: attempt
  ruleId: accept-clean
  priority: 10
  match:
    criterionTags: [required-for-attempt-acceptance]
    verdicts: [met]
    minimumCalibratedConfidence: 0.90
    hasUnresolvedFindings: false
  action: accept
  terminal: true
```

### 10.2 Annotation rules

```yaml
- target: annotation
  ruleId: rework-unfaithful-review
  priority: 1
  match:
    criterionTags: [annotation-fidelity]
    verdicts: [not_met]
    severities: [major, critical]
    minimumCalibratedConfidence: 0.90
  action: rework
  terminal: true

- target: annotation
  ruleId: accept-faithful-review
  priority: 10
  match:
    criterionTags: [annotation-fidelity]
    verdicts: [met]
    minimumCalibratedConfidence: 0.90
    hasUnresolvedFindings: false
  action: accept
  terminal: true
```

### 10.3 Composite rules

```yaml
- target: composite_case
  ruleId: escalate-any-target-uncertain
  priority: 1
  match:
    hasUnresolvedFindings: true
  action: escalate
  terminal: true

- target: composite_case
  ruleId: rework-bad-annotation
  priority: 2
  match:
    criterionIds: [annotation-fidelity]
    verdicts: [not_met]
    severities: [major, critical]
  action: rework
  terminal: true

- target: composite_case
  ruleId: accept-both-clean
  priority: 10
  match:
    criterionTags: [required-for-composite-acceptance]
    verdicts: [met]
    minimumCalibratedConfidence: 0.90
    hasUnresolvedFindings: false
  action: accept
  terminal: true
```

## 11. Policy trace requirements

Every decision records:

- target;
- chosen action;
- policy version;
- triggered rules;
- decisive criteria;
- unresolved findings;
- confidence;
- full rule trace;
- human-readable rationale;
- decision time.

A useful policy trace must allow an auditor to answer:

- Which rule was evaluated?
- What criterion state was supplied?
- Did it match?
- Why or why not?
- Which rule determined the action?
- What would have changed the action?

## 12. Policy testing

The decision engine should have deterministic unit tests for:

- each rule independently;
- tie-break behavior;
- no-rule default;
- uncalibrated confidence;
- not-applicable criteria;
- ambiguous and unknown criteria;
- positive-proof failure;
- missing expected elements;
- counterevidence;
- noncompensable failures;
- annotation isolation;
- composite mapping;
- human response updates;
- invalid rule identifiers;
- contract version mismatch;
- calibration profile mismatch;
- system error versus attempter failure.

Golden policy tests should be run whenever the contract, policy engine, or schema changes.

## 13. Policy-version changes

Create a new policy version when changing:

- action thresholds;
- rule priority;
- target mapping;
- hard gates;
- tie-break order;
- confidence requirements;
- default action;
- human escalation conditions;
- composite-case admission policy.

A policy-only change does not necessarily require rerunning claim and evidence extraction. It does require replaying the stored criterion state through the new policy and validating the operational effect on a locked set.

## 14. Initial default

For a new project without sufficient calibration:

```text
Deterministic clean pass/failure: project-defined automatic action
Semantic clear finding:          reviewer assistance
Semantic uncalibrated decision:   escalate
Subjective disagreement:          escalate or advisory
Reviewer performance consequence: adjudicated samples only
```

The default action should remain `escalate` until benchmark gates justify selective autonomy.