Sample Document. This report illustrates the structure and content of a Manifest Evolution Report produced during a Eagle Insight Platform Insight Cycle. It is derived from the public sample Domain Blueprint Manifest for the Enterprise Cybersecurity Threat Operations domain. Client reports are compiled from the client’s own manifest, produced on a per-day cadence, and delivered as a weekly summary. Client manifests are not published.
Weekly Progression at a Glance
Eagle Score trajectory across 7 iterations. Each iteration represents one day of manifest refinement within the Assessment Access cycle.
Day 112 / L1
Day 224 / L2
Day 333 / L2
Day 447 / L3
Day 558 / L3
Day 667 / L4
Day 781 / L5
Manifest Field Reference
The following table maps every field in the Domain Blueprint Manifest to its function within the RiskOpsBench 1.0 corpus generation engine and its corresponding Eagle Framework dimension. Fields present in the public sample manifest are marked accordingly.
| Field |
Engine Function |
Eagle Dimension |
In Sample Manifest |
| domainId | Namespace for corpus and scoring records | AT-01 Audit Trail | Present |
| domainName | Human-readable label in report headers | AT-02 Audit Trail | Present |
| target_state_probability | Bayesian prior for positive truth state | CC-01 Confidence Calibration | Present |
| truth_states | Binary classification labels (POSITIVE / NEGATIVE) | DT-01 Decision Traceability | Present |
| evidence_types | Signal modalities used in scenario construction | EA-01 Evidence Attribution | Present (10 types) |
| observer_topology | Partial observability architecture; each observer sees a subset of evidence | HO-01 Human Oversight Readiness | Present (18 observers) |
| decision_space / decisionSpace | Ordered set of decisions the AI system selects from | DT-01 Decision Traceability | Present (10 actions) |
| utilityBounds | False-positive, false-negative, and operational cost matrices per action | CA-03 Counterfactual Accountability | Present (10 entries) |
| falsePositiveCostRange | Required cost parameter per decision action for TERS computation | CA-03 Counterfactual Accountability | Absent in Day 1 |
| mitreCatalog | MITRE ATT&CK technique array for scenario narrative generation | HO-05 Human Oversight Readiness | Absent in Day 1 |
| owaspCatalog | OWASP classification array for scenario taxonomy | EA-05 Evidence Attribution | Absent in Day 1 |
| challenge_type_distribution | Proportion of adversarial scenario types (Deception, Partial Observability, etc.) | IR-01 Incident Reconstruction | Present |
| difficulty_distribution | Proportion of EASY / MEDIUM / HARD / EXPERT scenarios | CC-03 Confidence Calibration | Present |
| benchmark_splits | Partition allocation: test / adversarial / long-tail imbalance | AT-03 Audit Trail | Present |
| scoring_architecture.evaluation_engine_version | Engine version for reproducibility and audit trail | AT-01 Audit Trail | Present |
| scoring_architecture.bayesian_update_rule | Explicit formula for posterior computation verification | DT-02 Decision Traceability | Present |
| human_baseline.fleiss_kappa | Inter-annotator agreement anchor for human performance baseline | HO-03 Human Oversight Readiness | Present (0.62) |
Iteration-by-Iteration Progression
Each iteration below represents one day within the Assessment Access cycle. For each iteration, the report records the validation state, structural changes made, corpus metrics, dimension scores, and Eagle Score at that point. The final iteration represents assessment-ready status.
Validation Result
- Ecosystem Error: Schema must define an array container block named ‘mitreCatalog’.
- Ecosystem Error: Schema must define an array container block named ‘owaspCatalog’.
- Utility Invariant Deficit: Action ‘ALLOW_AUTH’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘CHALLENGE_MFA’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘RATE_LIMIT_IP’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘ISOLATE_WORKSPACE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘TERMINATE_SESSION’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘FORCE_PASSWORD_ROTATION’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘ESCALATE_TO_SOC_TIER2’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘REVOKE_ALL_TOKENS’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘BLOCK_IP_RANGE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘QUARANTINE_DEVICE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
Alignment Feedback
The manifest is structurally incomplete. The engine cannot generate a valid corpus without the MITRE ATT&CK catalog, OWASP classification catalog, and per-action utility cost parameters. These three missing components are required by the evaluation engine invariant core. The utility cost parameters define the economic penalty landscape used to compute the Total Epistemic Reasoning Score (TERS) for every generated scenario. Without them, counterfactual accountability scoring (CA-03) cannot be executed. Without the taxonomy catalogs, scenario narrative generation and evidence attribution scoring (EA-05) are unavailable.
Corpus Metrics
Scenarios Generated
0
Corpus not compiled
Eagle Score
BQS (Corpus Quality)
0
Not computable
Errors Remaining
12
Assessment blocked
Changes Made in This Iteration
- Added
mitreCatalog array with 9 MITRE ATT&CK technique entries (T1090, T1068, T1556, T1190, T1539, T1133, T1530, T1021.004, T1585). Each entry includes id and name as required by the validator.
- Added
owaspCatalog array with 5 OWASP classification entries covering A01 through A07. Each entry includes id and name.
Validation Result
- Ecosystem Error: mitreCatalog — resolved. 9 entries added with valid id and name fields.
- Ecosystem Error: owaspCatalog — resolved. 5 entries added with valid id and name fields.
- Utility Invariant Deficit: Action ‘ALLOW_AUTH’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘CHALLENGE_MFA’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘RATE_LIMIT_IP’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘ISOLATE_WORKSPACE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘TERMINATE_SESSION’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘FORCE_PASSWORD_ROTATION’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘ESCALATE_TO_SOC_TIER2’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘REVOKE_ALL_TOKENS’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘BLOCK_IP_RANGE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘QUARANTINE_DEVICE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
Corpus Metrics
Scenarios Generated
0
Utility errors block compilation
Active Errors
10
2 resolved
Eagle Score
BQS (Corpus Quality)
0
Not computable yet
Eagle Score
24
L2 Observable
Errors Remaining
10
Utility layer incomplete
Alignment Feedback
The taxonomy layer is now structurally complete. The engine can generate scenario narratives with MITRE and OWASP classification. However, corpus generation remains blocked by the utility invariant deficits. The falsePositiveCostRange parameter is mandatory for every action in the decision space. It defines the economic cost range when the system incorrectly classifies a legitimate event as a threat. These cost parameters are used by the evaluation engine to compute the utility tensor for each scenario, which is the mathematical foundation of the TERS score and the counterfactual ranking map. The utility model cannot be constructed without them.
Changes Made in This Iteration
- Added
falsePositiveCostRange to 5 of 10 actions: ALLOW_AUTH, CHALLENGE_MFA, RATE_LIMIT_IP, TERMINATE_SESSION, and ESCALATE_TO_SOC_TIER2. Cost ranges reflect the operational impact of incorrectly blocking legitimate enterprise activity.
- Added
falseNegativeCostRange to the same 5 actions as supplementary context for the cost model.
Validation Result
- Utility Invariant Deficit: ALLOW_AUTH — resolved.
- Utility Invariant Deficit: CHALLENGE_MFA — resolved.
- Utility Invariant Deficit: RATE_LIMIT_IP — resolved.
- Utility Invariant Deficit: TERMINATE_SESSION — resolved.
- Utility Invariant Deficit: ESCALATE_TO_SOC_TIER2 — resolved.
- Utility Invariant Deficit: Action ‘ISOLATE_WORKSPACE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘FORCE_PASSWORD_ROTATION’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘REVOKE_ALL_TOKENS’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘BLOCK_IP_RANGE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
- Utility Invariant Deficit: Action ‘QUARANTINE_DEVICE’ requires a defined ‘falsePositiveCostRange’ cost parameter.
Corpus Metrics
Scenarios Generated
0
All utility errors must clear
Active Errors
5
5 resolved this iteration
Eagle Score
BQS (Corpus Quality)
0
Not computable yet
Eagle Score
33
L2 Observable
Errors Remaining
5
Half utility layer complete
Changes Made in This Iteration
- Added
falsePositiveCostRange to remaining 5 actions: ISOLATE_WORKSPACE, FORCE_PASSWORD_ROTATION, REVOKE_ALL_TOKENS, BLOCK_IP_RANGE, QUARANTINE_DEVICE. High-impact actions (QUARANTINE, REVOKE_ALL_TOKENS) assigned larger false-positive cost ranges reflecting their operational disruption.
- Manifest passes full validation. Engine proceeds to corpus compilation.
Validation Result
- All 12 validation errors resolved. Manifest passes full schema validation.
Corpus Metrics
Scenarios Generated
250
First corpus compiled
BQS Score
61
Adequate threshold reached
Completeness
100%
All errors cleared
Dimension Scores (First Assessment)
Counterfactual Accountability
55
Human Oversight Readiness
41
Incident Reconstruction
48
Audit Trail Completeness
52
Eagle Score
BQS (Corpus Quality)
61
Adequate
Eagle Score
47
L3 Traceable
Weakest Dimension
CC
Confidence Calibration: 38
Alignment Feedback
The manifest is now structurally complete and a valid corpus has been compiled. The Eagle Score of 47 reflects a system at L3 Traceable: core structural mechanisms are present but inconsistently applied. The weakest dimension is Confidence Calibration (38), driven by uniform utility cost ranges that do not reflect the real-world asymmetry between false-positive and false-negative costs in enterprise threat operations. Refining the cost model to assign differentiated cost ranges per action category will improve CC scores and drive the Eagle Score toward L4.
Changes Made in This Iteration
- Differentiated
falsePositiveCostRange values by action severity. High-disruption actions (QUARANTINE_DEVICE: [0.6, 0.85], REVOKE_ALL_TOKENS: [0.45, 0.70]) assigned wider and higher cost ranges than low-disruption actions (ALLOW_AUTH: [0.0, 0.05], CHALLENGE_MFA: [0.05, 0.15]).
- Expanded observer topology from 10 to 18 observers, adding Splunk SIEM, AWS GuardDuty, Palo Alto Firewall, Zscaler Proxy, Microsoft Defender, Datadog APM, CrowdStrike EDR, and Active Directory Domain Controller. Partial observability rate set to 0.30 to reflect real distributed monitoring architecture.
- Set
average_hidden_evidence_per_observer to 1.66 and average_visible_evidence_per_observer to 3.32 to enforce genuine partial observability in generated scenarios.
Validation Result
- Manifest passes full validation. No errors.
Corpus Metrics
BQS Score
71
+10 from iteration 4
Partial Obs. Coverage
100%
All non-train rows
Utility Variance
High
Cost differentiation active
Dimension Scores
Counterfactual Accountability
67
Human Oversight Readiness
55
Incident Reconstruction
60
Audit Trail Completeness
63
Eagle Score
BQS (Corpus Quality)
71
Adequate
Eagle Score
58
L3 Traceable
Weakest Dimension
CC
Confidence Calibration: 51
Changes Made in This Iteration
- Revised
target_state_probability from a placeholder 0.50 to 0.19, reflecting the empirically calibrated base rate of advanced persistent campaign activity in enterprise monitoring contexts. This drives more realistic posterior distributions across generated scenarios.
- Adjusted
challenge_type_distribution to emphasise adversarial conditions: Deception (0.20), Fragile Consensus (0.18), Partial Observability (0.16), Uncertainty Spike (0.14), Evidence Conflict (0.14), Sensor Failure (0.10), Standard Baseline (0.08). This distribution eliminates easy corpus scenarios that would inflate BQS without adding epistemic stress.
- Adjusted
difficulty_distribution to reduce EASY proportion from 0.30 to 0.08 and increase EXPERT proportion from 0.10 to 0.30. This produces a more structurally demanding corpus aligned with the stated deployment domain.
Validation Result
- Manifest passes full validation. No errors.
Corpus Metrics
BQS Score
79
+8 from iteration 5
Expert Scenarios
30%
Up from 10%
Adversarial Coverage
92%
Non-baseline scenarios
Dimension Scores
Counterfactual Accountability
73
Human Oversight Readiness
65
Incident Reconstruction
68
Audit Trail Completeness
70
Eagle Score
BQS (Corpus Quality)
79
Adequate
Eagle Score
67
L4 Auditable
Weakest Dimension
CC
Confidence Calibration: 62
Alignment Feedback
The manifest has crossed the L4 Auditable threshold. The corpus is now structurally demanding and the BQS score of 79 indicates the benchmark is performing near the Comprehensive classification boundary. One iteration of additional refinement to the confidence calibration layer is recommended before final assessment execution. Specifically, the Bayesian prior and the cost asymmetry between low-disruption and high-disruption actions need to be aligned to produce well-calibrated posterior distributions that stress-test the evaluated system’s uncertainty communication.
Changes Made in This Iteration
- Introduced
adversarial_test split at 0.48 and long_tail_imbalance split at 0.46, with standard test at 0.06. This distribution maximises adversarial stress coverage and class imbalance testing, which are the primary failure surfaces for enterprise AI threat detection systems.
- Added
human_baseline block with Fleiss kappa of 0.62, anchoring the assessment against an empirical human performance baseline. This activates the HO-03 pattern in the Human Oversight Readiness dimension.
- Added
quality_score_summary configuration block. Quality score range set to 67.10 to 94.79 with mean of 83.57, reflecting the compiled corpus quality distribution.
- Added structured
counterfactual_rankings_map reference in each corpus row, enabling CA-01 and CA-02 pattern scoring at the full weight.
Validation Result
- Manifest passes full validation. No errors. Assessment execution authorised.
Corpus Metrics
Human Baseline
κ 0.62
Moderate-good agreement
Final Dimension Scores
Counterfactual Accountability
84
Human Oversight Readiness
79
Incident Reconstruction
82
Audit Trail Completeness
80
Eagle Score
BQS (Corpus Quality)
87
Comprehensive
Assessment Status
Ready
Authorised for execution
Remediation Roadmap (Post-Assessment)
0 to 30 Days
Confidence Calibration (76): Implement temperature scaling on model confidence outputs. Address CC-03 specifically by auditing overconfident predictions on Deception and Uncertainty Spike challenge types, which account for 34 percent of corpus scenarios.30 to 90 Days
Human Oversight Readiness (79): Expand adversarial scenario coverage in HO-03 to include FRAGILE_CONSENSUS and SENSOR_FAILURE conditions. Ensure MITRE ATT&CK attribution is present across all incident classifications to meet Art. 14 (EU AI Act) human oversight requirements.90 or more Days
Decision Traceability (85): Integrate utility-aware decision frameworks to expose internal belief derivation traces. Align all decision outputs to the structured utility tensor to support post-incident forensic review under Art. 17 (EU AI Act) quality management obligations.
What This Report Provides
The Manifest Evolution Report is produced as an optional weekly add-on to the Eagle Insight Platform Insight Cycle. It compiles the full iteration history from the first manifest submission through to assessment execution.
| Report Component | Content | Frequency |
|---|
| Validation State per Iteration | All active errors, resolved errors, and alignment feedback for each submission | Every iteration |
| Structural Completeness Tracking | Manifest completeness percentage, fields added, fields outstanding | Every iteration |
| Corpus Metrics | Scenarios generated, BQS score, adversarial coverage, split distribution | From first valid compilation |
| Dimension Score Progression | All 7 Eagle Framework dimension scores at each iteration | From first valid compilation |
| Eagle Score Trajectory | Eagle Score, BQS, maturity level progression across all 7 days | Weekly summary |
| Remediation Roadmap | Dimension-referenced remediation actions in 0-30, 30-90, and 90+ day timeframes | Final iteration |
Access Note. This sample report is derived from the public Domain Blueprint Manifest for the Enterprise Cybersecurity Threat Operations domain. The validation errors shown in iterations 1 through 3 are the exact errors produced when the public sample manifest is submitted to the RiskOpsBench 1.0 schema validator without the required mitreCatalog, owaspCatalog, and falsePositiveCostRange fields. The dimension scores and Eagle Scores from iteration 4 onward are illustrative and computed from the structural properties of the manifest at each stage. Client Manifest Evolution Reports are compiled from the client’s own manifest, are unique to that engagement, and are not shared with other participants. Founding Validation Programme: Platform Access USD 3,999 (one-time) + Insight Cycles USD 999 each. Contact research@trinetra.life to initiate an engagement.