reality-feedback-create

Solid

Layered eval architecture, Layer 4 auto-create. Fires from the Stop hook chain after critic_dispatch. Appends an entry to the reality_feedback.sqlite ledger when the classifier verdict is high-stakes, with surface inference + outcome window + Critic metadata. Operator labels the outcome later via weekly review (skills/reality-review-weekly/SKILL.md). Outcomes feed a Layer 1 v2 classifier's training set over months.

AI & Automation 3 stars 0 forks Updated today MIT

Install

View on GitHub

Quality Score: 79/100

Stars 20%
20
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Reality-feedback auto-create skill Layer 4 of the layered eval architecture. Auto-creates ledger entries for high-stakes outputs so their outcome can be graded later. The discipline is: at output time, capture the prediction. At outcome time (days/weeks later), grade actual vs predicted. Over months, the deltas feed a Layer 1 v2 classifier's training set. ## Example output (ledger entry shape) ```json { "id": "rfb-2026-05-11-001", "ts_created": "2026-05-11T05:40:00+00:00", "session_id": "s_2026-05-11-build", "stake_class": "high", "stake_signals": ["draft", "outbound", "high-value-proposal"], "predicted_surface": "counter-offer should test a higher retainer + shorter term", "outcome_window_days": 30, "outcome_due": "2026-06-10", "outcome": null, "critic_metadata": {"score": 8.4, "passes": 3, "concerns": ["could be premature pattern-lock at n=2"]} } ``` ## Conversation context (prior) **Auto-fired** by the Stop hook chain. runs AFTER classifier_dispatch + critic_dispatch + (optionally) save_handler. The prior conversation is the just-completed turn whose output triggered the high-stake classification. The script walks `logs/session.jsonl` back to find the most-recent `stake_classified` event with `class=high` + the matching tool-output that produced it. ## Output format Returns ONE row appended to `data/reality_feedback.sqlite` table `outcomes`: | column | type | description | |---|---|---| | id | TEXT PK | `rfb-YYYY-MM-DD-NNN` | | ts_created | IS...

Details

Author
0xUrsanomics
Repository
0xUrsanomics/utopia-os
Created
5 days ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

reality-review-weekly

Layered eval architecture, Layer 4 weekly review. Cron-fired weekly. Queries reality_feedback.sqlite for entries where outcome IS NULL AND timestamp + window has elapsed. Surfaces N due entries via a chat message for operator labeling. Outcomes feed a Layer 1 classifier's training set over months. Decision gate at month 1 - >=50% labeling rate then continue Layer 4; else DROP.

3 Updated today
0xUrsanomics
AI & Automation Listed

track-record

A calibration loop for material, genuinely uncertain AI judgments whose outcomes can be observed later - log the bet with a numeric probability and an observable trigger, settle against real evidence, and distill recurring biases into rules. Do not trigger for deterministic/trivial claims or claims with no observable settlement event. Use when logging or settling a material prediction, running a calibration review, or when track record, prediction ledger, settle bets, or learning from outcomes comes up. 中文触发:材料性判断登记、复盘、回填、结算、判断校准、校准回顾、赌局台账。

0 Updated 5 days ago
LucioLiu
AI & Automation Solid

critic

Tool-grounded Critic skill (layered eval architecture, Layer 2). Dispatched as a subagent on high-stakes outputs. Grounds the verdict against tool-fetched evidence, not self-reflection. Returns SHIP / SHIP_WITH_FIXES / BLOCK with fixes + reason + confidence. Pairs with the stake classifier (Layer 1) and branching-workflow (Layer 3). Used by content drafts, regulatory-style takes, partner outreach drafts, and proposals.

3 Updated today
0xUrsanomics