Learned rules you can read: how 'tell it once and it remembers' should actually work
3 min read
By BudAlly
Published Updated
"Tell it once that Vendor X bills in cases of 12 and it remembers." Every AI operations product says some version of this now. It is a good feature. It is also, done the obvious way, an unauditable one.
The obvious way, and why it fails an audit
The obvious way is to put the fact into the model's context — a memory store, a system prompt, a retrieved note. It works on Tuesday. The problems arrive later:
- Nobody can list what the system believes. An inspector asks why a bill was approved at 12× the invoice line; the answer is "the model remembered that vendor bills in cases." Where is that written?
- Nothing is versioned. The vendor changes to cases of 6. The memory is edited, and every past decision now points at a fact that no longer exists.
- Nothing is scoped. The rule learned for one store leaks to all stores; the rule learned for one brand leaks to the category.
- Nothing can be revoked cleanly. "Forget that" is a prompt, not a transaction.
What a learned rule needs
| Field | Example | Why |
|---|---|---|
| Kind | invoice.unit_convention, store.hours.exception, promo.exclusion | Rules are typed, so the engine knows where to apply them |
| Scope | tenant → store → vendor/brand/category | A rule applies exactly where it was learned |
| Expression | vendor=V123 → qty_unit=case(12) | Machine-checkable, human-readable |
| Version | v3 (supersedes v2, 2026-09-14) | Old proposals keep the version they ran against |
| Provenance | proposed by invoice-capture agent from 4 matching invoices; approved by A. Controller | Who, from what evidence |
| Effective from / to | 2026-09-14 → open | Time-bounded by default where facts expire |
| Confidence and evidence | 4 of 4 invoices in 90 days | So a person can judge whether to approve |
The lifecycle
- Observe. An agent notices a repeated pattern (four invoices where line quantity × 12 = received quantity).
- Propose. It drafts a rule with the fields above and marks the current proposal "applied as hint, pending rule."
- Approve. A person approves in the governance tab — or edits the scope first (this vendor, this store only).
- Apply. Every later proposal records the rule ID and version in its "rules applied" field.
- Supersede. Editing creates v(n+1); v(n) stays attached to the proposals that used it.
- Revoke. Retiring a rule sets
effective_to; the engine stops applying it and lists the open proposals that depended on it.
Three rules that look alike and aren't
- A regulatory rule (pack rule, e.g. RI-NY-06) comes from counsel, is versioned with the pack, and can't be edited by a store.
- A learned operational rule (vendor bills in cases) comes from evidence, is versioned per tenant, and is approved by the tenant.
- A model's inference (this looks like a case quantity) is neither; it is a hint shown in the evidence and never applied silently.
Keeping the three apart is most of what "trust" means in this category.
What to ask a vendor
Where does a learned rule live? Can I export the list? Can I see version history and who approved each one? Can I scope one to a store? What happens to past decisions when I edit it?
BudAlly's governance tab is described on the security page; the full field list for the audit trail is in what an audit trail should contain.