Example (Zeeguu): A word-translation pair can exist at several trust levels: generated by Google Translate (unverified), verified by an LLM checker (auto-verified, pattern #2), implicitly accepted by a user who practiced it without complaint, or explicitly corrected by a user. Each level carries different confidence, and the system can make different decisions depending on the validation state — for example, only including explicitly confirmed translations in exercises, while using auto-verified ones for reading assistance where the stakes are lower.
Forces: Once LLM-generated data is persisted, it becomes indistinguishable from human-verified data unless explicitly marked. Downstream features and users may silently treat unverified AI output as ground truth. Over time, the system accumulates a mix of trusted and untrusted data with no way to tell them apart.
Solution: Maintain an explicit, queryable validation state for all LLM-generated content in the data model. Never let LLM-generated content silently become trusted data. The validation state may be a simple flag, but more often it is a spectrum or state machine reflecting different levels of trust (e.g., unverified → auto-verified → implicitly accepted → explicitly confirmed by user; alternatively, one could track the number of users that have validated a given content).
Notes:
- This pattern complements Provenance (#7) — together they answer two essential questions about any piece of LLM-generated data: how was it produced? and has anyone confirmed it’s correct? The right granularity of validation tracking depends on the domain: some systems may need binary (verified/not), others may need multiple validators with agreement thresholds, and others — like Zeeguu — benefit from a trust spectrum that reflects different forms of implicit and explicit user feedback.
- Implicit validation: One could argue that “if a user practiced a word without complaint, it’s implicitly validated”