Simon Willison recently highlighted Jon Udell’s decades-long push for durable, linkable knowledge—permalinks, citations, and public annotation—reminding us that AI is only as credible as its sources. Here’s a practical playbook to bake provenance into every AI-assisted task. Source: Simon Willison.
Why information literacy is your AI superpower
LLMs are great at fluent language, not at truth. The difference between a risky guess and a reliable answer is provenance—clear, checkable trails back to primary sources.
Treat links, quotes, and timestamps as first-class data. That mindset, long championed by Jon Udell, turns AI from a persuasive storyteller into a transparent research assistant.
7 practical habits for AI-grade information literacy
- Save the source twice: include the live URL and an archive link (use the Wayback Machine’s Save Page Now at web.archive.org/save).
- Quote before you summarize: capture the exact sentence(s) you’re relying on, then ask the model to summarize with citations.
- Demand line-of-citation: ask models to pair each claim with a specific sentence and URL. Reject outputs that lack source lines.
- Track who and when: record author, publisher, and publication date. Without a timestamp, relevance decays fast.
- Annotate in public when possible: tools like Hypothes.is create durable breadcrumbs you (and teammates) can revisit.
- Prefer permalinks and canonical URLs: link to stable pages, not filtered views or ad-wrapped versions.
- Log decisions: keep a simple register (notes, spreadsheet, or repo) of claims, sources, and your confidence so others can audit your path.
Copy-paste prompts for better provenance
- Cited Summary Prompt: “Summarize the following sources. For each claim, quote the supporting sentence and provide a URL. If a claim lacks evidence, flag it.”
- Provenance Checker: “Given this paragraph and its citations, verify each sentence. Output a table: sentence | quoted evidence | URL | verdict (supported/unclear).”
- Link Hygiene: “Normalize all links to canonical URLs, add publication dates, and produce an archive link for each. Return JSON.”
Team tip: make it policy
Turn the seven habits into a lightweight checklist in PRDs, briefs, and research notes. Reward cited deliverables; send back anything that lacks sources.
For broader guidance, see the NIST AI Risk Management Framework—it emphasizes documentation, traceability, and transparency as foundations for trustworthy AI.
Takeaway
AI won’t fix bad sources. Build provenance into your workflow—links, quotes, timestamps—and your models will produce faster, more defensible outputs.
Like this? Get more practical AI playbooks in your inbox. Subscribe to The AI Nuggets.
