Skip to main content

Citations

AIs make things up​

Chat AIs inherently hallucinate: they know a lot, but, they also make things up. A large part of Telosnex's founding is due to a medical doctor's realization that the AI he was using was making things up, and even making up citations.

Telosnex can make AI cite its answer. Telosnex doesn't do this all the time: frequently, chat is enough. Citations are a feature that Scripts can use. For example, the Chat scripts don't use it, but Search scripts do.

Setting up Citations​

First, you'll need a Script that has a Retrieve step.

  • The Search scripts in the Script Library do, and also have the prompting technique that adds them that we'll discuss in a moment.
  • You can use the RAG: Search, Read, Retrieve Cookbook to make your own (takes ~10 minutes)

The pattern for a Script that uses citations uses 3 steps:

  • Search: The AI is asked to search for information. Technically, this is optional, if you have files attached they can be used instead.
  • Retrieve: The AI identifies information relevant in the web pages/files. This also adds a message to Chat containing snippets that looks something like:
Press to see example
# Snippets from source [^1^](https://wikipedia.org/wiki/bears)
"Bears are carnivoran mammals of the family Ursidae. They are classified as caniforms, or doglike carnivorans. Although only eight species of bears are extant, they are widespread, appearing in a wide variety of habitats throughout the Northern Hemisphere and partially in the Southern Hemisphere..."

"Common characteristics of modern bears include large bodies with stocky legs, long snouts, small rounded ears, shaggy hair, plantigrade paws with five nonretractile claws, and short tails."

# Snippets from source [^2^](https://www.nationalgeographic.com/animals/mammals/group/bears/)
...and so on
  • Dialogue: Give the AI instructions on how to use the retrieved snippets and how to cite.

For example: "The above message is search results that should be used to help answer {{prompt}}. If a result is useful and backs up part of the text, cite it at the end of the sentence by adding [^N^], where N is the source number."

Telosnex will replace the [^N^] with a themed link to the source, like below:

Chat message with citations.

Tips​

  • A number is included to let you differentiate between sources from the same site.
  • Hover, or long press, on the citation to see the URL.
  • Press the citation to open the URL in your browser.
  • You can tell if a citation is hallucinated, because it won't have the same visual style.
  • Telosnex proactively removes citations that don't match actual sources provided to AI.
  • AIs are specifically trained to do this, even though it isn't widely used. We found out about this first from Bing's February 2023 system prompt. This also means AI needs surprisingly little guidance on using it: the RAG Cookbook uses "Add citations using the format [^N^], where N is the snippet # of the citation." and it works.