====== Setting up the LLM Guru ======

This walks through turning Synchronet's Guru into an LLM-backed chat
personality that knows your board — answering from a real model, grounded in
your own message bases, files, and wiki via retrieval.  For the component
reference, see [[module:chat_llm]], [[config:chat_llm.ini]],
[[module:llm_tools]], and [[module:llm_index]].

===== 1. Stand up a model backend =====

The engine speaks the OpenAI-compatible chat API, so you have two paths:

  * **Local (private, no per-token cost).** Install [[https://ollama.com|Ollama]] on the BBS host or a machine on your LAN, and pull a model (a 7B–14B instruction model such as ''qwen2.5'' is a good starting point).  Ollama serves at ''%%http://localhost:11434%%''.
  * **Hosted.** Any OpenAI-compatible provider (Groq, OpenRouter, OpenAI, etc.).  You'll need that provider's endpoint URL and API key.

A GPU helps but isn't required; smaller models run acceptably on CPU for the
short replies a Guru produces.

===== 2. Configure chat_llm.ini =====

Edit ''ctrl/chat_llm.ini''.  At minimum set the backend in ''[default]'':

<code ini>
[default]
endpoint = http://localhost:11434/v1/chat/completions
model    = qwen2.5:14b
api_key  = ollama
</code>

For a hosted provider, use its endpoint URL, model name, and real ''api_key''.
The remaining knobs (reply length, temperature, timeout, memory) have sensible
defaults — see [[config:chat_llm.ini]].

===== 3. Tune the persona prompt =====

The bot's personality lives in the prompt files referenced from the ini — by
default ''ctrl/chat_llm_persona.utf8'' (the system prompt) and
''ctrl/chat_llm_greeting.utf8'' (the opening line).  Edit them in any UTF-8
editor.  ''@macro@'' tokens like ''@bot_name@'', ''@system_name@'', and
''@alias@'' are filled in per conversation; the available macros are listed in
[[config:chat_llm.ini#prompts]].

===== 4. Point the Guru at the engine =====

In [[util:SCFG]] → [[config:chat_features#artificial_gurus|Chat Features →
Artificial Gurus]], edit your Guru and set the **Module** field to
''chat_llm''.  (Create a Guru first if you have none.)  That's the switch that
replaces the legacy pattern engine with the LLM.

At this point you have a working LLM Guru.  Page it from a terminal session to
test.  The steps below add grounding and reach.

===== 5. (Optional) Build the retrieval index =====

Without retrieval, the Guru answers from the model's training alone and will
invent board-specific details.  Give it your real content:

  - Set ''index_sources'' in ''chat_llm.ini''.  A good community-grounded default: <code ini>
index_sources = msgbase:Local,DOVE-Net,FsxNet,FidoNet</code>
  - Build the index: <code>
jsexec llm_index.js guru</code> (use your Guru's persona/section name; omit for ''default'').
  - Keep it fresh with a nightly timed event that re-runs the builder.

See [[module:llm_index]] for all sources (message bases, file bases, a local
wiki tree) and [[config:chat_llm.ini#retrieval_rag]] for the retrieval knobs.

===== 6. (Optional) Tools =====

The bundled [[module:llm_tools|tools]] load automatically — the Guru can look
up other BBSes in the directory, report your board's live stats and contents,
and answer BBS-era history questions from a curated archive.  Nothing to
enable; they're offered to the model on every turn.

===== 7. (Optional) Put the Guru on IRC =====

To run the same persona as an IRC bot, see [[module:chat_llm_irc]].

===== Troubleshooting =====

  * **Long pause then nothing / "unable to read status"** — the request timed out.  A cold model load can take 15–30s; raise ''timeout'' and, for Ollama, keep the model resident (set ''OLLAMA_KEEP_ALIVE'' on the Ollama host).
  * **Bot invents subs, files, or board facts** — retrieval isn't covering it.  Build/refresh the index (step 5) and confirm the relevant content is in a configured source.  Lowering ''index_min_score_per_token'' lets more (but noisier) context through.
  * **Bot answers general questions but refuses board-specific ones** — the persona prompt's "only answer from board memory" rule is too strong for general knowledge; soften it in the persona file.

===== See Also =====

  * [[module:chat_llm]] — the chat engine
  * [[config:chat_llm.ini]] — configuration reference
  * [[module:llm_tools]] — the tool registry
  * [[module:llm_index]] — the RAG index builder
  * [[module:chat_llm_irc]] — the IRC bot adapter

{{tag>chat guru llm chat_llm howto ai ollama rag}}