Setting up the LLM Guru

This walks through turning Synchronet's Guru into an LLM-backed chat personality that knows your board — answering from a real model, grounded in your own message bases, files, and wiki via retrieval. For the component reference, see chat_llm, chat_llm.ini, llm_tools, and llm_index.

The engine speaks the OpenAI-compatible chat API, so you have two paths:

• Local (private, no per-token cost). Install Ollama on the BBS host or a machine on your LAN, and pull a model (a 7B–14B instruction model such as qwen2.5 is a good starting point). Ollama serves at http://localhost:11434.
• Hosted. Any OpenAI-compatible provider (Groq, OpenRouter, OpenAI, etc.). You'll need that provider's endpoint URL and API key.

A GPU helps but isn't required; smaller models run acceptably on CPU for the short replies a Guru produces.

Edit ctrl/chat_llm.ini. At minimum set the backend in [default]:

[default]
endpoint = http://localhost:11434/v1/chat/completions
model    = qwen2.5:14b
api_key  = ollama

For a hosted provider, use its endpoint URL, model name, and real api_key. The remaining knobs (reply length, temperature, timeout, memory) have sensible defaults — see chat_llm.ini.

The bot's personality lives in the prompt files referenced from the ini — by default ctrl/chat_llm_persona.utf8 (the system prompt) and ctrl/chat_llm_greeting.utf8 (the opening line). Edit them in any UTF-8 editor. @macro@ tokens like @bot_name@, @system_name@, and @alias@ are filled in per conversation; the available macros are listed in prompts.

In SCFG → Chat Features → Artificial Gurus, edit your Guru and set the Module field to chat_llm. (Create a Guru first if you have none.) That's the switch that replaces the legacy pattern engine with the LLM.

At this point you have a working LLM Guru. Page it from a terminal session to test. The steps below add grounding and reach.

Without retrieval, the Guru answers from the model's training alone and will invent board-specific details. Give it your real content:

1. Set index_sources in chat_llm.ini. A good community-grounded default:

   index_sources = msgbase:Local,DOVE-Net,FsxNet,FidoNet

2. Build the index:

   jsexec llm_index.js guru

   (use your Guru's persona/section name; omit for default).

3. Keep it fresh with a nightly timed event that re-runs the builder.

See llm_index for all sources (message bases, file bases, a local wiki tree) and retrieval_rag for the retrieval knobs.

The bundled tools load automatically — the Guru can look up other BBSes in the directory, report your board's live stats and contents, and answer BBS-era history questions from a curated archive. Nothing to enable; they're offered to the model on every turn.

To run the same persona as an IRC bot, see chat_llm_irc.

• Long pause then nothing / “unable to read status” — the request timed out. A cold model load can take 15–30s; raise timeout and, for Ollama, keep the model resident (set OLLAMA_KEEP_ALIVE on the Ollama host).
• Bot invents subs, files, or board facts — retrieval isn't covering it. Build/refresh the index (step 5) and confirm the relevant content is in a configured source. Lowering index_min_score_per_token lets more (but noisier) context through.
• Bot answers general questions but refuses board-specific ones — the persona prompt's “only answer from board memory” rule is too strong for general knowledge; soften it in the persona file.

• chat_llm — the chat engine
• chat_llm.ini — configuration reference
• llm_tools — the tool registry
• llm_index — the RAG index builder
• chat_llm_irc — the IRC bot adapter