1. It predicts the next piece of text
A chatbot is powered by a large language model (LLM). The model reads your message and writes a reply one small piece at a time. These pieces are called tokens — roughly a short word or part of a word. At every step the model picks a likely next token based on everything before it.
Because it builds answers step by step, asking it to “think step by step” or to plan before answering often improves quality on complex problems. Many assistants now do this automatically in a “reasoning” or “thinking” mode.
2. The context window is its short-term memory
Everything the model can “see” at once — your messages, its replies, uploaded files — is called the context window. Modern windows are large enough to hold whole books, but they are still limited. In a very long chat, early details can be forgotten or given less attention.
- Start a new chat when you switch to an unrelated task.
- Paste the important facts again if a long chat starts drifting.
- Use features like Projects, Custom GPTs or Gems (Module 4) to keep standing instructions available.
3. Training data has a cut-off date
Models learn from data collected up to a certain date. Anything after that — prices, news, new software versions — the model does not know unless it can search the web or you give it the information. If an answer depends on recent facts, turn on web search or paste the source yourself.
4. Tools extend what it can do
On its own a model only produces text. Assistants become much more capable when connected to tools: web search, running code, reading files, generating images, or controlling other apps. When you see an assistant “analysing data” or “browsing”, it is using a tool and reading the result back.
Why chatbots make things up
When the model does not have the right information, the most “likely-sounding” continuation can still be a confident, specific, wrong answer. This is called a hallucination. It happens most with exact numbers, citations, links, niche facts, and very recent events.
You reduce hallucinations by giving the source material, asking it to say “I don’t know” when unsure, asking for sources you can open, and checking anything that matters.
Answer only using the document I pasted below. If the answer is not in the document, say "Not in the document." Quote the exact sentence you used. [paste document]