Your First LLM Walkthrough: From Prompt to Answer

You type a question into a chat box and an answer appears. What happened between those two moments?

You can understand the basic loop without knowing neural networks or GPUs. By the end of this lesson, you will be able to mark the input, the next piece of output, and the repeat step in a short example. Those are the landmarks we will use throughout Track 1.

Start with one exchange

Imagine you enter this prompt:

Finish this sentence: The sky is

The model might continue with blue. We will use that answer as an illustration, not as a guaranteed output from a real model. Another model, or the same model with different settings, could choose something else.

The text you supply is the prompt. The text the model adds is the generated output. A large language model, or LLM, is a computer program with learned numerical settings called parameters. During generation, it uses those settings to score possible continuations of the text it has seen.

Those parameters are also called weights. A forward pass means running the model’s calculations from an input to its output scores using the weights it already has. The model may make many forward passes while producing one answer, but asking it a question does not change those weights.

The model is not searching a fixed list of complete answers. For an ordinary text-generation request, it repeatedly chooses a next token.

What is a token?

A token is one unit of text handled by the model. It may be a word, part of a word, punctuation, or a space joined to another piece. A tokenizer turns text into tokens and gives each token a numerical ID. The model works with those IDs rather than raw letters.

For this first walkthrough, pretend the prompt and answer are split like this:

Prompt:    [Finish] [this] [sentence] [:] [The] [sky] [is]
Output:    [ blue] [.]

These brackets show a made-up teaching split. They are not the output of a particular tokenizer. The leading space in [ blue] is intentional: real tokenizers can include spaces in tokens.

For a real implementation, OpenAI’s tiktoken repository shows how text is encoded to token IDs and decoded back to text. We will use a much smaller example in the next topic so you can see every step.

Follow the two choices

Once the prompt has become token IDs, the model scores candidate tokens that could come next. A score, often called a logit, says how strongly the model favors a candidate in this context. A selection rule turns those scores into a token choice. We will study that rule later; for now, follow the order of events.

Step	Text available to the model	Illustrative next choice
1	Finish this sentence: The sky is	[ blue]
2	Finish this sentence: The sky is blue	[.]

At step 2, the model can use [ blue] because it chose that token in step 1. The process repeats: read the text so far → score possible next tokens → choose one → add it to the text. Generation stops when a stopping rule applies, such as a special end token or a configured length limit.

This is called autoregressive generation: each new choice depends on the tokens already available, including earlier output. It does not mean every answer is inevitable. Different selection settings can lead to different continuations.

Where do training and inference fit?

Training is the earlier process that adjusts the model’s parameters using examples. Inference is using those learned parameters to produce an answer for a new prompt. During an ordinary inference request, the model runs calculations with its existing parameters; it does not retrain itself because you asked one question.

Think of the model’s calculation as a function that receives token IDs and returns scores for what may come next. Track 1 opens that function and shows where the scores come from. This starting lesson only needs the outer loop.

Check your understanding

For the illustrative prompt Finish this sentence: The sky is, write down:

1. The text supplied by the reader.
2. The first generated token in our example.
3. The text available immediately before the second choice.
4. Whether the parameters are being learned or used during this request.

Check: The supplied text is the prompt; [ blue] is the first generated token; before the second choice the model has Finish this sentence: The sky is blue; and inference uses learned parameters. If you can explain why step 2 includes step 1’s choice, you have the main idea.

What comes next

The remaining lessons in Start Here: LLM Foundations are organized into six parts: prompt foundations, the necessary math, the decoder-only transformer, prefill versus decode, the KV cache, and GPU basics. Those lessons are still being written. If you want to continue now, read How LLM inference works for the next layer of the same story, then use Track 1 when you are comfortable with its stated prerequisites.