2  Reasoning

Some LLMs have been trained to produce intermediate reasoning steps before generating the final response. Following a query, they first generate text that outlines their reasoning process. This becomes part of the model’s context and is used to produce the final response. This step is variously referred to as “reasoning” or “thinking”, with some providers preferring one term over the other. Both terms are metaphorical: models do not “reason” or “think” in a literal human sense. Reviewing the reasoning output can provide insights into how the model arrived at its answer, which can be particularly useful in research and decision-making contexts. For example, it can help reveal choices the model made while trying to disambiguate a query. The final response may focus on one interpretation of the query, while the reasoning output may show that the model considered multiple interpretations before settling on one. This can be of great help in guiding how a change in the context (e.g. the system or user messages) may help lead to the intended response.

The examples in this book will use reasoning models to show how to extract and display the reasoning steps alongside the final response.