10  Using Ollama in R

The goal of this brief tutorial is to demonstrate how to use Ollama from R to generate responses from a local LLM.

You will learn how to:

10.1 Requirements

  • Ollama
  • httr2 and jsonlite R packages

10.2 Setup

10.2.1 Install packages

If not already installed, use either install.packages() or pak to install the required packages:

pak::pak(c("httr2", "jsonlite"))

10.2.2 Load packages

10.2.3 Ollama API endpoint

ollama_url <- "http://localhost:11434/api/generate"
model_name <- "qwen3:8b"

10.3 Generate a free text response

10.3.1 Define the API endpoint and the request body

The following shows you how to define:

  • model: The name of the model to use.
  • prompt: The prompt to send to the model. This is often called the “user” message or prompt.
  • stream: Whether to stream the response token by token. Streaming is useful for interactive applications, like chat interfaces.
  • options: A list of options to control the model’s behavior, like temperature.
request_body <- list(
  model = model_name,
  system = "You are a meticulous research assistant.",
  prompt = "What is your name and who made you?",
  stream = FALSE,
  options = list(
    temperature = 0.2
  )
)

10.3.2 Build and perform the request

We use httr2’s req_url(), req_body_json(), and req_perform():

resp <- request(ollama_url) |>
  req_body_json(request_body) |>
  req_perform()

10.3.2.1 Process the response

The response is an httr2_response object.

class(resp)
[1] "httr2_response"

We can convert it to a named list by combining httr2’s resp_body_json():

resp_list <- resp_body_json(resp)

If the model used is a “reasoning” model, like qwen3:8b, the response will contain a “thinking” field. All models include a “response” field.

In this case, since we used a reasoning model, we can print the reasoning step followed by the final response:

{
  cat("--- Reasoning ---\n")
  cat(resp_list[["thinking"]])
  cat("\n--- Response ---\n")
  cat(resp_list[["response"]], "\n")
}
--- Reasoning ---
Okay, the user is asking for my name and who made me. Let me start by confirming my name. I'm Qwen, which is the name given by Alibaba Cloud. Now, about who made me, I should mention that I was developed by Alibaba Cloud's Tongyi Lab. It's important to note that I'm a large-scale language model, so I should explain that I was created through advanced AI research and development. I should also highlight the collaboration between the Tongyi Lab and Alibaba Cloud. Maybe I should add a bit about the purpose of my creation, like to assist with various tasks and provide information. Wait, the user might be interested in knowing more about the team or the process, but since they asked specifically for the name and creator, I should keep it concise. Let me make sure I don't mention any other details unless necessary. Also, I should use a friendly tone to keep the response approachable. Alright, that should cover it.

--- Response ---
My name is Qwen, and I was developed by Alibaba Cloud's Tongyi Lab. I am a large-scale language model created through advanced AI research and development, designed to assist with a wide range of tasks and provide helpful information. The team at Tongyi Lab and Alibaba Cloud collaborated to bring me to life, focusing on making me capable of understanding and responding to complex queries in a natural and efficient way. 😊

In the above code block, we used braces {} in order to make Quarto print all cat() outputs together.

10.4 Generate a free text response without reasoning

In the above example, we used a reasoning model which produces both a reasoning trace and a final response. Ollama allows turning off the reasoning step using the think option. One of the advantages of turning off the reasoning step is that the response is generated faster, as the model does not need to spend time generating the intermediate reasoning steps.

See Ollama’s supported reasoning models documentation for a current list of supported reasoning models.

Create and perform the request:

request_body <- list(
  model = model_name,
  system = "You are a meticulous research assistant.",
  prompt = "What is your name and who made you?",
  stream = FALSE,
  think = FALSE,
  options = list(
    temperature = 0.2
  )
)
resp <- request(ollama_url) |>
  req_body_json(request_body) |>
  req_perform()
resp_list <- resp_body_json(resp)

Verify that there is no “thinking” field in the response:

"thinking" %in% names(resp_list)
[1] FALSE

Print the final response:

cat(resp_list[["response"]], "\n")
My name is Qwen, and I was developed by Alibaba Cloud. I am a large-scale language model designed to assist with a wide range of tasks, including answering questions, creating content, and providing information. If you have any questions or need assistance, feel free to ask!

10.5 Generate structured output

There are many scenarios, especially in research, where we want to generate a structured response instead of free text. Many users try to achieve this using instructions added to the user prompt and LLMs are increasingly good at following these instructions. However, there is native support to define an output schema including the required names and their descriptions, which is much cleaner, easier, and more likely to succeed, without requiring laborious and extensive prompting.

We begin by defining the output schema. To do this, we create a named list that follows the JSON schema format:

  • type = "object": Indicates that the output is a JSON object.
  • properties: A named list where we define the fields we want in the output. The name of each element is the field name, and its value is another list defining the field’s type (e.g. number, string, etc.) and its description.
  • required: A character vector listing the names of the required fields that must be present in the output.

We can define the schema using a named list:

LLMinfo_schema <- list(
  type = "object",
  properties = list(
    name = list(
      type = "string",
      description = "Your name"
    ),
    manufacturer = list(
      type = "string",
      description = "The name of the person, group, or company that built you."
    ),
    knowledge_cutoff = list(
      type = "string",
      description = "Your knowledge cutoff date."
    )
  ),
  required = c("name", "manufacturer", "knowledge_cutoff")
)

Let’s rerun the previous query, but this time we will include the output schema in the request.

When generating structured output, you might choose to adjust the prompt to explicitly ask for responses that conform to the schema. This is not necessary, as models, especially those designed for structured outputs, will usually adhere to the schema without additional prompting. Note that in this case, we are purposely using the same prompt as before, while the schema is actually requesting for a third field, knowledge_cutoff, which is not mentioned in the prompt.

request_body_structured <- list(
  model = model_name,
  system = "You are a meticulous research assistant.",
  prompt = "What is your name and who made you?",
  stream = FALSE,
  format = LLMinfo_schema,
  options = list(
    temperature = 0.2
  )
)

Perform the request:

resp_structured <- req_perform(
  req_body_json(
    request(ollama_url),
    request_body_structured
  )
)

If you want, you can combine the above into a single piped expression:

resp_structured <- request(ollama_url) |>
  req_body_json(list(
    model = "qwen3:8b",
    system = "You are a meticulous research assistant.",
    prompt = "What is your name and who made you?",
    stream = FALSE,
    format = LLMinfo_schema,
    options = list(
      temperature = 0.2
    )
  )) |>
  req_perform()

10.5.1 Process the structured response

resp_structured_list <- resp_body_json(resp_structured)

The response field is now a string in JSON format:

cat(resp_structured_list[["response"]])
{"name": "Qwen", "manufacturer": "Alibaba Cloud", "knowledge_cutoff": "2024年04月"}

We can use jsonlite::prettify() on the response field to pretty print it:

prettify(resp_structured_list[["response"]])
{
    "name": "Qwen",
    "manufacturer": "Alibaba Cloud",
    "knowledge_cutoff": "2024年04月"
}

You can use jsonlite::fromJSON() to convert the response field into a named list:

fromJSON(resp_structured_list[["response"]])
$name
[1] "Qwen"

$manufacturer
[1] "Alibaba Cloud"

$knowledge_cutoff
[1] "2024年04月"

Ollama does not currently support output of both reasoning and structured response in a single call. This is likely to be fixed in future versions.

10.5.2 Validate the response

We can optionally use the jsonvalidate package to validate the response:

is_valid <- jsonvalidate::json_validate(
  json = resp_structured_list[["response"]],
  schema = jsonlite::toJSON(LLMinfo_schema),
  verbose = TRUE
)

Produce warning if the response does not conform to the schema:

if (is_valid) {
  cat("The response conforms to the schema\n")
} else {
  warning("The response does not conform to the schema.")
}
The response conforms to the schema