Code
cookbook/11_models/llama_cpp/basic.py
Usage
1
Set up your virtual environment
2
Install LlamaCpp
Follow the LlamaCpp installation guide and start the server:
3
Install dependencies
4
Run Agent
Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
from agno.agent import Agent, RunOutput # noqa
from agno.models.llama_cpp import LlamaCpp
agent = Agent(model=LlamaCpp(id="ggml-org/gpt-oss-20b-GGUF"), markdown=True)
# Get the response in a variable
# run: RunOutput = agent.run("Share a 2 sentence horror story")
# print(run.content)
# Print the response in the terminal
agent.print_response("Share a 2 sentence horror story")
Set up your virtual environment
uv venv --python 3.12
source .venv/bin/activate
uv venv --python 3.12
.venv\Scripts\activate
Install LlamaCpp
llama-server -hf ggml-org/gpt-oss-20b-GGUF --ctx-size 0 --jinja -ub 2048 -b 2048
Install dependencies
uv pip install -U agno
Run Agent
python cookbook/11_models/llama_cpp/basic.py
Was this page helpful?