Tag: Python

A working start-to-finish guide. Tested on an M-series MacBook Pro, August 2026.

Read this first: there is one non-obvious step that will waste your afternoon if you miss it — you must convert the weights at BF16, not F16. Details in step 6 and in Troubleshooting. Everything else is routine.

---

What Antares-1B actually is

An open-weight 1B model from Cisco Foundation AI, built on IBM Granite 4.0 1B, specialized for vulnerability localization. Given a CWE description and a repository, it explores the codebase through a terminal — issuing grep, find, cat — and submits a list of files it believes contain the vulnerability.

Two things to internalize before you start:

• It is not a chat assistant. The model card explicitly lists general-purpose chat and instruction following as out-of-scope. Asking it “what are your capabilities” gets you nothing useful even when the setup is perfect.
• It is one half of an agent loop. It emits <tool_call> JSON; your harness executes the command and feeds the output back as <tool_response>. It runs up to 15 terminal calls, then terminates via submit_vulnerable_files or submit_no_vulnerability_found.

License is Apache 2.0. The Hugging Face repo is gated behind a contact-info form. Expect ~100 tok/s on Apple silicon. Benchmark is File F1 0.209 on Cisco’s VLoc Bench — useful for triage, not a scanner replacement.

Prerequisites

• macOS on Apple Silicon
• Homebrew
• ~15 GB free disk (4 GB weights, 3.7 GB GGUF, plus PyTorch)
• A Hugging Face account

1. Accept the model gate

Go to https://huggingface.co/fdtn-ai/antares-1b and complete the access form. Downloads 403 without this. Approval is manual but was quick in practice.

2. Create a Hugging Face token

https://huggingface.co/settings/tokens — read access is enough.

Treat it like a password. If you ever paste one into a chat, ticket, or screen share, revoke it on that page immediately and issue a new one.

3. Install tooling

brew install llama.cpp

This gives you prebuilt llama-cli and llama-server — no compiler needed. You still need the llama.cpp source for the conversion script:

git clone https://github.com/ggml-org/llama.cpp ~/llama.cpp

4. Python environment

Use a venv. The conversion script pulls in PyTorch and you do not want that in your system or pyenv global Python.

cd ~/llama.cpp
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt

requirements.txt already installs huggingface_hub at a compatible version. Do not run pip install -U huggingface_hub — that pulls the 1.x line, which breaks transformers. If you already did:

pip install "huggingface_hub<1.0"
pip check    # should report no broken requirements

5. Download the weights

hf auth login          # paste your token
mkdir -p ~/models
hf download fdtn-ai/antares-1b --local-dir ~/models/antares-1b

Sanity check the chat template came down — you’ll need to match it later:

cat ~/models/antares-1b/chat_template.jinja

6. Convert to GGUF — BF16, not F16

cd ~/llama.cpp
source .venv/bin/activate
python convert_hf_to_gguf.py ~/models/antares-1b \
  --outfile ~/models/antares-1b-bf16.gguf --outtype bf16

This is the step that matters. Granite 4.0 is a hybrid Mamba/attention architecture. The SSM state updates exceed F16’s dynamic range, saturate, and the model collapses into emitting a single repeated token forever. BF16 has the same bit width but far more exponent headroom and survives fine.

Symptom if you get this wrong: the model loads cleanly, reports ~100 tok/s, and prints @@@@@@@@@@@... in response to anything.

This is also why the community mradermacher / mitkox GGUF conversions on the Hub don’t work — they appear to route through an F16 intermediate. Convert it yourself.

7. Verify with llama.cpp before touching Ollama

llama-cli -m ~/models/antares-1b-bf16.gguf --jinja -no-cnv \
  --temp 0.3 --top-p 1.0 \
  -p "Vulnerability to locate: CWE-78 OS Command Injection."

--jinja uses the template embedded in the GGUF, so there is nothing to hand-write and get wrong. This isolates model correctness from Ollama config.

Expected: coherent reasoning prose, then </think>, then a response. The reply starts mid-thought with no opening <think> tag — that is correct. The template pre-fills it.

If you see @@@@, stop here and go to Troubleshooting. Do not proceed.

8. Import into Ollama

Ollama does not pick up the chat template from this GGUF automatically, so supply it explicitly. Without it Ollama sends raw text with no role markers and no stop token, and you get @@@@ again even from a good BF16 file.

cd ~/models
cat > Modelfile <<'EOF'
FROM ./antares-1b-bf16.gguf

TEMPLATE """<|start_of_role|>system<|end_of_role|>{{ if .System }}{{ .System }}{{ else }}You are a helpful assistant. Please ensure responses are professional, accurate, and safe.{{ end }}<|end_of_text|>
{{ range .Messages }}{{ if eq .Role "user" }}<|start_of_role|>user<|end_of_role|>{{ .Content }}<|end_of_text|>
{{ else if eq .Role "assistant" }}<|start_of_role|>assistant<|end_of_role|>{{ .Content }}<|end_of_text|>
{{ end }}{{ end }}<|start_of_role|>assistant<|end_of_role|><think>
"""

PARAMETER stop "<|end_of_text|>"
PARAMETER stop "<|start_of_role|>"
PARAMETER temperature 0.3
PARAMETER top_p 1.0
PARAMETER num_ctx 32768
EOF

ollama create antares-1b -f Modelfile

Note there is no --quantize flag on the first build. Get it working unquantized, then shrink.

Three details in that template, all derived from the model’s real chat_template.jinja:

• A system block is always emitted — the Jinja has no “if system” branch, it injects a default. Making it conditional produces a prompt shape the model never saw in training.
• The generation prompt pre-fills <think>\n. The model starts already inside a reasoning block.
• Temperature 0.3 / top-p 1.0 are the settings Cisco used for every published evaluation. Ollama’s defaults (0.8 / top-k 40 / repeat-penalty 1.1) are far too loose for an agentic model.

Verify:

ollama show --modelfile antares-1b | head -30

If TEMPLATE shows {{ .Prompt }}, the template didn’t take — rebuild.

ollama run antares-1b "Vulnerability to locate: CWE-78 OS Command Injection."

9. Shrink it (optional)

3.7 GB is large for a 2B model. Step down one level at a time and test each:

ollama create antares-1b:q8 --quantize q8_0 -f Modelfile
ollama run antares-1b:q8 "Vulnerability to locate: CWE-78 OS Command Injection."

ollama create antares-1b:q4 --quantize q4_K_M -f Modelfile
ollama run antares-1b:q4 "Vulnerability to locate: CWE-78 OS Command Injection."

If a level produces @@@@ or degraded reasoning, you’ve found the floor for this architecture — stay one level up. Hybrid SSM models are more quantization-sensitive than plain transformers.

10. Run it as an agent

The REPL is a diagnostic, not a workflow. To see real tool calls, give it the system prompt it was trained against:

ollama run antares-1b

>>> /set system You are a security vulnerability localization agent. You have access to a terminal with the repository mounted at /workspace/repo/. Use shell commands (grep, find, cat, etc.) to explore the codebase and identify files that contain the reported vulnerability. When confident, submit your findings.
>>> Vulnerability to locate: CWE-78: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

Success looks like a <tool_call> block containing a shell command. The model is asking you to run it and return the output.

For the real loop, Cisco ships the Antares CLI as a ZIP in the Files tab of the model repo. It implements the full agent loop against any OpenAI-compatible endpoint:

ollama serve        # exposes http://localhost:11434/v1

Configure the CLI with base URL http://localhost:11434/v1, model antares-1b, and any dummy API key. It supports targeted CWE analyses and repo-wide sweeps, with output in human-readable, JSON, or SARIF.

Alternative: skip Ollama entirely and use llama-server, which gives you the same OpenAI-compatible endpoint and tends to get Granite hybrid fixes first:

llama-server -m ~/models/antares-1b-bf16.gguf --jinja \
  -c 32768 --temp 0.3 --top-p 1.0 --port 8080

Sandboxing — not optional

You are executing shell commands the model invents. Per Cisco’s recommendations:

• Run the target repo snapshot in a container with --network=none
• 10 second command timeout
• 2 CPU cores, 4 GB RAM limit
• Destroy the container after each run
• Human review before any remediation action

Troubleshooting

@@@@@@@@@ output

Ranked by likelihood:

1. GGUF converted at F16 instead of BF16 → reconvert (step 6)
2. Ollama Modelfile missing TEMPLATE → check ollama show --modelfile
3. Quantized too aggressively → rebuild unquantized and re-test
4. Using a community GGUF from the Hub → convert it yourself

401: Invalid username or password from ollama pull

Not an auth problem. Hugging Face returns 401 rather than 404 for repos it can’t show you, to avoid leaking whether private repos exist. You almost certainly have the repo name wrong. (mradermacher/Antares-11b-v2-GGUF is a different, unrelated model — an old 11B SOLAR finetune. Nothing to do with Cisco Antares.)

Ollama and gated/private repos

Ollama’s HF integration doesn’t use tokens at all. It authenticates with the SSH key at ~/.ollama/id_ed25519.pub — add that to https://huggingface.co/settings/keys.

pyenv: python: command not found

No pyenv version is active:

pyenv global 3.10.14 && pyenv rehash

If it still fails, your ~/.zshrc is missing eval "$(pyenv init -)".

zsh: bad pattern: ^[[200~

Bracketed-paste escape leaking into zsh. Run exec zsh, or paste one line at a time.

cmake: command not found

You don’t need it — brew install llama.cpp gives prebuilt binaries. Only brew install cmake if you specifically need to build from source (e.g. to pick up a fix newer than the brew formula).

transformers … requires huggingface-hub<1.0

See step 4.

Cross-checking against reference weights

If GGUF misbehaves and you want to know whether the weights themselves are fine:

cd ~/llama.cpp && source .venv/bin/activate && pip install accelerate
python - <<'EOF'
from transformers import AutoTokenizer, AutoModelForCausalLM
import os
p = os.path.expanduser("~/models/antares-1b")
tok = AutoTokenizer.from_pretrained(p)
m = AutoModelForCausalLM.from_pretrained(p, device_map="cpu")
msgs = [{"role":"user","content":"Vulnerability to locate: CWE-78 OS Command Injection."}]
i = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt", return_dict=True)
o = m.generate(**i, max_new_tokens=80, do_sample=True, temperature=0.3, top_p=1.0)
print(tok.decode(o[0][i["input_ids"].shape[-1]:], skip_special_tokens=False))
EOF

Coherent here but garbage from every GGUF = a llama.cpp conversion/runtime gap. Check the llama.cpp issue tracker, or serve with vLLM, which the model card documents as supported.

Quick reference

# Full sequence, assuming gate accepted and brew installed
brew install llama.cpp
git clone https://github.com/ggml-org/llama.cpp ~/llama.cpp
cd ~/llama.cpp && python3 -m venv .venv && source .venv/bin/activate
pip install -U pip && pip install -r requirements.txt
hf auth login
mkdir -p ~/models && hf download fdtn-ai/antares-1b --local-dir ~/models/antares-1b
python convert_hf_to_gguf.py ~/models/antares-1b \
  --outfile ~/models/antares-1b-bf16.gguf --outtype bf16
llama-cli -m ~/models/antares-1b-bf16.gguf --jinja -no-cnv --temp 0.3 -p "test"
# then the Modelfile from step 8
ollama create antares-1b -f ~/models/Modelfile

Known-good settings: BF16 GGUF, temperature 0.3, top-p 1.0, context 32768, explicit Granite <|start_of_role|> template with pre-filled <think>.