K2-Horizon-375B-A23B
Institute of Foundation Models' open-weights 375B-A23B MoE flagship LLM from the K2-Horizon family.
Model Description
K2-Horizon-375B-A23B is the flagship of the K2-Horizon family: a sparse Mixture-of-Experts model that stores 375B parameters and runs 23B per token, with a 512K context window. We have released the final checkpoint; intermediate checkpoints, along with the data and the training code, will be released.
K2-Horizon-375B-A23B Highlights
- Frontier-class agentic performance. On agentic tool use, terminal, and long-horizon workflow benchmarks it matches or beats open-weight MoE models up to 2.6× its size and is competitive with closed frontier models (see Benchmark Results).
- 512K context. Native 524,288-token context from the midtraining stages onward.
- Intermediate checkpoints. Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
- Fully open. Training data/recipe and the training code will be made public.
Benchmark Results
Quickstart
Serving
vLLM, recipe at recipes.vllm.ai/IFM:
vllm serve IFM/K2-Horizon-375B-A23B \
--revision main \
--tensor-parallel-size 8 \
--enable-expert-parallel \
--trust-remote-code \
--dtype bfloat16 \
--max-model-len 131072 \
--reasoning-parser k2_horizon \
--tool-call-parser k2_horizon \
--enable-auto-tool-choice
SGLang recipe validated on 8× H200 in the SGLang K2 Horizon cookbook:
python3 -m sglang.launch_server \
--model-path IFM/K2-Horizon-375B-A23B \
--revision main \
--tp 8 \
--ep 8 \
--dtype bfloat16 \
--attention-backend fa3 \
--model-loader-extra-config '{"enable_multithread_load":false}' \
--reasoning-parser k2_horizon \
--tool-call-parser k2_horizon \
--host 0.0.0.0 --port 30000
API Usage
[!Tip] Recommended settings:
reasoning_effort="high",temperature=1.0,top_p=0.95. Reasoning depth is selected per request throughchat_template_kwargs. Thinking is returned inreasoning_contentand the answer incontent.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="IFM/K2-Horizon-375B-A23B",
messages=[{"role": "user", "content": "Explain the result step by step."}],
temperature=1.0,
top_p=0.95,
max_tokens=32768,
extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)
Transformers
Validated with Transformers 4.57.6, PyTorch 2.13.0, Safetensors 0.8.0.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "IFM/K2-Horizon-375B-A23B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)
inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Best Practices
- Reasoning effort: always
high. All reported results use high reasoning effort. Pass{"chat_template_kwargs": {"reasoning_effort": "high"}}on every request. - Sampling parameters.
temperature=1.0,top_p=0.95. - Serving. Use the validated SGLang recipe above: BF16, TP=8 on one 8× H200 node, FlashAttention-3, wit
Sign up to read complete case studies, access detailed metrics, and unlock all use cases.
Sign up to read complete case studies, access detailed metrics, and unlock all use cases.