Neutrino-1 8B

Neutrino-1 8B is Fermion Research's open-weight 8.19B language model in a compact ternary-family format, designed to serve from one artifact on GPUs, Macs, and desktop CPUs.

Platform: Replicate

Language ModelLocal LLMTernary WeightsTool CallingOpen WeightFermion Research

Fermion Python engine / Fermion llama.cpp CPU-CUDA fork / MLX on Apple silicon

Open-weight Apache 2.0 model; local inference cost depends on hardware, context length, runtime, and electricity

Commercial

🚀Function Overview

Neutrino-1 8B is an 8.19B-parameter decoder-only model with a 40,960-token context window, a 2.56 GB coded download, a 3.88 GB runtime container, and official CPU, CUDA, MLX, and local API paths.

Key Features

Official Hugging Face model ID FermionResearch/Neutrino-8B
8,190,735,360 total parameters across 36 decoder layers
40,960-token context window
2.56 GB coded download and 3.88 GB runtime container
Apache License 2.0 open weights derived from Qwen3-8B
Grouped-query attention with 32 query heads and 8 key-value heads
Local OpenAI-compatible server with streaming and tool calling
CPU and CUDA path through Fermion's llama.cpp fork
MLX path for Apple silicon
Matched Neutrino-1 0.6B draft model for speculative decoding
Custom format is not compatible with stock llama.cpp, Ollama, or LM Studio

Use Cases

•Local chat and text generation on Apple silicon or a desktop CPU
•OpenAI-compatible local inference for existing applications
•Tool-calling experiments with a compact open-weight model
•Bandwidth-efficient single-stream GPU serving
•Speculative decoding with a matched 0.6B draft model
•Evaluating compact model quality, memory, and throughput on owned hardware

⚙️Input Parameters

model

string

Use FermionResearch/Neutrino-8B with the Fermion engine or the locally downloaded model path.

messages

array

OpenAI-compatible conversation payload for local chat, generation, and tool-use workflows.

tools

array

Optional function definitions for tool-calling workflows served by the local OpenAI-compatible endpoint.

max_tokens

integer

Maximum generated output length; keep the combined prompt and generated content inside the 40,960-token context.

💡Usage Examples

Example 1

Input Parameters

{
  "model": "FermionResearch/Neutrino-8B",
  "messages": [
    {
      "role": "user",
      "content": "Summarize the failure, identify the likely root cause, and list the smallest checks that could disprove it."
    }
  ],
  "max_tokens": 512
}

Output Results

The local server returns an OpenAI-compatible completion. Validate tool schemas, formatting, latency, and answer quality on the exact Fermion runtime and hardware used in production.

Quick Actions

Use NowView Documentation

Technical Specifications

Hardware Type: Fermion Python engine / Fermion llama.cpp CPU-CUDA fork / MLX on Apple silicon
Commercial Use: Supported
Pricing: Open-weight Apache 2.0 model; local inference cost depends on hardware, context length, runtime, and electricity
Platform: Replicate

Related Keywords

Neutrino-1 8BNeutrino 8BFermion Research Neutrinoternary language modelcompact 8B modellocal LLM 16GB MacNeutrino OpenAI serverNeutrino speculative decodingNeutrino llama.cpp

Related Models

Bielik 1.5B v3 Instruct

Bielik-1.5B-v3-Instruct is a generative text model featuring 1.6 billion parameters. It is result of collaboration between the open-science/open-souce project SpeakLeash and the High Performance Computing (HPC)

Cordia-A6 Text Generation Model

A model for generating text sequences based on input prompts and adjustable parameters.

Claude Sonnet 4

Claude Sonnet 4 is a significant upgrade to 3.7, delivering superior coding and reasoning while responding more precisely to your instructions