">
 

How to Run Cloudflare Clairic-Flash Locally (Its Runtime Fails on RTX 3090/4090)

Iniciado por joomlamz, Ontem às 23:25

Respostas: 1   |   Visualizações: 4

Tópico anterior - Tópico seguinte

0 Membros e 1 Visitante estão a ver este tópico.

Saudações à comunidade do **webmastersmz.com**. Como especialista em tecnologia, analisei o desafio técnico levantado sobre a execução local do *Cloudflare Clairic-Flash* em hardware robusto como as RTX 3090/4090.

### Análise Técnica: O Desafio do Clairic-Flash

O *Clairic-Flash* é uma ferramenta que utiliza modelos de *machine learning* para inferência e análise. Quando os utilizadores enfrentam falhas de *runtime* em GPUs topo de gama (série RTX 30/40), o problema raramente é falta de poder de processamento, mas sim uma incompatibilidade a nível de "stack" de software. Aqui estão os pontos críticos:

1.  **Compatibilidade CUDA e Toolkit:** Frequentemente, o código exige versões específicas do *NVIDIA CUDA Toolkit*. Se o seu ambiente estiver a rodar uma versão mais recente (ex: CUDA 12.x) e o projeto foi compilado para a versão 11.8, ocorrerão erros de *Illegal Instruction* ou falhas de carregamento de kernels.
2.  **Gestão de Memória VRAM (Flash Attention):** Embora estas GPUs tenham 24GB de VRAM, bibliotecas como a *Flash Attention* possuem requisitos de arquitetura específicos (Ampere para a 3090, Ada Lovelace para a 4090). A configuração incorreta destas dependências durante a instalação (`pip install` ou `conda`) pode causar *crashes* imediatos.
3.  **Drivers e Bibliotecas Dinâmicas:** A falta de compatibilidade entre o *driver* NVIDIA instalado no SO e a versão do *PyTorch/TensorFlow* é uma causa comum de falha silenciosa. Recomendo verificar sempre o log de erro (`dmesg` ou os logs do Python) em busca de "cuBLAS status error".
4.  **Limitações de Runtime:** Muitos destes projetos são desenvolvidos em ambientes Linux (Ubuntu). Tentar rodar sobre WSL2 no Windows exige uma configuração de *passthrough* de GPU que, por vezes, limita a capacidade da placa de gerir processos paralelos intensivos.

**Sugestão de Debug:** Tentem isolar o erro através de um ambiente virtual limpo (Conda), garantindo que as dependências do `requirements.txt` estão a ser instaladas com as flags de arquitetura da GPU (`TORCH_CUDA_ARCH_LIST`).

---

**Incentivo ao Debate:**
Gostaria de convidar os membros do **webmastersmz.com** a partilharem as suas experiências: Alguém aqui já conseguiu otimizar modelos de inferência local em hardware NVIDIA com sucesso? Quais foram os maiores obstáculos que encontraram com dependências de Python ou CUDA? Vamos elevar o nível técnico do nosso fórum com este debate!

---

Para garantir que os vossos projetos e fóruns rodam sem falhas, convido-vos a conhecer as soluções de alojamento de alta performance da AplicHost em https://aplichost.com.

How to Run Cloudflare Clairic-Flash Locally (Its Runtime Fails on RTX 3090/4090)



Tópico: How to Run Cloudflare Clairic-Flash Locally (Its Runtime Fails on RTX 3090/4090)
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
If you're planning to run Cloudflare's new open-weight vision model, clairic-flash, locally on a 24 GB RTX 3090, two things will bite you. Cloudflare's official runtime doesn't work on that card, even in BF16. And the GGUFs you'll find for llama.cpp quietly drop the part that makes Clairic worth using.

Hugging Face lists clairic-flash as an image-text-to-text model, so the obvious move is to load it into a chat UI. Don't. It's a 9B extraction engine: you hand it a document plus a typed schema, and it hands back values, confidence scores and null probabilities. Most of that behaviour lives in a clairic_runtime/ folder of Python code and a separate typed head, neither of which any GGUF contains.

I went through the VRAM numbers for BF16 and int4, the compute-capability check that breaks RTX 30- and 40-series cards, the MLX route on Apple Silicon, and what llama.cpp actually gives you.



What Cloudflare released: specs, license and Workers AI pricing


On November 18, 2026 Cloudflare published two open-weight models from a group it now calls the "Workers AI superintelligence team". The post opens with "Cloudflare has become a neolab." I'll leave that one alone.

The facts from the model card and config.json:

clairic-flash
clairic

Size
9B
27B

Base model
Qwen/Qwen3.5-9B
Qwen/Qwen3.8-27B

License
Apache-2.0
Apache-2.0

Architecture

Qwen3_5ForConditionalGeneration, 32 layers (24 linear-attention Gated DeltaNet + 8 full attention)

Config context
262,144 positions

Runtime default max_length

65,273 tokens

BF16 weights on disk
~19.3 GB in 4 shards, plus a 261 MB clair_head.safetensors

Workers AI ID
@cf/cloudflare/clairic-flash
@cf/cloudflare/clairic

Workers AI price
$0.10 per M input tokens
$0.24 per M input tokens ($0.10 until Dec 1, 2026)

The card itself calls the 9B "9B"; the Hugging Face size widget rounds it to 10B params (and the big one to 28B).

Ignore the 262K context number. The backbone supports it, but the official loader load_release_model() defaults to max_length=65_273, and systemone() refuses to change that per call. Your inputs are also capped at four images or PDF pages, 4 MiB per embedded file, 8 MiB combined. Remote URLs are rejected.



Benchmarks: nothing against Qwen-VL or Gemma


If you came for a comparison table against Qwen2.5-VL, Gemma 3 or SmolVLM, there isn't one. The model card has no benchmark section at all.

The blog post has evals, but they're Cloudflare's own sets, compared against Liquid AI's D1.2 (with Apple Vision OCR in front of it) and ScaleDown's Extract. I've left out the Clairic-preview column:

Eval
Clairic
Clairic-flash
D1.2 + OCR
ScaleDown

Fresh text fields, 96 requests
335/336
328/336
304/336
259/336

Mixed-input fields, 36 requests
221/226
213/226
116/226
156/226

Tracked customer assertions, 139 request shapes
1,263/1,263
1,263/1,263
973/1,263
653/1,263

Synthetic receipt fields
576/576
576/576
397/576
280/576

Cloudflare's own words: "take these evals as directional guidance since they are self-reported." One row is literally labelled "Internal eval (mchen)". I appreciate the honesty. These numbers still prove very little to anyone outside Cloudflare, and a perfect 576/576 on synthetic receipts mostly tells me the training data had synthetic receipts in it.

Latency is the more interesting claim: on a self-hosted server Cloudflare reports clairic-flash at 155/328 ms p50/p95 on text fields and 305/605 ms on image fields. Note that's the full HTTP response with every field in the request, measured on their hardware, which they don't name.



Why the clairic-flash GGUF quants miss the point


Clairic is a Qwen backbone plus three things:

• A request pipeline (clairic_runtime/) that rewrites each field into its own prompt, builds an xgrammar constraint from the field schema, scores null, retries, and assembles lists and tables.

• A typed head (clair_head.safetensors) that answers dates, durations, compute, estimate and forecast fields from hidden states in one forward pass.

• Grounding code that returns character spans into your source text.

The mlx-community port says it plainly: mlx_lm.load / mlx_vlm.load "will not reproduce Clairic". The Xylolabs GGUF repo is upfront too: the head and runtime "are not in these files."

That doesn't make the GGUFs useless. You get a Qwen3.5-9B that has seen a lot of extraction training, and with llama.cpp's JSON-schema grammar you can approximate the behaviour. But you lose the confidence scores, p_null, spans and the forecast head. Those were the reasons to pick this model over plain Qwen in the first place.



Running clairic-flash the official way (Transformers, CUDA)


There's no vLLM or SGLang recipe in the card. The documented path is Transformers plus Cloudflare's code, which is loaded from the snapshot via sys.path, so read it before you run it.

python3.11 -m venv .venv && source .venv/bin/activate
hf download Cloudflare/clairic-flash --local-dir ./clairic-flash
pip install -r ./clairic-flash/requirements.txt

That requirements.txt pins torch==2.13.0, transformers==5.10.2, xgrammar==0.2.1, flash-linear-attention==0.6.0, sgl-deep-gemm==0.1.5.post3, and a FlashInfer JIT cache built for cu130, so keep it in a fresh venv and away from any environment you care about.

import sys
sys.path.insert(0, "./clairic-flash")
from clairic_model import load_release_model, systemone

model, processor = load_release_model("./clairic-flash", device="cuda")
response = systemone(model, processor, {
"state": "Invoice from Northwind Supply. Issued 2026-09-02. "
"Items: Pens; Paper. Total: $1080.00.",
"fields": {
"vendor": {"type": "text", "instructions": "The invoice vendor."},
"total": {"type": "number", "decimals": 2},
"issued": {"type": "date"},
},
})
print(response["answers"])



Why the official runtime fails on RTX 3090 and 4090


This is where most home setups will break. According to stuchapin's AutoRound repo, the shipped runtime sends every contiguous BF16 GEMM to DeepGEMM, which raises Unsupported architecture below compute capability 9.0. The repo's own words: the original runtime "fails on RTX 30-series cards even with the BF16 weights." That repo only tested a 3090, but the cutoff is the cutoff: an RTX 4090 is compute capability 8.9, so it sits under the same line. The kernel choices assume Hopper-class data center GPUs.

That repo patches clairic_runtime/math.py to use the runtime's own Triton matmul fallback below 9.0 and quantizes the MLP and full-attention layers to int4. Their reported numbers on one 3090:

BF16
W4A16

Weights in GPU memory
18.2 GiB
11.1 GiB

Peak during test
21.2 GiB
14.2 GiB

13 requests, warm
62.0 s
32.6 s

Fields correct
46/49
47/49

So BF16 needs a 24 GB card and the int4 build fits a 16 GB one. The AutoRound kernels compile on first use: about 11 minutes on their first run, 20 to 135 s per request, cached on disk after that. Their absolute timings are also slower than Cloudflare's because of the fallback kernels, so look at the BF16 vs. int4 ratio, not the seconds. The accuracy misses are the part I'd act on: both precisions got a subtotal + tax + shipping sum and a "net 30" due date wrong, at confidence ≤ 0.18. Extract the inputs with the model and do the arithmetic in your own code.



Clairic-flash on Apple Silicon: MLX works, 4-bit costs accuracy


The mlx-community/clairic-flash-4bit build keeps Cloudflare's pipeline and swaps the engine. It quotes 6 GB at rest and up to 11 GB on their timed requests, so 16 GB is the practical floor. It also ships server.py with a /v1/systemone endpoint.

Against the official runtime on an H100, the 4-bit build matched 247/262 fields. The typical mistakes were misread digits like 2231.5 for 2231.25. On a smaller 77-request subset that leaves out the longest documents, 4-bit matched 231/246 and their 8-bit build (10.8 GB) matched 246/246. A mixed recipe with the first 20 layers at 6-bit and the rest at 4-bit (7.3 GB) got 241/246. If you're extracting invoice numbers, use 8-bit or the mixed 6/4-bit recipe; the extra RAM is cheaper than a wrong digit in an invoice ID.



llama.cpp, GGUF and Ollama


For llama.cpp, Xylolabs ships quants plus a vision projector (mmproj-clairic-flash-BF16.gguf), with Q4_K_M at 5.69 GB and a reported mean KL of 0.0468 against an f32 reference:

llama-server -m clairic-flash-Q4_K_M.gguf --mmproj mmproj-clairic-flash-BF16.gguf \
--jinja -ngl 99 -c 32768 --chat-template-kwargs '{"enable_thinking":false}'

mradermacher has static quants from Q2_K to Q8_0 plus f16, with no mmproj file in the list, so treat them as text-only. Like the Xylolabs files, they're the backbone without the head. I found no official Ollama listing in the card or blog.



Should you self-host it or just use Workers AI?


The weights are Apache-2.0, so you're free to fine-tune them, ship them and sell products on top. That part is a good release. The rest of the packaging is clearly Workers AI infrastructure with the door left open: Hopper-first kernels, a CUDA 13 wheel pinned in requirements, @cf/cloudflare/clairic-flash accepted as a model ID in the local API, and benchmarks that only compare against other niche extraction products. At $0.10 per million input tokens with output tokens free, Workers AI wins for most people unless the data can't leave your network.

If your data can't leave the building, though, the 16 GB int4 path is real, and it beats prompting a general VLM for JSON and hoping.

Has anyone gotten the official runtime going on a 4090 or an RTX 50-series card, or is everyone outside Hopper stuck on the patched build? And if you've put it through real invoices, I want to hear about the misses.



Sources


• Cloudflare/clairic-flash model card

• Cloudflare blog: An updated Clairic and Clairic-flash are now open-weight

• Cloudflare/clairic (27B)

• stuchapin/Clairic-Flash-W4A16-AutoRound

• mlx-community/clairic-flash-4bit

• Xylolabs/clairic-flash-GGUF

• mradermacher/clairic-flash-GGUF


Joomlamz
Consultoria em Informática
-------------------------------------------------------
Especialista em Sistemas Web & Manutenção de Servidores.
A desenvolver o novo AplPortal com suporte a PHP 8.
Precisa de ajuda profissional? Contacte-me.

Tags: