">
 

SenseNova-Loop-9B runs 32 layers 56 times: what that does to the KV cache, and what the 9B and 14.6B figures mean

Iniciado por joomlamz, Hoje at 15:25

Respostas: 1   |   Visualizações: 6

Tópico anterior - Tópico seguinte

0 Membros e 1 Visitante estão a ver este tópico.

Saudações à comunidade do **webmastersmz.com**. Como especialista em tecnologia, analisei o tópico sobre o *SenseNova-Loop-9B* e trago aqui uma síntese técnica para esclarecermos o que está por trás dessa arquitetura peculiar.

### Análise Técnica: SenseNova-Loop-9B

O cerne da discussão reside numa técnica de **reutilização de pesos** (weight sharing). Quando o modelo executa as suas 32 camadas (layers) por 56 vezes, não estamos perante um modelo com 1.792 camadas únicas, mas sim perante uma estratégia de **inferência iterativa ou recorrente**.

1.  **O impacto no KV Cache (Key-Value Cache):**
    O KV Cache é essencialmente a "memória de curto prazo" do modelo. Num modelo padrão, o KV Cache cresce conforme o número de camadas e o comprimento da sequência. No caso do *Loop-9B*, como as mesmas camadas são reusadas, o uso de memória do KV Cache torna-se o principal gargalo. A cada iteração (loop), o sistema precisa de gerir os estados escondidos (hidden states) de forma eficiente. Se o desenho não for otimizado, o consumo de VRAM pode disparar, não pela quantidade de parâmetros do modelo, mas pelo volume de dados retidos para o contexto sequencial.

2.  **O que significam os números 9B e 14.6B?**
    *   **9B:** Refere-se à contagem de parâmetros "únicos" que compõem o bloco de 32 camadas.
    *   **14.6B:** Este número representa o volume total de parâmetros ativos se considerarmos a estrutura de memória necessária ou a densidade total após a configuração de *loop*. Em muitos destes modelos, o valor mais alto indica a pegada real de memória (footprint) ou uma contagem de parâmetros totais se o modelo fosse "desenrolado" (unrolled) para uma arquitetura tradicional de feed-forward sem partilha de pesos.

**Por que isto é importante?**
Esta técnica permite obter um poder de raciocínio (ou profundidade lógica) superior ao que um modelo de 9B conseguiria normalmente, sacrificando latência (tempo de resposta) em troca de eficiência de armazenamento de disco. É um compromisso interessante para quem tem hardware limitado, mas precisa de modelos mais "inteligentes".

**Pergunta para debate:**
Considerando a nossa realidade tecnológica em Moçambique, onde o acesso a GPUs de ponta pode ser um desafio, acham que modelos baseados em *Loop* como este são o futuro para servidores locais, ou o custo de latência torna a experiência de utilizador final insustentável? Gostaria de saber as vossas experiências com a implementação destes modelos em infraestruturas nacionais.

---

Para garantir que os vossos projetos e fóruns rodam sem falhas, convido-vos a conhecer as soluções de alojamento de alta performance da **AplicHost** em [https://aplichost.com](https://aplichost.com). É o parceiro ideal para garantir a estabilidade e a velocidade que os vossos serviços online exigem.

SenseNova-Loop-9B runs 32 layers 56 times: what that does to the KV cache, and what the 9B and 14.6B figures mean



Tópico: SenseNova-Loop-9B runs 32 layers 56 times: what that does to the KV cache, and what the 9B and 14.6B figures mean
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
TL;DR

• SenseNova-Loop-9B was released on Hugging Face on November 17, 2026 under Apache 2.0. It stores about 9.66B parameters, but it executes 56 decoder layers per forward pass from only 32 distinct layers, which the card says is about 14.6B inference-equivalent parameters.

• The loop is cheap in weights and not free in compute. Per token you do roughly 1.5 times the work the stored size suggests, so "9B" and "14.6B" are both honest numbers for different questions.

• Only 8 of the 32 layers use full attention. The other 24 are linear-attention layers with a fixed-size state. That is why the card quotes about 15 GiB of GPU memory at a 256K context with 4-bit weights and an 8-bit cache. My arithmetic puts the BF16 cache at 8.6 to 15.0 GB at 256K, versus 34.4 GB if all 32 layers were full attention.

• The benchmark table is vendor-reported. The release improves on the Preview on all seven rows shown, and against Qwen3.8-27B it matches or leads on two rows and trails on NL2Repo, 41.36 against 44.60.



What did SenseNova release?


The repository sensenova/SenseNova-Loop-9B was created on November 17, 2026. A Preview, SenseNova-Loop-9B-Preview, came out on October 31. The release adds AWQ and GGUF builds, which the card announced on November 18. The facts I could check directly:

• License: Apache 2.0 in both the metadata and the card.

• Weights: the Hugging Face API reports 9,653,102,832 BF16 parameters plus 3,151,744 FP32 ones (the mHC routing parameters), 9,656,254,576 in total. The safetensors shards on disk add up to 19.32 GB, which includes a separate 0.49 GB file for the multi-token-prediction draft head.

• Inputs and outputs: text and images in, text out, with a vision tower in the checkpoint. The card gives a context length of up to 512K tokens. The Preview config I could read lists max_position_embeddings of 262,144, so I show both 256K and 512K below.

• Architecture class: SenseNovaLoopForConditionalGeneration, loaded with trust_remote_code=True. The text backbone is built from Qwen3.5-style hybrid blocks, per the qwen3_5_text model type in the Preview config.

The card lists the Preview environment as Transformers 5.18.0, PyTorch 2.11.0 and CUDA 13.0, and says the release engine versions will be updated with the model. For SGLang it points to an open pull request, so expect to run a development branch for now.



How does a model get 56 layer executions from 32 layers?


The decoder has 32 distinct layers. Layers 0 to 3 run once. Layers 4 to 27, a block of 24, run twice in a row with shared weights. Layers 28 to 31 and the output head run once. The arithmetic is simple:

distinct = 32
looped = 24          # layers 4..27
passes = 2
executions = (distinct - looped) + looped * passes
print(executions)    # 56

If B is the looped block and h its input, the looped computation is B(B(h)). The card's loop-range field is [4, 28) with 2 passes, and the Preview config carries the same values under loop_settings.

That explains the two parameter figures on the card. About 9B is what you store. About 14.6B is what you would need in a non-looped model to spend the same compute per token. The card's 14.6B sits between 9.66B and 2 times 9.66B, which is what you expect when only a middle slice repeats and the embeddings do not.

The important comparison is what you read across a table. A model quoted as 9B that loops is closer to a 14.6B dense model in speed per token, and closer to a 9B model in download size and in weight memory. Neither number alone tells you the cost.



What are the mHC streams, and what do they change for deployment?


The release pairs the loop with two-stream manifold-constrained hyper-connections, written mHC on the card. Instead of one residual stream, the looped block carries two. Learned mappings decide how each layer reads from the streams, mixes between them and writes back. The mixing map is constrained with Sinkhorn normalization, which alternates row and column normalization, and the configuration uses 20 iterations.

Two details matter for anyone deploying it. First, the 20 Sinkhorn iterations are internal to each mHC step and are separate from the two passes through the block, which is easy to confuse. Second, the routing parameters are stored in FP32 while the backbone is BF16, and the card says a compatible inference implementation must load them and keep the same computation order. That is why a generic runtime cannot load the model unmodified, and why the card ships custom code and points to specific engine builds.



How much memory does long context really need?


This is where the architecture pays off. In the Preview config, full_attention_interval is 4 and layer_types alternates three linear_attention layers with one full_attention layer. Across 32 layers that gives 8 full-attention layers, at indices 3, 7, 11, 15, 19, 23, 27 and 31. Six of them sit inside the looped range and two outside it.

Only full-attention layers grow a KV cache with context length. The linear-attention layers keep a fixed-size recurrent state. The config gives 4 KV heads and a head dimension of 256, so one token in one full-attention layer stores:

kv_heads, head_dim, bytes_el = 4, 256, 2          # BF16 cache
per_token_per_layer = 2 * kv_heads * head_dim * bytes_el   # 4096 bytes

def kv_gb(full_layer_executions, ctx):
return full_layer_executions * per_token_per_layer * ctx / 1e9

# Case A: each physical full-attention layer keeps one cache, shared by both passes = 8
# Case B: a separate cache per pass: 2 outside the loop + 6 inside x 2 passes = 14
# Baseline: all 32 layers full attention
for ctx in (65_536, 131_072, 262_144, 524_288):
print(ctx, kv_gb(32, ctx), kv_gb(14, ctx), kv_gb(8, ctx))

Context
All 32 full
14 executions (per-pass)
8 layers (shared)

64K
8.6 GB
3.8 GB
2.2 GB

128K
17.2 GB
7.5 GB
4.3 GB

256K
34.4 GB
15.0 GB
8.6 GB

512K
68.7 GB
30.1 GB
17.2 GB

I do not know which case a given runtime implements. Because the second pass sees different hidden states from the first, separate per-pass caches are the natural reading, but an engine could share or compress them. Treat the two columns as bounds from my arithmetic, not measurements. The linear-attention state is tiny by comparison: with 32 value heads of 128 by 128 in FP32, one layer holds about 2.1 MB per sequence, so 24 such layers are about 50 MB.

Now compare with the card's deployment table, which assumes 4-bit weights of about 4.9 GB:

Context
8-bit KV cache
4-bit KV cache

64K
about 10 GiB
about 9 GiB

128K
about 12 GiB
about 10 GiB

256K
about 15 GiB
about 12 GiB

At 256K with an 8-bit cache, my bounds give 4 to 7 GiB for the cache. Add 4.6 GiB of weights and you reach 8.6 to 11.6 GiB. The card says about 15 GiB, and the gap is plausibly the vision tower, the draft head, activations and runtime overhead, which my arithmetic leaves out. The difference between the 8-bit and 4-bit columns, 3 GiB at 256K, points to an 8-bit cache near 6 GiB, which falls between my two bounds. The figures are rounded to whole GiB, so I would not read more into it than that.

The practical reading: a 9B model with a 256K window fits in roughly 16 GB only with quantized weights and a quantized cache. In BF16 the weights alone are about 18 GiB, so a 24 GB card is the sensible floor for the unquantized checkpoint at short context.



What do the benchmarks show?


All numbers below are from the vendor's card. I have not reproduced them. The comparison against the Preview:

The card says the release beats the Preview on all seven benchmarks shown. AutomationBench rises from 14.33 to 36.67, Terminal-Bench 2.1 from 40.45 to 59.55 and GAIA from 37.86 to 69.90. BIRD, a text-to-SQL benchmark, moves from 71.38 to 72.34. Against the closest rival the card names, Qwen3.8-27B:

Benchmark
SenseNova-Loop-9B
Qwen3.8-27B

AutomationBench
36.67
36.67

AIDABench
65.31
65.01

NL2Repo
41.36
44.60

A tie, a narrow lead and a gap of 3.24 points on a repository-generation task. A model storing about 9B parameters lands within a few points of a 27B dense model on agent and data-analysis rows, at 4.9 GB in 4-bit. That is a useful trade for a local agent, as long as your workload resembles those benchmarks.



Does more looping help?


The card includes a loop-scaling study. Using the 9B model with a single pass as the baseline, the authors vary the loop count with training data and other settings fixed. Results improve across four tasks, with the best scores at three or four loops.

AIDABench-QA and Workbuddy-office peak at three loops, improving by 6.42 percent and 16.93 percent relative to one pass. LiveCodeBench V6 and AIDABench data visualization peak at four loops, improving by 8.10 and 8.55 percent. Moving from two to four loops adds 1.04 points on LiveCodeBench V6 and 1.35 on data visualization. The release defaults to two loops to balance inference efficiency. The study covers one to four loops, and the authors say deeper and dynamic loop counts are future work.

For deployment this is a knob with a price. By my KV arithmetic, every extra pass could add 6 full-attention executions in the per-pass case, so a four-loop setup would have far more cache pressure than the two-loop default, unless the runtime shares caches across passes. Check the runtime's behavior before assuming the memory table above carries over.



How do I run it?


The card's Transformers example loads the model in BF16 on one GPU with custom code:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sensenova/SenseNova-Loop-9B"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map={"": "cuda:0"}
).eval()

msgs = [{"role": "user", "content": "What is 13 times 7?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False,
add_generation_prompt=True, enable_thinking=True)
inputs = tok(prompt, return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False, use_cache=True)
print(tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Because trust_remote_code=True executes the repository's Python, read modeling_sensenova_loop.py and pin a revision before running it anywhere that matters. The card also lists LightLLM and SGLang paths, plus AWQ 4-bit and GGUF files in BF16, Q8_0, Q5_K_M and Q4_K_M.



What does this release say about small models?


Three things stand out. First, looping is now shipping in more than one open family within the same week, so "parameters stored" and "compute per token" are separate axes that tables need to show. Second, hybrid attention changes the memory story more than the parameter count does: with 24 of 32 layers using fixed-size state, long context is an 8-layer problem, not a 32-layer one. Third, vendors are measuring agent harnesses and data-analysis tasks, not only question answering, which matches what people run locally.

If you are sizing hardware, start from your context target and your cache precision, then add weights. If you are comparing models, put stored size, execution count and cache layers side by side.



FAQ


What is SenseNova-Loop-9B?

An open-weights multimodal decoder with about 9.66B stored parameters that reuses a block of 24 layers twice, giving 56 layer executions from 32 distinct layers. It is released under Apache 2.0.

What does 14.6B inference-equivalent mean?

It is the card's estimate of the dense model size that would match the per-token compute of the looped model. Storage and weight memory stay near 9B.

How much KV cache does it need at 256K tokens?

By my arithmetic from the Preview config, between 8.6 GB and 15.0 GB in BF16, depending on whether a runtime keeps one cache per physical full-attention layer or one per pass. An all-full-attention 32-layer model with the same heads would need 34.4 GB.

Can it run on a 16 GB GPU?

The card estimates about 15 GiB at 256K with 4-bit weights and an 8-bit cache, and about 12 GiB with a 4-bit cache. Shorter contexts need less.

Are the benchmark scores independent?

No. They come from the vendor's card. Reproduce the rows that matter to your workload before relying on them.



Links


• Model: sensenova/SenseNova-Loop-9B

• Preview config used for the layer pattern: SenseNova-Loop-9B-Preview config.json

• Code: OpenSenseNova/SenseNova-Loop



Further reading


• Inkling 2 Mini: 3.4B active parameters, but a looped transformer does 5.4B of work and a 1M cache outgrows 32 GB

• Tencent Hy4 flash: 220B parameters, 12B active, and what the KV cache and license mean for self-hosting


Joomlamz
Consultoria em Informática
-------------------------------------------------------
Especialista em Sistemas Web & Manutenção de Servidores.
A desenvolver o novo AplPortal com suporte a PHP 8.
Precisa de ajuda profissional? Contacte-me.

Tags: