Skip to content
NutsNews chestnut logoNutsNews Wiki

Technical guide · Core platform

Page status: Draft

NutsNews 3B versus 32B: summaries and resource utilization

Historical screening results and their production limitations.

Visual overview

Primary diagram

System map

A small controlled model screening provides observations, but production stays unchanged until representative repeated chat tests and human review support a change.

Render the repository-owned system map when you need it.

Diagram is not rendered yet.

View as text
NutsNews 3B versus 32B: summaries and resource utilization decision

A small controlled model screening provides observations, but production stays unchanged until representative repeated chat tests and human review support a change.

flowchart TD
  accTitle: NutsNews 3B versus 32B: summaries and resource utilization decision
  accDescr: A small controlled model screening provides observations, but production stays unchanged until representative repeated chat tests and human review support a change.
  inputs["Three source-checked story digests"] --> test["One controlled run per story and model"]
  test --> results["Record speed, length and factual observations"]
  results --> limits["Small sample; not the production chat workflow"]
  limits --> decision["Keep production unchanged"]
  decision --> future["Repeated representative chat tests and human review"]

NutsNews 3B versus 32B: summaries and resource utilization decision

Fullscreen diagram view.

NutsNews 3B versus 32B: summaries and resource utilization

Section titled “NutsNews 3B versus 32B: summaries and resource utilization”

September 13, 2026. Production remains on 3B. Same three factual digests and deployed buildPrompt as the earlier screening. One run per story and model, JSON generate API, temperature 0, seed 42, 8192 context capacity, 220 output-token cap, two inference threads. This is not an exact production chat replay: the extra system message, production sampling and normalization are excluded. No concurrent load test or publishing-queue benchmark was performed. The model download required a retry; download time is excluded.

Measure3B32B
Mean output tokens/sec14.791.56
Mean whole-server CPU12.16%12.44%
Peak sampled whole-server CPU12.74%13.93%
Peak sampled runner RSS2.21 GiB20.71 GiB
Minimum available system RAM55.01 GiB36.44 GiB
Peak swap allocated4 KiB480 KiB
Max sampled health latency4.31 ms3.46 ms
Health failures00
NASA request19.43 sec197.26 sec
Mouse-study request9.64 sec98.88 sec
Lost-toy request9.38 sec100.26 sec
Meets 200–250 character limit3/31/3

CPU uses successive /proc/stat samples approximately two seconds apart, expressed across all 16 logical CPUs; idle and iowait are excluded from busy time. Similar CPU percentages reflect the equal thread cap, not equal work or energy. Percentages include all server activity. Runner RSS is the observed llama-server process, not total Ollama service memory; concurrent runners would make attribution ambiguous. MemAvailable includes reclaimable cache. Swap allocation was negligible, and allocation alone does not measure swap I/O. Short spikes between samples are not measured. Generation was about 9.5 times slower, with 9.4 times the resident model-process memory. First requests include load phases of 2.76 and 20.23 seconds respectively; prefix reuse and live traffic affect timings.

Manual quality review: 32B clearly stated that human effects from the mouse study require clinical confirmation, unlike 3B’s less explicit caveat. It avoided 3B’s bedding distortion in the toy story. NASA’s summary included the project name and AI purpose. However, lengths were 236, 254 and 253 characters, so two still exceeded the limit. The toy summary added generic feel-good commentary. All outputs were valid JSON, accepted the stories, and included reasons. These observations on three digests do not establish general accuracy rates.

Recommendation: 32B may be useful for occasional background review where latency is acceptable, but is not a demonstrated replacement without slowdown. Retain production 3B. A production decision needs representative chat replays, repeated quality evaluation, and NutsNews generation/queue measurements. HTTP health success does not prove unchanged AI processing latency. 32B was unloaded after testing; model files remain for reuse. No service configuration or publishing changed.