Skip to content
NutsNews chestnut logoNutsNews Wiki

Simple guide · Public product

Page status: Active

Multi-language summaries

Issue #26 adds a future-ready language layer for NutsNews summaries.

Visual overview

Primary diagram

System map

Issue #26 adds a future-ready language layer for NutsNews summaries.

Render the repository-owned system map when you need it.

Diagram is not rendered yet.

View as text
Multi-language summaries

Issue #26 adds a future-ready language layer for NutsNews summaries.

flowchart TD
  accTitle: Multi-language summaries
  accDescr {
    Issue #26 adds a future-ready language layer for NutsNews summaries.
  }
  A[Article accepted in worker] --> B[Persist base English article row]
  B --> C[Create translation tasks per enabled language]
  C --> D[Try local AI /translate]
  D -->|Success| E[Store summary row in article_summaries]
  D -->|Retry or fail| F[Attempt OpenAI fallback]
  F -->|Success| E
  F -->|Failure or retry exhausted| G[Mark translation pending/skip publish]
  E --> H[Publish only when required language rows exist]
  H --> I[API serves /api/articles with lang parameter]

Multi-language summaries

Fullscreen diagram view.

Multi-language summaries

Section titled “Multi-language summaries”

Issue #26 adds a future-ready language layer for NutsNews summaries.

The first supported translated language is French (fr). The design keeps the original article/source link unchanged and stores generated translated titles and summaries separately from the canonical article row.

Supported languages

Section titled “Supported languages”

Current public language options:

CodeLabel
enEnglish
frFrançais

Future languages should be added to web/lib/languages.ts and to the Worker SummaryLanguageCode union/config parsing.

Database design

Section titled “Database design”

Canonical articles stay in public.articles.

Localized summaries are stored in:

public.article_summaries

Important columns:

ColumnPurpose
original_urlStable key back to public.articles.original_url
language_codeTarget language such as fr
source_language_codeOriginal summary language, currently en
titleLocalized card title
summaryLocalized card summary
generated_byTranslation provider: local when the home server generated it, otherwise openai after fallback
modelTranslation model

unique(original_url, language_code) makes each article/language pair upsertable.

Worker behavior

Section titled “Worker behavior”

When an article is accepted and saved, the Worker:

  1. Saves the English/default article row in public.articles.
  2. Builds one translation task for each enabled language.
  3. Calls the home-server local AI service first through POST {LOCAL_AI_URL}/translate.
  4. Retries the home-server translation once when the request throws, returns a non-OK status, returns invalid JSON, or misses the translated title/summary.
  5. Falls back to OpenAI if the home-server translation still fails.
  6. Retries the OpenAI translation once before giving up.
  7. Upserts successful results into public.article_summaries.
  8. Publishes only articles that have all enabled summary-language rows.
  9. Keeps original_url unchanged.

This makes new card translations prefer the home server while preserving OpenAI as a safety net.

Default Worker config:

ENABLED_SUMMARY_LANGUAGES=fr,ja,de-CH,de,el
SUMMARY_TRANSLATION_LIMIT=5
HOLD_ARTICLES_FOR_TRANSLATIONS=true

Home-server translation config uses the same local AI variables as local review:

LOCAL_AI_URL=https://ai.nutsnews.com
LOCAL_AI_MODEL=qwen2.5:3b
LOCAL_AI_API_KEY=<secret binding>

AI_PROVIDER can still be openai; translations will use the home server first whenever LOCAL_AI_URL and LOCAL_AI_API_KEY are available.

To disable translations temporarily:

ENABLED_SUMMARY_LANGUAGES=none

Public API behavior

Section titled “Public API behavior”

The articles API supports a lang query parameter:

/api/articles?page=0&lang=fr

Fallback behavior for existing rows:

French summary exists → return French title/summary
French summary missing → return English/default title/summary

Newly accepted articles should not reach the public snapshot until the enabled article_summaries rows exist. English fallback remains available for old rows, invalid rows, and emergency recovery, but missing rows should be treated as diagnostics/backlog work rather than normal publication flow.

Web UI behavior

Section titled “Web UI behavior”

The Settings panel includes a Language selector:

English
Français

The selected language is stored in local storage under:

nutsnews.web.language

Changing language refetches the feed with the selected language.

Backfilling existing articles

Section titled “Backfilling existing articles”

Use the backfill script for existing articles that were published before the feature existed:

Terminal window
cd /Users/ramideltoro/WebstormProjects/nutsnews2


SUPABASE_URL="https://YOUR_PROJECT.supabase.co" \
SUPABASE_SERVICE_ROLE_KEY="YOUR_SERVICE_ROLE_KEY" \
OPENAI_API_KEY="YOUR_OPENAI_KEY" \
BACKFILL_LIMIT=25 \
node scripts/backfill_french_summaries.mjs

Run small batches first to control cost.

Adding another language later

Section titled “Adding another language later”
  1. Add the language to SUPPORTED_LANGUAGES in web/lib/languages.ts.
  2. Add the code to the Worker SummaryLanguageCode union.
  3. Update isSummaryLanguageCode.
  4. Add the language code to ENABLED_SUMMARY_LANGUAGES, for example:
ENABLED_SUMMARY_LANGUAGES=fr,es

No new column is required in public.articles.

Japanese support (ja)

Section titled “Japanese support (ja)”

Japanese is supported as an additional summary language using the same public.article_summaries table.

  • UI language code: ja
  • Language label: 日本語
  • Flag badge: 🇯🇵
  • Article API: /api/articles?lang=ja
  • Backfill script: scripts/backfill_japanese_summaries.mjs
  • Worker setting: ENABLED_SUMMARY_LANGUAGES=fr,ja,de-CH,de,el

Existing articles must be backfilled once. New accepted articles are translated by the Workers after the updated Worker code is deployed. Source links remain unchanged and continue to point to the original publisher.

Diagnosing missing translated cards

Section titled “Diagnosing missing translated cards”

If a few articles remain in English while the surrounding home-page articles are translated, do not assume the front end is broken. The frontend intentionally falls back to the English articles row whenever a matching row is missing from public.article_summaries, and it logs articles.localized_summaries_missing with the requested language and missing-row counts.

The first thing to check is whether a Worker run accepted more articles than the available summary translation task budget. Articles beyond that budget should remain translation_pending until a backlog or backfill run creates every enabled language row and publishes them.

Run:

Terminal window
LANGUAGE_CODES=fr,ja,de-CH,de,el AUDIT_LIMIT=80 WINDOW_MINUTES=45 \
node scripts/diagnose_missing_article_translations.mjs

Then search Better Stack for:

service:nutsnews-worker event:worker.translation.skipped_by_limit
service:nutsnews-worker event:worker.translation.summary_completed
service:nutsnews-worker articleUrl:"PASTE_ARTICLE_URL_HERE"

If the root cause is the limit, increase the Worker SUMMARY_TRANSLATION_LIMIT variable to match the largest maxAiReviews value you allow, then run scripts/backfill_article_summaries.mjs for the missing rows.

If the root cause is provider failure, use the per-article Worker logs. Translation failures are logged with articleUrl and languageCode for local AI request failures, local invalid payloads, OpenAI fallback, OpenAI request failures, OpenAI invalid JSON, and Supabase article summary save failures.

Quality checks and fallback policy

Section titled “Quality checks and fallback policy”

NutsNews also maintains a formal translation quality policy. See Multilingual Quality and Fallbacks for the daily coverage report, /admin/translations dashboard, quality rules, English-leak detection, Worker save policy, and public fallback behavior.