arne/news
0
0
Fork 1
Code Issues Pull requests Projects Releases Packages Wiki Activity
No description
233 commits 6 branches 0 tags 4.1 MiB
  • Go 41.7%
  • HTML 40.3%
  • CSS 16.1%
  • JavaScript 1.4%
  • Shell 0.5%
Find a file
Exact
Arne Skaar Fismen bcce6596fe Update stale desk and snapshot tests to current behavior (#14)
2026-06-15 22:02:24 +02:00
design Add orbit design system, conventions, and project guide 2026-05-30 23:53:10 +02:00
docs Add orbit design system, conventions, and project guide 2026-05-30 23:53:10 +02:00
scrapers Fix VG and Aftenposten scrapers for Schibsted redesign (#12) 2026-06-15 21:50:14 +02:00
scripts Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
static feat: redesign markdown preview — back arrow left, outlet + time right 2026-03-31 17:14:39 +02:00
templates fix: use outerHTML swap on body element to prevent tab flash 2026-03-31 21:26:01 +02:00
ai_worker.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
ai_worker_test.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
analyzer.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
auth.go feat: rewrite auth.go with OIDC authorization code flow 2026-03-31 10:15:03 +02:00
auth_test.go refactor: simplify journalist queries and fix stale tests 2026-03-31 11:47:17 +02:00
CLAUDE.md Add orbit design system, conventions, and project guide 2026-05-30 23:53:10 +02:00
cluster.go feat: tighter clusters — direct connections only, no transitive chaining 2026-03-31 15:47:02 +02:00
cluster_test.go fix: raise cluster threshold to 3, limit AI to 8 specific keywords 2026-03-30 12:12:59 +02:00
config.go Switch AI primary to qwen3.6:35b and raise timeout (#10) 2026-06-07 21:14:19 +02:00
config_test.go Switch AI primary to qwen3.6:35b and raise timeout (#10) 2026-06-07 21:14:19 +02:00
content_worker.go feat: use article page titles (og:title) instead of front-page headlines 2026-03-31 14:07:26 +02:00
content_worker_test.go feat: log extraction failures to extraction_errors table 2026-03-30 15:20:03 +02:00
conventions.yaml Add orbit design system, conventions, and project guide 2026-05-30 23:53:10 +02:00
db.go fix: also ban TT and AFP-DPA (stripped NTB- prefix versions) 2026-03-31 20:13:25 +02:00
db_articles.go feat: enforce 24-hour max age for edition candidates 2026-04-02 11:23:13 +02:00
db_auth.go feat: add sessions table, remove WebAuthn DB methods 2026-03-31 10:13:59 +02:00
db_clusters.go fix: eliminate N+1 query in AllClusters — single join for all articles 2026-03-31 15:37:46 +02:00
db_editions.go feat: new edition selection logic + CLI generate command 2026-03-30 19:00:49 +02:00
db_extraction_errors.go feat: new edition selection logic + CLI generate command 2026-03-30 19:00:49 +02:00
db_metadata.go fix: also ban TT and AFP-DPA (stripped NTB- prefix versions) 2026-03-31 20:13:25 +02:00
db_scrapers.go refactor: remove dead code, wire DeleteExpiredSessions 2026-03-30 14:46:39 +02:00
db_test.go Update stale desk and snapshot tests to current behavior (#14) 2026-06-15 22:02:24 +02:00
edition.go feat: add recency decay to evening edition scoring 2026-03-31 18:49:58 +02:00
edition_test.go feat: rewrite edition generation with prominence-based scoring 2026-03-30 20:10:45 +02:00
fallback.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
fallback_test.go Switch AI primary to qwen3.6:35b and raise timeout (#10) 2026-06-07 21:14:19 +02:00
go.mod feat: add html-to-markdown and goldmark dependencies 2026-03-31 13:04:37 +02:00
go.sum feat: add html-to-markdown and goldmark dependencies 2026-03-31 13:04:37 +02:00
main.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
markdown.go feat: add markdown converter with front-matter generation 2026-03-31 13:06:12 +02:00
ollama.go Deploy from git and add fox-primary AI summaries 2026-05-31 10:24:22 +02:00
ollama_test.go feat: AI-generated keywords for clustering, switch to qwen3:14b 2026-03-30 09:25:11 +02:00
openai.go Switch AI primary to qwen3.6:35b and raise timeout (#10) 2026-06-07 21:14:19 +02:00
openai_test.go Switch AI primary to qwen3.6:35b and raise timeout (#10) 2026-06-07 21:14:19 +02:00
README.md rename app from news to nyheter 2026-03-29 22:34:01 +02:00
scheduler.go feat: persist clusters after each scrape cycle 2026-03-31 15:32:33 +02:00
scheduler_test.go feat: record front page snapshots on every scrape cycle 2026-03-30 20:04:06 +02:00
server.go fix: check HX-Target instead of HX-Request for snapshot panel partial 2026-03-31 21:23:45 +02:00
server_test.go Update stale desk and snapshot tests to current behavior (#14) 2026-06-15 22:02:24 +02:00
smoke_test.go refactor: simplify journalist queries and fix stale tests 2026-03-31 11:47:17 +02:00

README.md

nyheter

A Norwegian news syndication app. Scrapes major news sites, tracks story prominence, and produces daily morning and evening editions of the 10 most important stories with AI-generated summaries.

How it works

  1. Scrapes the front pages of NRK, VG, TV2, Dagbladet, ABC Nyheter, and Aftenposten every 10 minutes
  2. Tracks which stories stay near the top and appear across multiple sites
  3. At 6am and 6pm, generates an edition of the top 10 stories
  4. Summaries are generated via Ollama (ai.fismen.no)

Stack

  • Go
  • SQLite
  • HTMX
  • Ollama

Configuration

Variable Default Description
DATABASE_PATH news.db Path to SQLite database
LISTEN_ADDR :8080 HTTP listen address
OLLAMA_URL https://ai.fismen.no Ollama instance URL
OLLAMA_MODEL — Model to use for summaries
SCRAPE_INTERVAL 10m Polling interval

Routes

  • / — Current edition with navigation to past editions
  • /desk — Admin interface (scraper status, article browser, ranking preview)