oliver/rss-news
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases 12 Packages Wiki Activity Actions
rss-news/backend
History
Exact
OliverGiertz f710141828 fix(scheduler): prevent duplicate slot assignment from concurrent pipeline runs 
Two bugs caused multiple articles to land on the same publish slot:

1. main.py: asyncio.create_task() returned immediately, allowing a second
   pipeline trigger (N8N + Telegram /run or two N8N calls) to start a
   second concurrent run. Added asyncio.Lock (_pipeline_lock) so any
   second trigger while the pipeline is running is rejected immediately.

2. scheduler.py: reserve_publish_slot() read the list of occupied slots
   and wrote the new slot in two separate DB connections. Concurrent threads
   could both see the same "free" slot before either committed its write.
   Fixed by wrapping the entire read-find-write cycle in a threading.Lock
   (_slot_lock) and a single DB connection, so the slot check and the
   slot assignment are atomic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 05:03:13 +00:00
..
app fix(scheduler): prevent duplicate slot assignment from concurrent pipeline runs 2026-05-08 05:03:13 +00:00
data chore: finalize current state and prepare next wordpress-focused roadmap 2026-02-18 11:11:49 +01:00
static feat(admin): add feed/source management, rewrite editor, reopen flow, and WP block output 2026-02-21 14:03:49 +01:00
templates feat(admin): bulk-editable article list with WP ID inline editing 2026-04-10 09:00:25 +00:00
tests test(ingestion): update test for removed Ampel risk-level check 2026-03-21 09:41:34 +00:00
.env.example feat(automation): autonomous pipeline with Telegram bot and N8N integration 2026-03-21 09:40:15 +00:00
__init__.py feat: rebuild rss-news backend, admin ui, and legal extraction pipeline 2026-02-18 09:52:36 +01:00
README.md feat: rebuild rss-news backend, admin ui, and legal extraction pipeline 2026-02-18 09:52:36 +01:00
requirements-test.txt feat: rebuild rss-news backend, admin ui, and legal extraction pipeline 2026-02-18 09:52:36 +01:00
requirements.txt feat: rebuild rss-news backend, admin ui, and legal extraction pipeline 2026-02-18 09:52:36 +01:00

README.md

Backend Skeleton (FastAPI)

Dieses Verzeichnis enthaelt das technische Grundgeruest fuer den Rebuild von rss-news.

Start (lokal)

python -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
uvicorn backend.app.main:app --reload --port 8501

Admin UI

  • Login: http://127.0.0.1:8501/admin/login
  • Dashboard: http://127.0.0.1:8501/admin/dashboard

Environment

  • Datei: backend/.env
  • Vorlage: backend/.env.example

Endpoints

  • GET /health - Healthcheck
  • POST /auth/login - Login mit Admin-User
  • POST /auth/logout - Logout
  • GET /auth/me - Aktiver User
  • GET /api/protected - Geschuetzter Test-Endpoint
  • GET /api/pipeline/status - Basisstatus inkl. Datensatzzaehler
  • GET /api/sources - Quellenliste
  • POST /api/sources - Quelle anlegen
  • GET /api/sources/{source_id}/policy-check - Policy-Pruefung fuer Quelle
  • GET /api/feeds - Feedliste
  • POST /api/feeds - Feed anlegen
  • GET /api/feeds/{feed_id}/policy-check - Policy-Pruefung fuer Feed
  • GET /api/runs - Import-/Job-Runs anzeigen
  • GET /api/runs/{run_id} - Detailansicht eines Runs
  • POST /api/runs - Run starten
  • POST /api/runs/{run_id}/finish - Run abschliessen
  • GET /api/articles - Artikel anzeigen
  • GET /api/articles/{article_id} - Artikeldetail
  • POST /api/articles/upsert - Artikel idempotent anlegen/aktualisieren
  • POST /api/articles/{article_id}/transition - Statuswechsel nach Workflow-Regeln
  • POST /api/articles/{article_id}/review - Review-Entscheidung (approve/reject)
  • POST /api/ingestion/run - Feed-Ingestion starten (optional pro Feed)

Datenbank

  • SQLite-Datei unter backend/data/rss_news.db
  • Tabellen werden beim App-Start initialisiert.
  • Tabellen: sources, feeds, runs, articles
  • Dedupe-Strategie Artikel: source_url -> (feed_id, source_article_id) -> source_hash

Policy-Enforcement

  • Ingestion blockiert Feeds automatisch, wenn die zugeordnete Quelle nicht policy-konform ist.
  • Mindestanforderungen: risk_level=green, terms_url, license_name, last_reviewed_at, is_enabled=1.
  • Pro importiertem Artikel wird ein attribution-Block in meta_json gespeichert.

Review-Workflow

  • Statuskette: new -> review -> approved -> published
  • Ablehnung im Review setzt auf rewrite
  • Ungueltige Statuswechsel werden per API blockiert

Verifikation

python -m unittest backend.tests.test_db_repositories
python -m unittest backend.tests.test_ingestion
python -m unittest backend.tests.test_api_auth

CI / Online-Auswertung

  • GitHub Actions Workflow: .github/workflows/test.yml
  • Fuehrt Tests inkl. Coverage auf Push/PR gegen main aus.

Hetzner Smoketest

BASE_URL="https://news.vanityontour.de" \
APP_ADMIN_USERNAME="admin" \
APP_ADMIN_PASSWORD="..." \
bash scripts/smoke_backend.sh

Hinweis

Passwort-Hashing und CSRF/Rate-Limit sind als naechste Ausbaustufe vorgesehen.