refactor(library): two bounded background pools

Every @Async names a pool: with the scheduler there are two TaskExecutor beans, and a bare @Async ran on an unbounded executor. Page views queue only stale games; the long runs after a sync get their own pool.
This commit is contained in:
lobo committed 2026-09-30 12:08:57 +02:00
1 parent 9a09d535fc
commit 2c97e2ae96
15 files changed
+375 -56

No files matched your search

+34 -2
View File
@@ -98,7 +98,7 @@ only guard.
Every service method a controller calls carries one role marker, on the method or on its class (the method wins):
`@RequiresRole(Role.X)`, `@Public` (no login, e.g. landing stats; the public profile checks its ticket) or
`@Internal` (only Auth's service token). A Spring Security method interceptor checks it while a request is served;
scheduled jobs and `@Async` work run without a user and are not checked. A denied call answers `401` without a
scheduled jobs and background work run without a user and are not checked. A denied call answers `401` without a
login, else `403`. The URL rules in `SecurityConfig` stay as the outer guard. ArchUnit fails the build for an
unmarked method, two markers, a controller calling an `@Async` method (the check would never run), and a service
method calling another service's method that needs more.
@@ -221,7 +221,7 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and
dates finished games by the unlock of their ending achievement
(`FinishedEstimateService`, `user_game.finished_at_estimate`).
3. `LibrarySyncWorker` runs in the background. `user_steam_sync` lets only one run per user start, and none for
3. `LibrarySyncWorker` runs on the background pool. `user_steam_sync` lets only one run per user start, and none for
a user in `deleted_user`.
4. The worker loads the wishlist and the owned games from Steam. It updates `steam_game` and `user_game`, adds
the playtime growth of the day to `user_game_playtime_day`, and soft-deletes Steam games the user no longer
@@ -239,6 +239,38 @@ The frontend polls `GET /api/library/sync-status` until the run ends.
At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/library/broken-images` (table
`broken_image`, one row per game and kind). Reports only count for Steam games the user owns.
### Background work
Background work runs on two pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it):
- The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view
starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with
a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user
starts it again.
- The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun
refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page
views' threads. A task is small (a user or a few appids), and none is dropped.
Imports have their own two threads, and `game_import` is their queue. Scheduled jobs (nightly sync, image repair,
import watchdog) run on the scheduler.
```mermaid
flowchart LR
View["page view, add, merge"] -->|"stale games only"| Pool["default pool<br/>4 threads, queue 100"]
Sync["sync request"] --> Pool
Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
View -->|"speedrun"| FollowUp
Pool --> Apis["Steam, IGDB, speedrun.com"]
FollowUp --> Apis
Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"]
```
Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks
live in memory: a second instance would ask Steam and IGDB twice, and the database would stay consistent (sync
claim, upserts). A restart drops what waits in the pools; it is all refresh work and is asked again. Imports survive
a restart.
### Corrections by an auditor
An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag and genres for