feat(import): import many games with background matching, review and undo

This commit is contained in:
lobo committed 2026-09-29 10:20:54 +02:00
1 parent 3bd1eda0ec
commit 2a5fcc1c00
20 files changed
+1639 -24

No files matched your search

+46 -1
View File
@@ -103,6 +103,11 @@ method calling another service's method that needs more.
| `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT |
| `PUT` | `/api/library/games/{appid}/finished-hint` (`{dismissed: ENDING\|PLAYTIME}`, "Not yet"; `204`) | JWT |
| `PUT` | `/api/library/games/{appid}/hidden` (`{hidden}`; `409 MANUAL_GAME` for a game added by hand) | JWT |
| `POST` | `/api/library/imports` (`{source: TEXT\|CSV\|SEARCH, rows: [{title, platform, playtimeMinutes, status, igdbId}]}`, 1–1000 rows; `202 {id}` in `QUEUED`, `409 IMPORT_RUNNING`, `503 IGDB_OFF`) | JWT |
| `GET` | `/api/library/imports` (`{imports}`, the last 20), `/api/library/imports/{id}` (status, progress counts, `ahead` while it waits in a queue, `attempts`; the rows with their candidates while `READY`) | JWT |
| `POST` | `/api/library/imports/{id}/save` (`{rows: [{lineNo, igdbId, platform, status, playtimeMinutes}]}`; `202`, then poll until `SAVED`; `409 IMPORT_NOT_READY`) | JWT |
| `POST` | `/api/library/imports/{id}/retry` (a `FAILED` import; `202`, `409 IMPORT_NOT_FAILED` / `IMPORT_RUNNING`) | JWT |
| `GET`, `DELETE` | `/api/library/imports/{id}/undo-check` (`{games, changed}`), `/api/library/imports/{id}/games` (undo; `204`, `409 IMPORT_NOT_SAVED`) | JWT |
| `POST` | `/api/library/sync` | JWT |
| `POST` | `/api/library/broken-images` (`{appid, kind}`, kind `COVER` or `ACHIEVEMENT_ICONS`; `204`) | JWT |
| `GET` | `/api/library/picker?maxMinutes=&mood=&count=&exclude=` | JWT |
@@ -163,7 +168,7 @@ flowchart LR
| Module | Content |
|---|---|
| `library` | Sync, backlog, manual games, playtime history, wishlist, achievements, public profile data |
| `library` | Sync, backlog, manual games, imports, playtime history, wishlist, achievements, public profile data |
| `stats` | Public numbers for the landing page |
| `version` | `GET /api/version`: commit and build time of the running image |
| `account` | Deleted accounts, driven by Auth: tombstones in `deleted_user`, purge and orphan prune of the per-user rows |
@@ -265,6 +270,46 @@ id for negative keys. Search and add check both keys of a game. A game added und
a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two
(see "Merging duplicates").
### Imports
Many games at once (V22, `GameImportService`): a list, a CSV or several picks of the game search become one
`game_import` with a `game_import_row` per line. `game_import` is also the queue of `GameImportWorker`, which has
two queues, each on one thread for all users:
- **matching**: IGDB searches with typo help. One thread, so an import never takes the whole IGDB budget from the
interactive search; other users' searches may wait for the limiter meanwhile. Up to 5 candidates per line are
stored whole (JSONB), so the save needs no IGDB call, and `exact` marks a best one with the line's name.
- **saving**: adds the chosen lines the way the manual add does (same keys and duplicate check) with
`user_game.import_id` in the same transaction, then loads their time to beat and moods. A save never waits
behind another user's big matching.
Each line is stored when done, so a stopped try goes on where it stopped. A worker sets `progress_at` when it takes
an import and after every line. A watchdog (every minute) stops a try without progress for 5 minutes
(`app.library.import-stall-minutes`): the import goes to the end of its queue and the queue gets a new thread. A
restart counts as a stopped try too. After 3 tries the import is `FAILED`; the user can try it again. One import
per user can be in the matching queue (a unique partial index).
```mermaid
stateDiagram-v2
[*] --> QUEUED: POST /imports
QUEUED --> MATCHING: matching worker takes it
MATCHING --> READY: every line matched
READY --> SAVING: POST /save (the review)
SAVING --> SAVED: games added, time to beat and moods loaded
SAVED --> UNDONE: DELETE /games (undo)
MATCHING --> QUEUED: no progress for 5 min or restart
SAVING --> SAVING: no progress for 5 min or restart (end of the queue)
MATCHING --> FAILED: 3rd stopped try
SAVING --> FAILED: 3rd stopped try
FAILED --> QUEUED: POST /retry (stopped while matching)
FAILED --> SAVING: POST /retry (stopped while saving)
FAILED --> UNDONE: DELETE /games (the games added before it failed)
```
The undo deletes the games of the import that are still `MANUAL` (a game the Steam sync took over stays). A removed
row the import brings back gets no `import_id`: it holds rating, notes and history from before, so the undo keeps it;
`undo-check` counts those with a rating, notes, priority or a status change after the save.
## Run locally
### Local development without keys (profile `local`)