Public Access
Picker returns categories in fixed order: NEXT_UP, SIMILAR, LONGEST, RANDOM, each with up to 3 games. Empty rows are left out. Param `count` is removed; new optional `category` loads one row (reroll). Budget and exclude apply to all rows; mood only to RANDOM. SIMILAR takes the 3 most played games, one similar game each. Remove PickMix.
652 lines
52 KiB
Markdown
652 lines
52 KiB
Markdown
# To Be Played – Core Service
|
||
|
||
Business service of **To Be Played**, a Steam backlog tracker. It syncs the user's Steam library, stores the backlog
|
||
(status, rating, notes, priority), tracks playtime per day, syncs the wishlist and adds time-to-beat data from IGDB.
|
||
|
||
To Be Played is split into three repositories:
|
||
|
||
| Repository | Role |
|
||
|---|---|
|
||
| ToBePlayed-Auth | Users, login, settings, signs JWTs |
|
||
| **ToBePlayed-Core** (this one) | Steam library, backlog, wishlist, stats; verifies JWTs |
|
||
| ToBePlayed-Frontend | React single-page app |
|
||
|
||
## Features
|
||
|
||
- **Steam sync**: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly
|
||
sync. Resilience4j rate-limits every Steam call and tries the owned games up to three times (never again on a 4xx;
|
||
the wishlist and the calls a page waits for once); Caffeine caches game counts and player summaries. A sync commits its claim (`RUNNING`, with
|
||
`sync_started_at`), its writes and its finish apart, and calls Steam and IGDB outside any transaction: a second
|
||
sync sees `RUNNING` and skips, a failed one stores `FAILED`. A run older than `app.library.running-stale-minutes`
|
||
can be taken over; its own finish then changes nothing.
|
||
- **Backlog**: status (`BACKLOG`, `PLAYING`, `FINISHED`, `COMPLETED` = 100 %, `DROPPED`), rating 1–5, notes, priority,
|
||
filters and sorting. "Pile of Shame" summary with a random suggestion of what to play.
|
||
- **Picker**: four rows of up to 3 backlog games each, for a time budget (IGDB normal time to beat), in this order:
|
||
`NEXT_UP` (the first games of the user's own backlog order), `SIMILAR` ("because you played": for each of the 3 most
|
||
played games the first backlog game sharing its moods, no game twice), `LONGEST` (oldest `firstSeenAt`) and `RANDOM`
|
||
(the mood row: random, HIGH priority first, then MEDIUM, then the rest). The mood filters only the `RANDOM` row; budget
|
||
and exclude filters apply to every row. A row with no game is left out. A game can show in more than one row.
|
||
Every game carries a `reason` (`source`, plus the anchor game for `SIMILAR` or `firstSeenAt` for `LONGEST`).
|
||
Games already shown can be excluded for a reroll. Moods `STORY`, `COOP`,
|
||
`CHILL`, `ACTION`, `SCARY` come from IGDB genres, themes and game modes; `SPEEDRUN` from speedrun.com.
|
||
- **Follow users (#27)**: a logged-in user follows people with a public profile by display name (`/api/follows`, at most
|
||
500, never an id in an answer) and gets one inbox note per followed user and batch (`/api/inbox`): a sync with its
|
||
achievement run, or the hand changes of a quiet 10 minutes. A note counts new games, achievement unlocks, status
|
||
changes and Steam wishlist adds, and leaves out what the followed user's profile hides (`public_user.sections`).
|
||
Follows end for good, with a note, when the followed user goes private, and without a note when the user is blocked or
|
||
deleted. The nightly job at 03:15 deletes notes older than 90 days (`app.inbox.keep-days`); the list shows the newest 50.
|
||
- **Backlog order**: every listed `BACKLOG` game (not removed, not hidden) has a row in `backlog_order (user_id,
|
||
appid, position)`, kept by a database trigger on `user_game` like the status history: a game entering the backlog
|
||
goes to the bottom, a game leaving it loses its row. V34 seeded the first order (priority HIGH, MEDIUM, rest, then
|
||
date added). `PUT /api/library/backlog` replaces the whole order under a row lock; ids not in the backlog are
|
||
ignored and games not sent go after the sent ones. A merge keeps the better position of the two games. Coming
|
||
games are in the list; `released=true` leaves them out.
|
||
- **Coming games**: a game with a release date after today (Berlin date) or Steam "coming soon", and no playtime or
|
||
achievement of any user, is not picked and not on the pile (live count, new snapshots, pile stats, time to beat
|
||
sum). It keeps its status and shows `coming` / `comingOn`. Date: IGDB `first_release_date`, else Steam's store
|
||
(`GetItems` with `include_release`), else unknown (= released). View `coming_game`, table `game_release`; a
|
||
year-only IGDB date stays a placeholder until the real date comes (coming rows refresh daily).
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
A["anchor: random of top 5 played"] --> S[similar]
|
||
S --> M["PickMix: rotate 4 sources, dedupe, random fallback"]
|
||
L[longest waiting] --> M
|
||
R[random] --> M
|
||
N["next up: random of top 5 in order"] --> M
|
||
M --> O["games + reason"]
|
||
```
|
||
- **Speedrun games**: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and
|
||
a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns
|
||
the speedrun.com link as `speedrunUrl`. speedrun.com has no Steam or IGDB link and no key, so one queue works for
|
||
all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days.
|
||
- **Achievements**: after every library sync a background job fetches the achievement schema of each owned game
|
||
and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half
|
||
of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never
|
||
overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5
|
||
games closest to 100 %.
|
||
- **Playtime history**: one row per game and day, for the activity view and the "most played in the last two weeks"
|
||
dashboard banner.
|
||
- **Finished from achievements or playtime**: `EndingAchievement` finds the one achievement of a game that means
|
||
"story end" (English text rule, Steam's global unlock rate between 1 and 80 %, the easiest variant wins, more than
|
||
3 candidates = a game without one end). When the user unlocks it, never chose the status and never answered
|
||
"Not yet", the achievement sync sets FINISHED, but only for a certain ending: its text says so and no other
|
||
achievement of the game could be the end. A status the user chose is never changed; such games get a hint on the game page instead. A second,
|
||
weaker hint compares the playtime with IGDB's normal time to beat (at least 90 %), only for games IGDB does not
|
||
mark as endless (sandbox, survival, 4X, simulator, MMO). "Not yet" hides a hint per game; a dismissed playtime hint
|
||
still lets the ending hint through.
|
||
- **Hide games**: a user can hide a game, Steam or added by hand (e.g. a playtest or a tool). It leaves every list and number: library,
|
||
picker, Pile of Shame, activity, achievement summary, public profile and landing page stats. The sync still
|
||
updates its playtime, but no achievements, time to beat, moods or speedrun data are fetched for it. The library
|
||
lists hidden games with `hidden=true`.
|
||
- **Stats**: `UserStatsService` builds the numbers of the stats page per group, so every page asks only for the
|
||
groups it shows: totals and statuses; the pile (time to beat, weeks "at your pace" of the last 90 days, years of
|
||
40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres
|
||
and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the
|
||
status history); games and playtime per platform (`PLATFORMS`: Steam games as "Steam", manual games by platform, trimmed and ignoring case, the most used spelling is the label); the Pile of Shame trend (90 days of `backlog_snapshot`, plus "this month: added / done", where the
|
||
first sync of a Steam account is no "added", see `user_steam_sync.history_since`). Every number counts only listed games (still owned, not hidden: the view
|
||
`listed_user_game`, V24). A linked set (see "Editions of one game") counts once: its most advanced edition (status, then
|
||
playtime, then the older `first_seen_at`; `counted_user_game`, V28), so the set is on the pile only when every owned edition is `BACKLOG`. Playtime
|
||
sums, most played, genres, moods and platforms still use every listed game.
|
||
- **Drill-down**: every countable number opens the games behind it. A `StatKey` (library), `CommunityKey` (landing page)
|
||
or a Pile Points track/bonus row names a number; its SQL "game set" is the same text the number counts, so the
|
||
list and the number cannot drift. 40 games per page (at most page 250), a fixed order per key. A profile visitor
|
||
gets a list only when the ticket shares that section, or the `STATS` section and the key's group (`403 NOT_SHARED`;
|
||
`OVERVIEW` keys are for the owner only). The community list has one row per game with `libraries` and `minutes`,
|
||
never a user, and is cached 10 minutes like the numbers. A bad argument is `400 BAD_ARG`.
|
||
- **Status history**: a database trigger logs every status change of a library game in `user_game_status_change`
|
||
(old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older
|
||
changes are not known.
|
||
- **Pile Points** (`modules/rewards`): users earn points for playing games off the pile, in an append-only ledger
|
||
(`reward_ledger`, V26; the balance is the sum). No code path pays: a settle (on `GET /api/rewards/wallet` and
|
||
nightly at 02:30, `app.rewards.settle-cron`) derives rows from the status log, playtime days, achievements and pile
|
||
snapshots, once per user, kind and game (`ref`). Rules and numbers: `RewardRules`; a Steam finish needs playtime and one
|
||
proof of fresh play (ending achievement or playtime around the finish, or half of the IGDB time played since the finish; old history pays nothing). The month bonus needs a finished game in that month. Games added by hand pay 15 for
|
||
a finish, at most 3 per ISO week. The settle takes a per-user advisory lock. V26 also pays a one-time starter grant
|
||
for finishes from before the status log. Shop: a whitelist of 57 looks (`ShopItem`; 45 buyable and 12 unlock-only) in eight slots, including animated completed effects, page backgrounds (CSS, SVG and two canvas engines in the Frontend),
|
||
banner backgrounds (V27 widens `reward_equipped.slot`) and three buyable skins (full themes, slot `SKIN`). The unlock-only
|
||
items (price 0, never sold): the settle books an `UNLOCK` row (ref = item code, 0 points) once a
|
||
track reaches the item's target, retroactively and never lost. The five tracks (`Unlocks`): games finished
|
||
(`FINISHED` + `FINISHED_MANUAL` rows + starter grant / 20), games completed, weeks in a row with a finish, months in
|
||
a row with a `MONTH` bonus, months in a row with more Steam playtime than the month before (hidden games count too).
|
||
A `BUY` or `UNLOCK` row proves ownership,
|
||
`reward_equipped` holds what is worn. A platform admin owns every item (the role of the login token, or the `role`
|
||
claim of the profile ticket for visitors).
|
||
- **Editions and links**: editions, remasters and ports of one game stay separate library rows, but an AUDITOR can
|
||
link their keys as one game for every user (see "Editions of one game"). IGDB suggests pairs, users can ask for
|
||
one, and one queue holds both. Every link, unlink and dismissal goes to `admin_audit`
|
||
(`GAME_LINK`, `GAME_UNLINK`, `GAME_LINK_DISMISS`); auditors read that log through `GET /api/library/audit-log`, with
|
||
the actor's display name from Auth's account state push.
|
||
- **Wishlist**: synced from Steam with store price and discount.
|
||
- **Broken image repair**: the frontend reports a Steam cover or achievement icon that does not load. At 01:30
|
||
(Europe/Berlin) a job checks up to 200 reports. A cover that loads again is left alone; else the store's hashed
|
||
header picture (`IStoreBrowseService/GetItems` with assets) goes to `steam_game.image_url`. When a few achievement
|
||
icons do not load, the schema comes fresh from Steam; an empty answer keeps the stored one. A report that cannot be repaired is tried again on the next report, at most 3 times,
|
||
then after 30 days.
|
||
- **Time to beat**: story, extras and completionist hours from the IGDB API. Optional: without credentials the
|
||
feature is off and the rest works.
|
||
- **Public profile data**: totals, status counts, games in progress and finished games with rating, perfect games
|
||
and average completion (the numbers count a linked set once; the lists keep every edition). Readable only
|
||
with a profile ticket issued by Auth. Never returns notes, priority or the Steam ID. `stats` holds only the stat
|
||
groups the user shares (the ticket's `profileStats` claim, derived in Auth from the profile tiles); none by
|
||
default. A section the user
|
||
hides (the ticket's `sections` claim: `SUMMARY`, `STATUS`, `PLAYING`, `FINISHED`, `STATS`) is null and runs no
|
||
query; a ticket without the claim shows all.
|
||
|
||
## Authentication
|
||
|
||
Core never issues tokens. It verifies Auth's EdDSA (Ed25519) JWTs with public keys only. It loads them from Auth's
|
||
JWKS by the token's `kid` and loads again when a new `kid` shows up (at most every 10 seconds). Each load replaces
|
||
the key set: a key that left the JWKS stops working. `JWT_PUBLIC_KEY` is used only until the first JWKS load works.
|
||
The token comes from Auth's `tbp_access` cookie or a `Bearer` header, whichever verifies first. Every token names its kind in `aud`
|
||
(`tbp-login`, `tbp-public-profile`, `tbp-internal`, with `iss=tbp-auth`); a login's `sub` is the user id. Core reads
|
||
`steamId` and the active `role` (missing or unknown = USER) from the claims and grants that role and every lower one
|
||
(`USER < AUDITOR < PLATFORM_ADMIN`). A token without `aud` is refused. The claims are pinned in
|
||
`src/test/resources/contract/auth-core-contract.json`, the same file as in Auth; the tests build tokens only from it.
|
||
`POST`, `PUT` and `DELETE` need an `X-TBP-Client` or `Authorization` header (CSRF guard), and CORS allows only the
|
||
frontend origin.
|
||
A token for another audience (for example a profile ticket) is rejected on normal endpoints; the profile ticket is
|
||
accepted only by `GET /api/public/profiles/library` in the `X-Profile-Ticket` header.
|
||
Every error answer of a controller has the body `{code, message}` (a 401 or 403 of the security filters has none); a refused request is `400 VALIDATION_FAILED` with
|
||
`fields: [{field, message}]`, a duplicate key `409 CONFLICT`, anything unexpected `500 INTERNAL_ERROR` without details
|
||
(`shared/api/GlobalExceptionController`).
|
||
`/internal/**` has its own filter chain: it accepts only a service token of Auth (`aud=tbp-internal`, with `exp`, sent
|
||
as `Bearer`), never a login, also not in the dev profile. Traefik exposes the path, so the signed token is its
|
||
only guard.
|
||
|
||
Every service method a controller calls carries one role marker, on the method or on its class (the method wins):
|
||
`@RequiresRole(Role.X)`, `@Public` (no login, e.g. landing stats; the public profile checks its ticket) or
|
||
`@Internal` (only Auth's service token). A Spring Security method interceptor checks it while a request is served;
|
||
scheduled jobs and background work run without a user and are not checked. That stays on purpose: the sync
|
||
worker calls `SteamApiService` and the import worker `ManualGameService`, both `@RequiresRole(USER)`, without a
|
||
request. Auth denies instead, because it has no such caller. A denied call answers `401` without a
|
||
login, else `403`. The URL rules in `SecurityConfig` stay as the outer guard. ArchUnit fails the build for an
|
||
unmarked method, two markers, a controller calling an `@Async` method (the check would never run), and a service
|
||
method calling another service's method that needs more.
|
||
|
||
**Mirrored in Auth.** These files exist in both services with small differences; a change in one needs the same
|
||
change in the other: `ClientHeaderFilter`, `RoleAuthorizationManager`, `MethodSecurityConfig`, `Role`, `RequiresRole`,
|
||
`Public` (only Core has `Internal`), `UserPrincipal`, the login token check in `JwtService.parseLoginToken`, the
|
||
ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/auth-core-contract.json`
|
||
(`ToBePlayed-Deployment/scripts/check-contract.sh` compares the two).
|
||
|
||
## API
|
||
|
||
| Method | Path | Auth |
|
||
|---|---|---|
|
||
| `GET` | `/api/rewards/wallet` (balance, catalog, owned, equipped, last 50 ledger rows with the game name, unlock `progress` per track) | JWT |
|
||
| `POST` | `/api/rewards/purchases` (`{item}`; `200` wallet, `409 ALREADY_OWNED` / `NOT_ENOUGH_POINTS` / `NOT_FOR_SALE`) | JWT |
|
||
| `GET` | `/api/rewards/progress/{track}/games?page=`, `/api/rewards/history/{kind}/{ref}/games?page=` (the games behind a track or a `MONTH` / `STREAK` bonus; `400 NOT_A_GAME_LIST`) | JWT |
|
||
| `PUT` | `/api/rewards/equipped/{slot}` (`{item}`, null = take off; `200` wallet, `409 NOT_OWNED`, `400 WRONG_SLOT`) | JWT |
|
||
| `GET` | `/api/library/games` (`hideDuplicates=true` leaves out the lesser editions of a linked set), `/api/library/games/{appid}` | JWT |
|
||
| `PATCH` | `/api/library/games/{appid}/backlog` (only the sent fields; `null` clears one) | JWT |
|
||
| `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`, `time_to_beat` = `{normallyMinutes, completelyMinutes}`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR |
|
||
| `GET` | `/api/library/catalog?q=&page=&size=` (every game some user still has, by name part, `PageDto` of `{appid, name, imageUrl, platforms}`; `q` at most 100 characters; size 20, 40 or 80) | JWT, AUDITOR |
|
||
| `GET` | `/api/library/catalog/{appid}` (what the correction dialog reads: name, image, speedrun, moods, endless, genres, timeToBeat, corrections; `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
|
||
| `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
|
||
| `POST` | `/api/library/merges/{mergeId}/undo` (`mergeId` = the `id` of a `GAME_MERGE` audit row, within 30 days; `204`, `404 MERGE_NOT_FOUND`, `409 UNDO_NOT_AVAILABLE` / `NEWER_MERGE`, `410 UNDO_EXPIRED`) | JWT, AUDITOR |
|
||
| `POST`, `DELETE` | `/api/library/games/{appid}/link` (POST `{otherAppid}`, `204`, `400 SAME_GAME`, `404 GAME_NOT_FOUND`, `409 ALREADY_LINKED`; DELETE takes the game out of its set, `204`, `404 NOT_LINKED`) | JWT, AUDITOR |
|
||
| `GET` | `/api/library/audit-log?action=&page=&size=` (the newest `admin_audit` rows, `PageDto` of `{id, actorUserId, actorName, action, appid, gameName, targetAppid, targetGameName, detail, createdAt, undoable}`; `action` is `GAME_MERGE`, `GAME_MERGE_UNDO`, `GAME_LINK`, `GAME_UNLINK`, `GAME_LINK_DISMISS`, `GAME_CORRECT` or `GAME_CORRECTION_RESET`; size 20, 40 or 80) | JWT, AUDITOR |
|
||
| `GET` | `/api/library/link-queue?page=&size=` (`PageDto` of `{a, b, igdb, requests, createdAt}`: both games, most requested first; size 20, 40 or 80) | JWT, AUDITOR |
|
||
| `DELETE` | `/api/library/link-queue/{appid}/{otherAppid}` (dismiss; `204`, `404 PAIR_NOT_OPEN`) | JWT, AUDITOR |
|
||
| `POST` | `/api/library/games/{appid}/link-requests` (`{otherAppid}`, both in the caller's library; `204`, `400 SAME_GAME`, `404 GAME_NOT_FOUND`, `409 ALREADY_LINKED` / `PAIR_DISMISSED`, `429 TOO_MANY_REQUESTS`) | JWT |
|
||
| `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT |
|
||
| `PUT` | `/api/library/games/{appid}/finished-hint` (`{dismissed: ENDING\|PLAYTIME}`, "Not yet"; `204`) | JWT |
|
||
| `PUT` | `/api/library/games/{appid}/hidden` (`{hidden}`; Steam and manual games) | JWT |
|
||
| `POST` | `/api/library/imports` (`{source: TEXT\|CSV\|SEARCH\|GALAXY\|GROUVEE\|HLTB\|PLAYNITE\|BOOKMARKLET, rows: [{title, platform, playtimeMinutes, status, igdbId}]}`, 1–1000 rows; `202 {id}` in `QUEUED`, `409 IMPORT_RUNNING`, `503 IGDB_OFF`) | JWT |
|
||
| `GET` | `/api/library/imports` (`{imports}`, the last 20), `/api/library/imports/{id}` (status, progress counts, `ahead` while it waits in a queue, `attempts`; the rows with their candidates while `READY`) | JWT |
|
||
| `POST` | `/api/library/imports/{id}/save` (`{rows: [{lineNo, igdbId, platform, status, playtimeMinutes}]}`; `202`, then poll until `SAVED`; `409 IMPORT_NOT_READY`) | JWT |
|
||
| `POST` | `/api/library/imports/{id}/retry` (a `FAILED` import; `202`, `409 IMPORT_NOT_FAILED` / `IMPORT_RUNNING`) | JWT |
|
||
| `GET`, `DELETE` | `/api/library/imports/{id}/undo-check` (`{games, changed}`), `/api/library/imports/{id}/games` (undo; `204`, `409 IMPORT_NOT_SAVED`) | JWT |
|
||
| `POST` | `/api/library/sync` | JWT |
|
||
| `POST` | `/api/library/broken-images` (`{appid, kind}`, kind `COVER` or `ACHIEVEMENT_ICONS`; `204`) | JWT |
|
||
| `GET` | `/api/library/backlog?limit=&released=` (`{games}` in the user's order; `limit` 1-20, none = all) | JWT |
|
||
| `PUT` | `/api/library/backlog` (`{appids}` = the whole order; `204`) | JWT |
|
||
| `GET` | `/api/library/picker?maxMinutes=&mood=&exclude=&category=` (`{categories:[{source, games}]}`; each game has a `reason` object; `category` returns that row only) | JWT |
|
||
| `GET` | `/api/library/stats?groups=` (`OVERVIEW`, `PILE`, `TOP_GAMES`, `TASTE`, `HABITS`, `FINISHED`, `TREND`, `PLATFORMS`; none = all) | JWT |
|
||
| `GET` | `/api/library/stats/games?key=&arg=&page=` (the games behind a stats number; `400 BAD_ARG`) | JWT |
|
||
| `GET` | `/api/library/achievements/summary` | JWT |
|
||
| `GET` | `/api/library/sync-status`, `/summary`, `/activity`, `/wishlist`, `/steam-profile`, `/games-count` | JWT |
|
||
| `GET` | `/api/public/profiles/library` | profile ticket |
|
||
| `GET` | `/api/public/profiles/library/stats/games?key=&arg=&page=` (`403 NOT_SHARED`, `400 BAD_ARG`) | profile ticket |
|
||
| `GET` | `/api/public/profiles/rewards` (`{equipped}`, the worn looks) | profile ticket |
|
||
| `GET` | `/api/stats/public` (+ games per platform, top genres, finished per year) | public |
|
||
| `GET` | `/api/stats/owners?appid=&key=&arg=&page=` (who owns a game in the set behind a community number, `key` default `OWNED`: `listed` display names only, `listedTotal`, `privateCount`, `includesYou`, not cached, `400 BAD_ARG`) | login |
|
||
| `GET` | `/api/stats/public/games?key=&arg=&page=` (the games behind a landing number, per game `libraries` and `minutes`; `400 BAD_ARG`) | public |
|
||
| `GET` | `/api/follows` (`{following: [{id, displayName}], followers: [{id, displayName}]}`, `id` is the follow row; a follower with a private profile has `displayName` null) | login |
|
||
| `PUT`, `DELETE` | `/api/follows/by-name/{displayName}` (follow, unfollow; `204`; PUT: `404 PROFILE_NOT_FOUND` for an unknown, private, blocked or deleted name, `400 FOLLOW_SELF`, `400 FOLLOW_LIMIT`) | login |
|
||
| `DELETE` | `/api/follows/{id}` (unfollow, or remove a follower; only rows of the caller change; `204`) | login |
|
||
| `GET` | `/api/inbox` (`{results: [{id, kind, displayName, newGames, achievements, statusChanges, wishlistAdds, createdAt, read}], unread}`, newest 50, `kind` `DIGEST` or `VISIBILITY`) | login |
|
||
| `PUT` | `/api/inbox/read` (marks all read; `204`) | login |
|
||
| `POST`, `PUT` | `/internal/accounts/deleted` (`{userIds}`; POST adds, PUT replaces the whole set; `204`) | service token |
|
||
| `POST`, `PUT` | `/internal/accounts/states` (`{accounts: [{userId, steamId, displayName, blocked, roleChangedAt, listed, publicProfile, profileSections}]}`, `displayName`, `roleChangedAt`, `listed` and `publicProfile` optional (`listed`: Auth lists the user by name, kept in `listed_user`, `null` keeps it; `publicProfile`: the profile is public, kept as a row in `public_user` with `profileSections`, `null` keeps row and sections, `false` ends the follows of the user with one `VISIBILITY` note each; `profileSections` `null` = all shown), `displayName` at most 100 characters, kept in `account_name` for the audit log, `roleChangedAt` kept in `account_role_change`; POST updates, PUT replaces the whole set; `204`) | service token |
|
||
| `POST` | `/internal/accounts/purge` (`{userIds}`), `/internal/accounts/prune` (`{knownUserIds}`, not empty) | service token |
|
||
| `GET` | `/api/version` (`{commit, builtAt}` of the running image, from the CI build; `dev` locally) | public |
|
||
|
||
## Tech stack
|
||
|
||
- Java 25, Spring Boot 4.1 (Web MVC, Security, Data JPA, Validation, Cache with Caffeine), virtual threads
|
||
- PostgreSQL 17, Flyway migrations
|
||
- Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com
|
||
- JJWT with Ed25519 public key
|
||
- Spring Boot Actuator + Micrometer (Prometheus)
|
||
- Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit
|
||
- CI: Forgejo Actions, multi-stage Docker image
|
||
|
||
## Architecture
|
||
|
||
A modular monolith. Each module under `core_service.modules` is a vertical slice that could move to its own
|
||
service later.
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
Browser["Frontend in the browser"]
|
||
subgraph CoreService["ToBePlayed-Core"]
|
||
Security["configuration.security<br/>JwtAuthenticationFilter"]
|
||
subgraph Library["library"]
|
||
LibApi["api"] --> LibService["service"] --> LibRepo["repo"]
|
||
end
|
||
Stats["stats"]
|
||
Version["version"]
|
||
Account["account"]
|
||
Follow["follow"]
|
||
Inbox["inbox"]
|
||
Rewards["rewards"]
|
||
Steam["steam_adapter"]
|
||
Igdb["igdb_adapter"]
|
||
Speedrun["speedrun_adapter"]
|
||
LibService --> Steam
|
||
LibService --> Igdb
|
||
LibService --> Speedrun
|
||
end
|
||
Browser -->|"tbp_access cookie"| Security
|
||
Rewards --> Db
|
||
Security -->|"JWKS by kid"| Auth["ToBePlayed-Auth"]
|
||
Steam --> SteamApi["Steam Web API"]
|
||
Steam -->|"HEAD"| SteamCdn["Steam CDN"]
|
||
Igdb --> IgdbApi["IGDB API"]
|
||
Igdb -->|"OAuth token"| Twitch["Twitch"]
|
||
Speedrun --> SpeedrunApi["speedrun.com API"]
|
||
LibRepo --> Db[("PostgreSQL tbp_core")]
|
||
Stats -->|"listed_user_game, counted_user_game, game_mood, coming_game, user_game_status_change, deleted_user, blocked_user, listed_user, account_name, game_alias"| Db
|
||
Follow -->|"follow, follow_digest, inbox_note, public_user, sync_change, user_game_status_change, account_name"| Db
|
||
Inbox --> Db
|
||
Auth -->|"service token<br/>/internal/accounts"| Account
|
||
Account -->|"per-user rows, deleted_user, blocked_user, listed_user, public_user, follow, account_name, user_seen_at"| Db
|
||
```
|
||
|
||
| Module | Content |
|
||
|---|---|
|
||
| `library` | Sync, backlog, manual games, imports, playtime history, wishlist, achievements, public profile data |
|
||
| `stats` | Public numbers for the landing page |
|
||
| `version` | `GET /api/version`: commit and build time of the running image |
|
||
| `follow` | Follows (`follow`), the digest cursor per followed user (`follow_digest`) and `FollowDigestService`: one note per follower and batch, from `sync_change` and the hand rows of `user_game_status_change` |
|
||
| `inbox` | `inbox_note`: the notes of a user, `GET /api/inbox`, the 90-day purge |
|
||
| `account` | Driven by Auth: tombstones in `deleted_user`, purge and orphan prune of the per-user rows; blocks in `blocked_user`, listing in `listed_user` (users Auth shows by name in owner lists, #26), public profiles and their sections in `public_user` (#27), role changes in `account_role_change`, display names in `account_name`, Steam ID changes stop the nightly sync |
|
||
| `steam_adapter` | Steam Web API client, availability check of Steam CDN pictures |
|
||
| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search |
|
||
| `speedrun_adapter` | speedrun.com client: game search by name, verified runs |
|
||
|
||
`follow` and `inbox` read `account_name`, `public_user`, `blocked_user`, `deleted_user`, `sync_result`, `sync_change`,
|
||
`user_game` and `user_game_status_change` in SQL; the sync tells `follow` through `shared.event.LibraryActivityEvent`.
|
||
|
||
**One shared schema.** All modules use one database `tbp_core`, and some read or write another module's tables in
|
||
SQL, which ArchUnit cannot see: `stats` reads `listed_user`, `account_name`, `game_alias`, `listed_user_game`, `counted_user_game`, `steam_game`, `game_mood`, `coming_game`, `user_game_status_change`, `deleted_user` and `blocked_user`; `account`
|
||
deletes the per-user rows of the library tables (`AccountCleanupRepository`; a new library table with `user_id` goes
|
||
into its list) and writes `user_seen_at` and `last_synced_at` of `user_steam_sync` (`AccountStateRepository`); `library`
|
||
reads `deleted_user` and `blocked_user`. A module that moves to its own service takes these reads along as calls.
|
||
The listed rule ("still owned, not hidden") is `UserGameRepository.LISTED` for JPQL and the view
|
||
`listed_user_game` for native SQL; V24 says what a migration of `user_game` must do with the view. The numbers that
|
||
count games once per linked set read `counted_user_game` (V28), built on `listed_user_game`: a migration that changes
|
||
a column of `user_game` drops this view first, then `listed_user_game`.
|
||
|
||
Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` (JPA entities) and `repo`
|
||
(repositories). `src/test/java/core_service/architecture/ArchitectureTest.java` checks these rules on every build:
|
||
|
||
- Modules do not depend on each other. Only `library` may use `steam_adapter`, `igdb_adapter` and
|
||
`speedrun_adapter`.
|
||
- No cycles between `modules`, `shared` and `configuration`.
|
||
- `api` never uses `repo`, and `repo` never uses `api`.
|
||
- `domain` never uses `api`, `service`, `repo` or an adapter: the import stores IGDB hits as the own record
|
||
`IgdbGame`, so a change of the adapter's `IgdbApiService.Game` cannot break stored imports.
|
||
- `service` never uses `api`, except the adapters' response DTOs and a fixed list of 8 services that still build
|
||
their module's response DTOs. The list only gets shorter.
|
||
- Every class in `repo` ends with `Repository`.
|
||
- `shared` never uses `modules`, and `modules` never use `configuration`.
|
||
- A `*Service` class in a module's `service` package has `@Service`. A `*Controller` class in `api` has
|
||
`@RestController`, `@ControllerAdvice` or `@RestControllerAdvice`.
|
||
|
||
### Sync flow
|
||
|
||
1. `GET /api/library/games`, `/games-count` and `/picker` call `LibrarySyncService.syncIfStale`. It starts a sync
|
||
when the last one is older than 24 hours, older than 15 minutes after a failure, or was for another Steam ID.
|
||
`POST /api/library/sync` starts one at once, but at most every 60 seconds (else `429` with `Retry-After`).
|
||
2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A
|
||
request without a `steamId` claim (the user unlinked the Steam account in Auth) takes the user out of it.
|
||
Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and
|
||
dates finished games by the unlock of their ending achievement
|
||
(`FinishedEstimateService`, `user_game.finished_at_estimate`).
|
||
3. `LibrarySyncWorker` runs on the background pool. `user_steam_sync` lets only one run per user start, and none for
|
||
a user in `deleted_user` or `blocked_user`.
|
||
4. The worker loads the wishlist and the owned games from Steam. It updates `steam_game` and `user_game`, adds
|
||
the playtime growth of the day to `user_game_playtime_day`, and soft-deletes Steam games the user no longer
|
||
owns. The growth counts at most Steam's playtime of the last two weeks, so a game seen for the first time brings
|
||
no old playtime, and one game gets at most 24 hours per day. A BACKLOG game with new playtime becomes PLAYING, unless the user chose its status. When Steam returns no
|
||
games, the library stays as it is.
|
||
5. `TimeToBeatService` and `MoodService` ask IGDB for missing or old time-to-beat, mood and genre data, for Steam
|
||
games and manual games. Hidden games only get their playtime updated.
|
||
6. After the commit, `LibrarySyncedEvent` starts `AchievementSyncService` and `SpeedrunService` (backlog games only)
|
||
in the background. The achievement sync also fetches Steam's global unlock rates with each schema, and sets
|
||
FINISHED for a game whose ending achievement is unlocked while the user never chose its status.
|
||
|
||
The frontend polls `GET /api/library/sync-status` until the run ends.
|
||
|
||
Followers (#27): when the achievement run ends, `AchievementSyncService` publishes `LibraryActivityEvent`, and
|
||
`FollowDigestService` writes one inbox note per follower from the new sync result (`ADDED`, `ACHIEVEMENTS`, `STATUS`
|
||
and `WISHLIST` rows of `sync_change`) and the hand changes since the cursor. Kinds that the profile hides (`public_user.sections`)
|
||
are left out, and the cursor moves anyway. A job every 10 minutes reports the hand changes alone once a user was quiet for 10
|
||
minutes; it never takes a sync result, which may still wait for its achievements.
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant W as LibrarySyncWorker
|
||
participant A as AchievementSyncService
|
||
participant F as FollowDigestService
|
||
participant D as Db
|
||
W->>D: sync_result, sync_change (ADDED, STATUS, WISHLIST)
|
||
W-->>A: LibrarySyncedEvent
|
||
A->>D: sync_change (ACHIEVEMENTS)
|
||
A-->>F: LibraryActivityEvent
|
||
F->>D: lock follow_digest, count sync + hand changes
|
||
F->>D: inbox_note per follower (public, not blocked, not deleted)
|
||
F->>D: move cursor
|
||
```
|
||
|
||
At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/library/broken-images` (table
|
||
`broken_image`, one row per game and kind). Reports only count for Steam games the user owns.
|
||
|
||
### Background work
|
||
|
||
Background work runs on two pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it):
|
||
|
||
- The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view
|
||
starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with
|
||
a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user
|
||
starts it again.
|
||
- The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun
|
||
refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page
|
||
views' threads. A task is small (a user or a few appids), and none is dropped.
|
||
|
||
Imports have their own two threads, and `game_import` is their queue. Scheduled jobs (nightly sync, image repair,
|
||
import watchdog) run on the scheduler.
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
View["page view, add, merge"] -->|"stale games only"| Pool["default pool<br/>4 threads, queue 100"]
|
||
Sync["sync request"] --> Pool
|
||
Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
|
||
View -->|"speedrun"| FollowUp
|
||
Pool --> Apis["Steam, IGDB, speedrun.com"]
|
||
FollowUp --> Apis
|
||
Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
|
||
Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"]
|
||
```
|
||
|
||
Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks
|
||
live in memory: a second instance would ask Steam and IGDB twice, and the database would stay consistent (sync
|
||
claim, upserts). A restart drops what waits in the pools; it is all refresh work and is asked again. Imports survive
|
||
a restart.
|
||
|
||
### Corrections by an auditor
|
||
|
||
An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag, genres and time to beat for
|
||
every user (`game_correction`, V20). The value is also written into the cached row (`steam_game`, `game_speedrun`,
|
||
`game_mood`), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name),
|
||
the image repair (image), the IGDB add (name, image), the mood, speedrun and time to beat upserts (SQL). `SteamGame` has no
|
||
setter for name and image: `applyCorrection` is the only Java writer, a refresh uses the guarded updates in
|
||
`SteamGameRepository`. "Back to source" (`DELETE`) writes back the source value from the first correction and
|
||
lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP.
|
||
The time to beat is one field for both times (`normallyMinutes`, `completelyMinutes`, each empty or 1 to 60000,
|
||
completely not below normally) written into `game_time_to_beat`, so pile hours and stats use it; the IGDB `hastily`
|
||
time is not correctable. Every correction and every reset writes `admin_audit` (`GAME_CORRECT`,
|
||
`GAME_CORRECTION_RESET`). `GET /api/library/catalog` lets an auditor find and correct a game they do not own.
|
||
Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the
|
||
source value once (its guard sees the state from before the save); the auditor saves again. `save(new SteamGame(...))` on a
|
||
stored key also overwrites a corrected name; the sync and the IGDB add do it only after `findById` found no row.
|
||
|
||
### Merging duplicates
|
||
|
||
One game can have two keys: a Steam appid and minus its IGDB id, or two IGDB ids. An AUDITOR merges them for every
|
||
user (`GameMergeService`, V21). The key off Steam always goes into the other one; two Steam appids are refused,
|
||
since the sync would bring both back. In one transaction:
|
||
|
||
- A user with only the old row: the row moves to the new key with its status history (FK `ON UPDATE CASCADE`).
|
||
- A user with both rows: one row stays on the new key. Status, rating, notes and priority come from the row whose
|
||
current status the user chose (both: the one chosen last); a gap is filled from the other row. Source, platform and playtime stay, unless the
|
||
Steam copy is no longer owned. Both status histories stay; the merge itself logs no status change.
|
||
- The shared rows of the old key (catalog row, moods, speedrun, time to beat, corrections, broken images) are
|
||
deleted. So are its queue pairs, its requests and its `game_edition` row; its set row goes to the kept key (see
|
||
"Editions of one game").
|
||
- `game_alias` points the old key to the new one (kept flat), so a later add of the old IGDB game lands on the new
|
||
key. `admin_audit` gets one `GAME_MERGE` row: who, both keys, both names and how many rows moved or were combined.
|
||
|
||
Merges made since V29 can be undone for 30 days (`POST /api/library/merges/{mergeId}/undo`, AUDITOR). The merge saves
|
||
a copy in its own transaction: `game_merge_undo` (the deleted shared rows as JSON per table, the link set after the
|
||
merge, the flattened aliases) and `game_merge_undo_user` (per user: both `user_game` rows as JSON, the history ids and the
|
||
reward ledger ids the merge moved). The copy holds notes, so it goes with the account. A nightly job
|
||
(`app.library.merge-undo-purge-cron`) deletes copies older than 30 days; older merges answer `410`.
|
||
|
||
```mermaid
|
||
stateDiagram-v2
|
||
[*] --> MERGED: POST /games/{appid}/merge (copy saved)
|
||
MERGED --> UNDONE: POST /merges/{id}/undo (30 days, newest first)
|
||
MERGED --> FINAL: nightly purge after 30 days
|
||
```
|
||
|
||
The undo, in one transaction (the newest merge of a kept key goes first, else `409 NEWER_MERGE`):
|
||
|
||
- Shared rows (catalog row, moods, speedrun, time to beat, corrections, broken images, `game_edition`) come back;
|
||
`game_alias` of the old key is removed and the aliases the merge flattened point to the old key again.
|
||
- A user with only the old row: the row moves back if it is still on the new key, still added by hand (no Steam
|
||
takeover) and the user has no old row. It keeps its current status, rating and notes.
|
||
- A user with both rows: both rows are back if the kept row still exists. If the user did not change status, rating, notes or
|
||
priority of the kept row since the merge (compared with a copy taken right after the merge), each row gets its own old
|
||
values. If they did, both rows get the current values. The kept row gets its Steam ownership back when the merge
|
||
had replaced it and nothing took the row over since. The old status history goes back to the old key; the log
|
||
entries the undo itself causes are removed.
|
||
- Ledger rows the merge moved go back to the old key (so does what a moved user earned on the row after the merge), only when the user has the old row
|
||
again. Nothing is paid twice. A game the user hid after the merge stays hidden on the restored rows.
|
||
- Links: when the kept key's set is still as the merge left it, the old sets come back with their `set_id`s; if
|
||
links changed since, they stay and the old key comes back unlinked (the audit detail says "links kept").
|
||
- Users who deleted their account are not restored. `admin_audit` gets a `GAME_MERGE_UNDO` row; the `GAME_MERGE` row stays.
|
||
|
||
`GET /api/library/games/{appid}` follows `game_alias` when the user has no row on the asked key, and answers with the
|
||
kept `appid` (the frontend then replaces the URL). After an undo the alias is gone. The game search shows games already in ToBePlayed first, with their catalog name and image, and
|
||
games other users added by hand that IGDB did not return; `inCatalog` marks both (only those can be merged).
|
||
|
||
### Editions of one game
|
||
|
||
IGDB tells which library keys are one game: `game_edition` (V28) holds a "same game" key per library key, refreshed
|
||
like `game_mood` (stale check, nightly sync, add, detail view; `app.library.edition-ttl-days`). The key is
|
||
`version_parent` for an edition, else `parent_game` when `game_type` is 9 (remaster) or 11 (port), else the game's own
|
||
IGDB id; remakes (8) keep their own id. `null` when IGDB does not know the game.
|
||
|
||
- `game_link(appid, set_id)`: the keys an auditor linked, for every user. A set is every row with one `set_id`.
|
||
An unlink takes one key out; a set of one is dropped. A merge puts the kept key in the merged key's set; when both have a set, the sets join.
|
||
- `game_link_pair` is the auditor queue (`OPEN`, `DISMISSED`, `LINKED`). IGDB pairs are inserted when the queue is
|
||
read; a row in any state keeps the pair out for good. An unlink dismisses the pairs of that key with the former
|
||
members. `game_link_request` holds who asked for a pair (the queue shows only the count). A user asks for a pair of
|
||
two games of their own library; at most 20 open requests per user (a per-user advisory lock makes the cap hold for parallel calls), a dismissed pair answers `409 PAIR_DISMISSED`.
|
||
- Pile, stats, snapshots and the dashboard count a set once (`counted_user_game`, see "Stats"); the library
|
||
filter `hideDuplicates=true` hides the lesser editions. Pile Points are unchanged: `STARTED` and `FINISHED` stay per row,
|
||
and the `MONTH` bonus reads `backlog_snapshot`, which counts a set once.
|
||
- The game detail has `linked` and `editions` (the user's other listed games of the set); a game row has `editions`
|
||
(size of the set in the user's library, `null` below 2).
|
||
|
||
### Deleted accounts
|
||
|
||
Auth drives it (see Auth's readme). `POST /internal/accounts/deleted` adds a tombstone in `deleted_user`: the user
|
||
gets no more syncs and leaves the landing stats and the pile snapshots at once. Auth's nightly job sends the whole
|
||
flagged set (`PUT`), then `purge` for users flagged 30 days ago (only tombstoned ones), then `prune` with all its
|
||
user ids, which removes the rows of users Auth does not know (refused for more than
|
||
`app.account-cleanup.max-orphans`, default 5). `app.account-cleanup.dry-run` (default `true`) only counts the rows.
|
||
The tombstone also ends the user's follows in both directions without a note and removes the user's `public_user` and
|
||
`follow_digest` rows; the purge removes the user's inbox notes and the notes about the user.
|
||
The purge also removes the user's `account_name` and `listed_user` rows and link requests (`game_link_request`).
|
||
|
||
### Blocked users and Steam ID changes
|
||
|
||
Auth sends a user's state (`POST /internal/accounts/states`) after a Steam link, an unlink, a Steam ID another account
|
||
took by a verified link, a display name change, a block, an unblock and a change of the owner-list opt-in (`listed`, #26), of the public profile or of the profile tiles (`publicProfile`, `profileSections`, #27); its nightly job sends every user (`PUT`). A blocked user is in
|
||
`blocked_user`: no nightly sync, no new sync run, not in the landing stats. `JwtAuthenticationFilter` refuses a blocked
|
||
user's token and a token issued before the user's last role change (`account_role_change`, sent as `roleChangedAt`),
|
||
through the cache `loginStates` (30 s, evicted by every state call). An Auth that does not send `roleChangedAt` changes
|
||
only the blocks. A new Steam ID or no Steam ID takes the user out of the nightly sync; the next visit syncs
|
||
the ID of the token (Core's copy changes only then, so that sync sees the change and drops the old account's games). A visit with an access token that still has the old ID (max 15 min) can sync it once
|
||
more and put the user back on the nightly list; the nightly `PUT` at 04:15 takes the user off again, so at most one
|
||
nightly sync (23:55) runs with the old ID. Unblock and the nightly `PUT` lift the block.
|
||
|
||
### Manual games next to Steam games
|
||
|
||
A game the user adds by hand comes from IGDB (`ManualGameService`). Steam games and manual games share
|
||
`steam_game` and `user_game` and one key, the column `appid` (see `GameKeys`):
|
||
|
||
- `appid > 0`: the Steam appid. A manual game gets it when IGDB knows the game on Steam.
|
||
- `appid < 0`: minus the IGDB id, for a game that is not on Steam. Steam appids are always positive, so the two
|
||
never meet.
|
||
|
||
`user_game.source` (`GameSource`) says who owns the row. The sync never changes or removes a `MANUAL` row, with
|
||
one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes
|
||
`STEAM`, and status, rating and notes stay. Only `MANUAL` rows can be edited (`PUT /api/library/games/{appid}/manual`)
|
||
or deleted (`DELETE /api/library/games/{appid}`). Time to beat uses the Steam appid for positive keys and the IGDB
|
||
id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get
|
||
a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two
|
||
(see "Merging duplicates").
|
||
|
||
### Imports
|
||
|
||
Many games at once (V22, `GameImportService`): a list, a CSV or several picks of the game search become one
|
||
`game_import` with a `game_import_row` per line. `game_import` is also the queue of `GameImportWorker`, which has
|
||
two queues, each on one thread for all users:
|
||
|
||
- **matching**: IGDB searches with typo help. One thread, so an import never takes the whole IGDB budget from the
|
||
interactive search; other users' searches may wait for the limiter meanwhile. Up to 5 candidates per line are
|
||
stored whole (JSONB), so the save needs no IGDB call, and `exact` marks a best one with the line's name.
|
||
- **saving**: adds the chosen lines the way the manual add does (same keys and duplicate check) with
|
||
`user_game.import_id` in the same transaction, then loads their time to beat and moods. A save never waits
|
||
behind another user's big matching.
|
||
|
||
Each line is stored when done, so a stopped try goes on where it stopped. A worker sets `progress_at` when it takes
|
||
an import and after every line. A watchdog (every minute) stops a try without progress for 5 minutes
|
||
(`app.library.import-stall-minutes`): the import goes to the end of its queue and the queue gets a new thread. A
|
||
restart counts as a stopped try too. After 3 tries the import is `FAILED`; the user can try it again. One import
|
||
per user can be in the matching queue (a unique partial index).
|
||
|
||
```mermaid
|
||
stateDiagram-v2
|
||
[*] --> QUEUED: POST /imports
|
||
QUEUED --> MATCHING: matching worker takes it
|
||
MATCHING --> READY: every line matched
|
||
READY --> SAVING: POST /save (the review)
|
||
SAVING --> SAVED: games added, time to beat and moods loaded
|
||
SAVED --> UNDONE: DELETE /games (undo)
|
||
MATCHING --> QUEUED: no progress for 5 min or restart
|
||
SAVING --> SAVING: no progress for 5 min or restart (end of the queue)
|
||
MATCHING --> FAILED: 3rd stopped try
|
||
SAVING --> FAILED: 3rd stopped try
|
||
FAILED --> QUEUED: POST /retry (stopped while matching)
|
||
FAILED --> SAVING: POST /retry (stopped while saving)
|
||
FAILED --> UNDONE: DELETE /games (the games added before it failed)
|
||
```
|
||
|
||
The undo deletes the games of the import that are still `MANUAL` (a game the Steam sync took over stays). A removed
|
||
row the import brings back gets no `import_id`: it holds rating, notes and history from before, so the undo keeps it;
|
||
`undo-check` counts those with a rating, notes, priority or a status change after the save.
|
||
|
||
## Run locally
|
||
|
||
### Local development without keys (profile `local`)
|
||
|
||
The whole stack runs on your machine with no API keys and no calls to Steam, Twitch/IGDB or speedrun.com. Needs
|
||
Java 25, Node 24 and Docker. Clone `ToBePlayed-Auth`, `ToBePlayed-Core` and `ToBePlayed-Frontend` next to each
|
||
other, then start them in this order (Core also starts without Auth and finds it later):
|
||
|
||
1. **Auth**: in `ToBePlayed-Auth`, `./mvnw spring-boot:run -Dspring-boot.run.profiles=local`
|
||
(IDE: active profile `local`). Port 8080.
|
||
2. **Core**: in `ToBePlayed-Core`, `./mvnw spring-boot:run -Dspring-boot.run.profiles=local`. Port 8090.
|
||
3. **Frontend**: in `ToBePlayed-Frontend`, `npm install`, then `npm run dev`, and open `http://localhost:5173`
|
||
(`localhost`, not `127.0.0.1`, so the browser sends the login cookies to every local port). The development
|
||
values must be `VITE_API_URL=http://localhost:8080` and `VITE_CORE_API_URL=http://localhost:8090`.
|
||
4. **Log in** with `dev@localhost` / `password123`. On the dashboard, start a Steam sync: the library fills with 8
|
||
games, achievements, time to beat, moods, a wishlist and one speedrun game from recorded answers.
|
||
|
||
What the `local` profile does:
|
||
|
||
- Spring Boot's Docker Compose support starts each service's `compose.yaml`: PostgreSQL for Auth (port 5432) and
|
||
for Core (port 5433), and in Core a WireMock container (port 8089, compose profile `local`) that answers in
|
||
place of Steam, Twitch/IGDB and speedrun.com from `ToBePlayed-Core/local/wiremock`.
|
||
- Auth makes a new signing key at each start (`LocalKeysEnvironmentPostProcessor`, only in this profile), so no
|
||
key is in the repo. After an Auth restart (also a devtools restart), log in again.
|
||
- Never use `local` on a server: it adds an admin with a public password. Auth refuses `local` together with
|
||
`prod`.
|
||
- Auth adds the dev user (Flyway callback in `db/local`): role `PLATFORM_ADMIN`, so user, auditor and admin pages
|
||
can all be tried ("View as" in the menu switches down), and a linked Steam ID that the recorded Steam answers
|
||
belong to. The invite code for more users is `local`.
|
||
- Core checks the login with Auth's local JWKS, like in production.
|
||
- The browser still loads game pictures from the Steam and IGDB image servers; offline they stay gray.
|
||
- "Sign in through Steam" still opens the real Steam page (no key needed). The dev user is already linked.
|
||
- A request to WireMock that has no recording answers 404 and shows up in Core's log.
|
||
- To see an IGDB link suggestion: add Hollow Knight by hand (game search, PS5). It shares IGDB's "same game" key with the
|
||
Steam copy, so the link queue (auditor) lists the pair with the IGDB badge.
|
||
|
||
To change the recorded answers, edit the JSON in `local/wiremock/mappings` (which request) and
|
||
`local/wiremock/__files` (the answer), then restart Core (it restarts the container). `LocalMocksParseTest` checks
|
||
that every answer still fits the adapters' response types. `http://localhost:8089/__admin/requests` lists what Core
|
||
asked.
|
||
|
||
### With the real services (profile `dev`)
|
||
|
||
Needs Java 25, Docker and your own keys.
|
||
|
||
```bash
|
||
./mvnw spring-boot:run -Dspring-boot.run.profiles=dev
|
||
```
|
||
|
||
Spring Boot's Docker Compose support starts PostgreSQL from `compose.yaml`, Flyway applies the migrations, and the
|
||
service listens on `http://localhost:8090`. Health: `/actuator/health`, metrics: `/actuator/prometheus`.
|
||
|
||
In the `dev` profile a request without an `Authorization` header gets a dummy user, so the API is usable without a
|
||
running Auth service.
|
||
|
||
Configuration comes from environment variables:
|
||
|
||
| Variable | Purpose |
|
||
|---|---|
|
||
| `JWT_JWKS_URL` | Auth's `/.well-known/jwks.json` |
|
||
| `JWT_PUBLIC_KEY` | Ed25519 public key (Base64 of the X.509 DER bytes), used until the first JWKS load. Set this, `JWT_JWKS_URL`, or both |
|
||
| `STEAM_API_KEY` | Steam Web API key |
|
||
| `IGDB_CLIENT_ID`, `IGDB_CLIENT_SECRET` | Twitch app for IGDB (optional) |
|
||
| `DB_PASSWORD` | Local PostgreSQL password |
|
||
|
||
## Tests
|
||
|
||
```bash
|
||
./mvnw clean verify
|
||
```
|
||
|
||
Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers.
|
||
In integration tests background work runs on the calling thread, so a test reads the result at once.
|
||
`AbstractMockedAdaptersTest` mocks Steam, IGDB and speedrun.com, `AbstractMockedLibraryServicesTest` the sync and
|
||
the refreshes of the picker and stats pages; each is one shared Spring context. Race tests wait until the other
|
||
connection waits for the row lock (`awaitLockWaiter`), not for a fixed time.
|
||
|
||
## Roadmap
|
||
|
||
- Sale alerts for wishlist games.
|
||
- Move the adapter DTOs out of `api`, then turn on strict module independence in ArchUnit and drop the adapter
|
||
exception of the "`service` never uses `api`" rule.
|
||
|
||
## License
|
||
|
||
All rights reserved. The source is public for reading only.
|