lobo 82d3b59165 feat(library): group same game across platforms into sets
One entry per game across platforms (Gitea #34, slice 1 of 3).
Same IGDB game is linked; a user can split one key off its set.
V42 keeps status, rating and notes per set; rewards pay once per set.
2026-10-09 10:57:23 +02:00
2026-09-30 15:00:47 +02:00

To Be Played – Core Service

Business service of To Be Played, a Steam backlog tracker. It syncs the user's Steam library, stores the backlog (status, rating, notes, priority), tracks playtime per day, syncs the wishlist and adds time-to-beat data from IGDB.

To Be Played is split into three repositories:

Repository Role
ToBePlayed-Auth Users, login, settings, signs JWTs
ToBePlayed-Core (this one) Steam library, backlog, wishlist, stats; verifies JWTs
ToBePlayed-Frontend React single-page app

Features

  • Steam sync: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly sync. Resilience4j rate-limits every Steam call and tries the owned games up to three times (never again on a 4xx; the wishlist and the calls a page waits for once); Caffeine caches game counts and player summaries. A sync commits its claim (RUNNING, with sync_started_at), its writes and its finish apart, and calls Steam and IGDB outside any transaction: a second sync sees RUNNING and skips, a failed one stores FAILED. A run older than app.library.running-stale-minutes can be taken over; its own finish then changes nothing.

  • Backlog: status (BACKLOG, PLAYING, FINISHED, COMPLETED = 100 %, DROPPED), rating 1–5, notes, priority, filters and sorting. "Pile of Shame" summary with a random suggestion of what to play.

  • Picker: seven rows of up to 3 backlog games each, for a time budget (IGDB normal time to beat), in this order: NEXT_UP (the first games of the user's own backlog order), SERIES (next in series: for each of the 6 most played IGDB series the user played, one backlog part, the parts released after the latest played part first), SIMILAR ("because you played": for each of the 3 most played games the first backlog game sharing its moods, no game twice), STUDIO (for each of the 6 developers of the user's most played games, one backlog game with a credit of that company, developer credits first; publishers are never anchors, so a big publisher adds at most one card), FRIENDS (for each followed user with a public profile, one backlog game of the user that the friend finished or played 5 h or more, strongest signal first; a finish and playtime need the SUMMARY or STATUS profile section (FINISHED alone lists only 12 games); DROPPED, hidden and removed games never count; the friend's current display name, never an id), LONGEST (oldest firstSeenAt) and RANDOM (the mood row: random, HIGH priority first, then MEDIUM, then the rest). The mood filters only the RANDOM row; budget and exclude filters apply to every row. A row with no game is left out. A game can show in more than one row. IGDB series (collections) and companies are stored per game (game_credit, game_credit_check) and refreshed with the other IGDB data; a scheduled backfill (app.library.credit-backfill-*) fetches them for existing games, 250 games per 30 s, so the two rows fill up over time after a deploy. Every game carries a reason (source, plus the anchor game for SIMILAR, seriesName for SERIES, studioName for STUDIO, friendName, friendFinished and friendPlaytimeMinutes for FRIENDS or firstSeenAt for LONGEST). Games already shown can be excluded for a reroll. Moods STORY, COOP, CHILL, ACTION, SCARY come from IGDB genres, themes and game modes; SPEEDRUN from speedrun.com.

  • Follow users (#27): a logged-in user follows people with a public profile by display name (/api/follows, at most 500, never an id in an answer) and gets one inbox note per followed user and batch (/api/inbox): a sync with its achievement run, or the hand changes of a quiet 10 minutes. A note counts new games, achievement unlocks, status changes and Steam wishlist adds, and leaves out what the followed user's profile hides (public_user.sections). Follows end for good, with a note, when the followed user goes private, and without a note when the user is blocked or deleted. The nightly job at 03:15 deletes notes older than 90 days (app.inbox.keep-days); the list shows the newest 50.

  • Backlog order: every listed BACKLOG game (not removed, not hidden) has a row in backlog_order (user_id, appid, position), kept by a database trigger on user_game like the status history: a game entering the backlog goes to the bottom, a game leaving it loses its row. V34 seeded the first order (priority HIGH, MEDIUM, rest, then date added). PUT /api/library/backlog replaces the whole order under a row lock; ids not in the backlog are ignored and games not sent go after the sent ones. A merge keeps the better position of the two games. Coming games are in the list; released=true leaves them out.

  • Coming games: a game with a release date after today (Berlin date) or Steam "coming soon", and no playtime or achievement of any user, is not picked and not on the pile (live count, new snapshots, pile stats, time to beat sum). It keeps its status and shows coming / comingOn. Date: IGDB first_release_date, else Steam's store (GetItems with include_release), else unknown (= released). View coming_game, table game_release; a year-only IGDB date stays a placeholder until the real date comes (coming rows refresh daily).

    flowchart LR
        A["anchor: random of top 5 played"] --> S[similar]
        S --> M["PickMix: rotate 4 sources, dedupe, random fallback"]
        L[longest waiting] --> M
        R[random] --> M
        N["next up: random of top 5 in order"] --> M
        M --> O["games + reason"]
    
  • Speedrun games: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns the speedrun.com link as speedrunUrl. speedrun.com has no Steam or IGDB link and no key, so one queue works for all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days.

  • Achievements: after every library sync a background job fetches the achievement schema of each owned game and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5 games closest to 100 %.

  • Playtime history: one row per game and day, for the activity view and the "most played in the last two weeks" dashboard banner.

  • Finished from achievements or playtime: EndingAchievement finds the one achievement of a game that means "story end" (English text rule, Steam's global unlock rate between 1 and 80 %, the easiest variant wins, more than 3 candidates = a game without one end). When the user unlocks it, never chose the status and never answered "Not yet", the achievement sync sets FINISHED, but only for a certain ending: its text says so and no other achievement of the game could be the end. A status the user chose is never changed; such games get a hint on the game page instead. A second, weaker hint compares the playtime with IGDB's normal time to beat (at least 90 %), only for games IGDB does not mark as endless (sandbox, survival, 4X, simulator, MMO). "Not yet" hides a hint per game; a dismissed playtime hint still lets the ending hint through.

  • Question queue (Gitea #32): GET /api/library/questions lists one open question per game: Start it? (BACKLOG with playtime), Finished? (BACKLOG or PLAYING, playtime at least IGDB's normal time to beat, else 10 h; never for IGDB-endless games), Dropped? (PLAYING, no Steam playtime in the last 2 weeks and no sign of play for 90 days) and Rate it? (FINISHED or COMPLETED without a rating). The order is the priority. OpenQuestion.of is a pure rule over QuestionFacts, which one SQL query loads for the whole library (counted_user_game, so a linked set is one card). Every rule starts from the status, so a game the sync already finished or started drops out by itself. A "No" is stored in user_game_question (V41): Start it? and Finished? come back after 5 more hours of playtime, Dropped? after another 90 idle days, a Rate it? "Skip" never. POST /api/library/questions/{appid} ({question, yes, rating?}) checks that the question is still open (409 QUESTION_GONE, 404 GAME_NOT_FOUND, 400 INVALID_ANSWER for a yes to RATE without a rating) and then calls BacklogService, the same write path as the game page. GET /api/library/questions/count uses the same code as the list, so the badge equals the cards.

flowchart LR
    Facts["counted_user_game + playtime days + status log + time to beat + user_game_question"] --> Rule["OpenQuestion.of"]
    Rule --> Card["question card"]
    Answer["answer"] --> Backlog["BacklogService"] --> UserGame["user_game"] --> Trigger["V14 trigger: status log"] --> Settle["RewardService.settle"]
    Answer -->|no| Snooze["user_game_question"]
  • Hide games: a user can hide a game, Steam or added by hand (e.g. a playtest or a tool). It leaves every list and number: library, picker, Pile of Shame, activity, achievement summary, public profile and landing page stats. The sync still updates its playtime, but no achievements, time to beat, moods or speedrun data are fetched for it. The library lists hidden games with hidden=true.
  • Stats: UserStatsService builds the numbers of the stats page per group, so every page asks only for the groups it shows: totals and statuses; the pile (time to beat, weeks "at your pace" of the last 90 days, years of 40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the status history); games and playtime per platform (PLATFORMS: Steam games as "Steam", manual games by platform, trimmed and ignoring case, the most used spelling is the label); the Pile of Shame trend (90 days of backlog_snapshot, plus "this month: added / done", where the first sync of a Steam account is no "added", see user_steam_sync.history_since). Every number counts only listed games (still owned, not hidden: the view listed_user_game, V24). A linked set (see "Editions of one game") counts once: its most advanced edition (status, then playtime, then the older first_seen_at; counted_user_game, V28), so the set is on the pile only when every owned edition is BACKLOG. Playtime sums, most played, genres, moods and platforms still use every listed game.
  • Drill-down: every countable number opens the games behind it. A StatKey (library), CommunityKey (landing page) or a Pile Points track/bonus row names a number; its SQL "game set" is the same text the number counts, so the list and the number cannot drift. 40 games per page (at most page 250), a fixed order per key. A profile visitor gets a list only when the ticket shares that section, or the STATS section and the key's group (403 NOT_SHARED; OVERVIEW keys are for the owner only). The community list has one row per game with libraries and minutes, never a user, and is cached 10 minutes like the numbers. A bad argument is 400 BAD_ARG.
  • Status history: a database trigger logs every status change of a library game in user_game_status_change (old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older changes are not known.
  • Pile Points (modules/rewards): users earn points for playing games off the pile, in an append-only ledger (reward_ledger, V26; the balance is the sum). No code path pays: a settle (on GET /api/rewards/wallet and nightly at 02:30, app.rewards.settle-cron) derives rows from the status log, playtime days, achievements and pile snapshots, once per user, kind and game (ref). Rules and numbers: RewardRules; a Steam finish needs playtime and one proof of fresh play (ending achievement or playtime around the finish, or half of the IGDB time played since the finish; old history pays nothing). The month bonus needs a finished game in that month. Games added by hand pay 15 for a finish, at most 3 per ISO week. The settle takes a per-user advisory lock. V26 also pays a one-time starter grant for finishes from before the status log. Shop: a whitelist of 57 looks (ShopItem; 45 buyable and 12 unlock-only) in eight slots, including animated completed effects, page backgrounds (CSS, SVG and two canvas engines in the Frontend), banner backgrounds (V27 widens reward_equipped.slot) and three buyable skins (full themes, slot SKIN). The unlock-only items (price 0, never sold): the settle books an UNLOCK row (ref = item code, 0 points) once a track reaches the item's target, retroactively and never lost. The five tracks (Unlocks): games finished (FINISHED + FINISHED_MANUAL rows + starter grant / 20), games completed, weeks in a row with a finish, months in a row with a MONTH bonus, months in a row with more Steam playtime than the month before (hidden games count too). A set (user_game_set, see "Editions of one game") pays STARTED, FINISHED/FINISHED_MANUAL and COMPLETED once: a game whose other edition was paid is no candidate, and two editions of a set give one candidate (the first event). A BUY or UNLOCK row proves ownership, reward_equipped holds what is worn. A platform admin owns every item (the role of the login token, or the role claim of the profile ticket for visitors).
  • Editions and links: editions, remasters and ports of one game stay separate library rows, but their keys are one game (a set): IGDB's "same game" pairs are linked automatically, an AUDITOR can link or unlink keys for every user, and a user can split a key off a set for themselves (see "Editions of one game"). A set shares one status, rating, note and priority. Users can ask for a pair; the auditor queue holds user requests and the IGDB pairs not linked yet (OPEN only). Every link, unlink and dismissal goes to admin_audit (GAME_LINK, GAME_UNLINK, GAME_LINK_DISMISS); auditors read that log through GET /api/library/audit-log, with the actor's display name from Auth's account state push.
  • Wishlist: synced from Steam with store price and discount.
  • Broken image repair: the frontend reports a Steam cover or achievement icon that does not load. At 01:30 (Europe/Berlin) a job checks up to 200 reports. A cover that loads again is left alone; else the store's hashed header picture (IStoreBrowseService/GetItems with assets) goes to steam_game.image_url. When a few achievement icons do not load, the schema comes fresh from Steam; an empty answer keeps the stored one. A report that cannot be repaired is tried again on the next report, at most 3 times, then after 30 days.
  • Time to beat: story, extras and completionist hours from the IGDB API. Optional: without credentials the feature is off and the rest works.
  • PlayStation link (optional, unofficial API): the user pastes an NPSSO once, Core keeps only the encrypted refresh token (about 10 days). Every library sync then adds the played PS4/PS5 titles that match IGDB exactly, keeps the unsure ones for the import review, and updates playtime and last played date. The same sync reads the PS4/PS5 trophy sets of the matched games: they count as achievements (platinum = perfect game) in the detail page, stats, tiles, public profile, sync banner and follower digest. See "PlayStation link". Off without APP_PSN_TOKEN_KEY.
  • Public profile data: totals, status counts, games in progress and finished games with rating, perfect games and average completion (the numbers count a linked set once; the lists keep every edition). Readable only with a profile ticket issued by Auth. Never returns notes, priority or the Steam ID. stats holds only the stat groups the user shares (the ticket's profileStats claim, derived in Auth from the profile tiles); none by default. A section the user hides (the ticket's sections claim: SUMMARY, STATUS, PLAYING, FINISHED, STATS) is null and runs no query; a ticket without the claim shows all.

Authentication

Core never issues tokens. It verifies Auth's EdDSA (Ed25519) JWTs with public keys only. It loads them from Auth's JWKS by the token's kid and loads again when a new kid shows up (at most every 10 seconds). Each load replaces the key set: a key that left the JWKS stops working. JWT_PUBLIC_KEY is used only until the first JWKS load works. The token comes from Auth's tbp_access cookie or a Bearer header, whichever verifies first. Every token names its kind in aud (tbp-login, tbp-public-profile, tbp-internal, with iss=tbp-auth); a login's sub is the user id. Core reads steamId and the active role (missing or unknown = USER) from the claims and grants that role and every lower one (USER < AUDITOR < PLATFORM_ADMIN). A token without aud is refused. The claims are pinned in src/test/resources/contract/auth-core-contract.json, the same file as in Auth; the tests build tokens only from it. POST, PUT and DELETE need an X-TBP-Client or Authorization header (CSRF guard), and CORS allows only the frontend origin. A token for another audience (for example a profile ticket) is rejected on normal endpoints; the profile ticket is accepted only by GET /api/public/profiles/library in the X-Profile-Ticket header. Every error answer of a controller has the body {code, message} (a 401 or 403 of the security filters has none); a refused request is 400 VALIDATION_FAILED with fields: [{field, message}], a duplicate key 409 CONFLICT, anything unexpected 500 INTERNAL_ERROR without details (shared/api/GlobalExceptionController). /internal/** has its own filter chain: it accepts only a service token of Auth (aud=tbp-internal, with exp, sent as Bearer), never a login, also not in the dev profile. Traefik exposes the path, so the signed token is its only guard.

Every service method a controller calls carries one role marker, on the method or on its class (the method wins): @RequiresRole(Role.X), @Public (no login, e.g. landing stats; the public profile checks its ticket) or @Internal (only Auth's service token). A Spring Security method interceptor checks it while a request is served; scheduled jobs and background work run without a user and are not checked. That stays on purpose: the sync worker calls SteamApiService and the import worker ManualGameService, both @RequiresRole(USER), without a request. Auth denies instead, because it has no such caller. A denied call answers 401 without a login, else 403. The URL rules in SecurityConfig stay as the outer guard. ArchUnit fails the build for an unmarked method, two markers, a controller calling an @Async method (the check would never run), and a service method calling another service's method that needs more.

Mirrored in Auth. These files exist in both services with small differences; a change in one needs the same change in the other: ClientHeaderFilter, RoleAuthorizationManager, MethodSecurityConfig, Role, RequiresRole, Public (only Core has Internal), UserPrincipal, the login token check in JwtService.parseLoginToken, the ArchUnit role rules (RoleAnnotationRules) and src/test/resources/contract/auth-core-contract.json (ToBePlayed-Deployment/scripts/check-contract.sh compares the two).

API

Method Path Auth
GET /api/rewards/wallet (balance, catalog, owned, equipped, last 50 ledger rows with the game name, unlock progress per track) JWT
POST /api/rewards/purchases ({item}; 200 wallet, 409 ALREADY_OWNED / NOT_ENOUGH_POINTS / NOT_FOR_SALE) JWT
GET /api/rewards/progress/{track}/games?page=, /api/rewards/history/{kind}/{ref}/games?page= (the games behind a track or a MONTH / STREAK bonus; 400 NOT_A_GAME_LIST) JWT
PUT /api/rewards/equipped/{slot} ({item}, null = take off; 200 wallet, 409 NOT_OWNED, 400 WRONG_SLOT) JWT
GET /api/library/games (hideDuplicates=true leaves out the lesser editions of a linked set; source=STEAM,PSN keeps rows whose own source or a listed linked edition's source is selected; ignored with hidden=true; 400 on an unknown source), /api/library/games/{appid} JWT
PATCH /api/library/games/{appid}/backlog (only the sent fields; null clears one) JWT
PUT, DELETE /api/library/games/{appid}/corrections/{field} ({value}; name, image, speedrun, moods, endless, genres, time_to_beat = {normallyMinutes, completelyMinutes}; 204, 400 INVALID_CORRECTION) JWT, AUDITOR
GET /api/library/catalog?q=&page=&size= (every game some user still has, by name part, PageDto of {appid, name, imageUrl, platforms}; q at most 100 characters; size 20, 40 or 80) JWT, AUDITOR
GET /api/library/catalog/{appid} (what the correction dialog reads: name, image, speedrun, moods, endless, genres, timeToBeat, corrections; 404 GAME_NOT_FOUND) JWT, AUDITOR
POST /api/library/games/{appid}/merge ({otherAppid}; 200 {appid} = the key that stays, 400 SAME_GAME / STEAM_GAMES, 404 GAME_NOT_FOUND) JWT, AUDITOR
POST /api/library/merges/{mergeId}/undo (mergeId = the id of a GAME_MERGE audit row, within 30 days; 204, 404 MERGE_NOT_FOUND, 409 UNDO_NOT_AVAILABLE / NEWER_MERGE, 410 UNDO_EXPIRED) JWT, AUDITOR
DELETE /api/library/games/{appid}/set (the user splits the key off its set for themselves; 204, 404 NOT_LINKED when no other listed edition is in the user's set) JWT
POST, DELETE /api/library/games/{appid}/link (POST {otherAppid}, 204, 400 SAME_GAME, 404 GAME_NOT_FOUND, 409 ALREADY_LINKED; DELETE takes the game out of its set, 204, 404 NOT_LINKED) JWT, AUDITOR
GET /api/library/audit-log?action=&page=&size= (the newest admin_audit rows, PageDto of {id, actorUserId, actorName, action, appid, gameName, targetAppid, targetGameName, detail, createdAt, undoable}; action is GAME_MERGE, GAME_MERGE_UNDO, GAME_LINK, GAME_UNLINK, GAME_LINK_DISMISS, GAME_CORRECT or GAME_CORRECTION_RESET; size 20, 40 or 80) JWT, AUDITOR
GET /api/library/link-queue?page=&size= (PageDto of {a, b, igdb, requests, createdAt}: both games, most requested first; size 20, 40 or 80) JWT, AUDITOR
DELETE /api/library/link-queue/{appid}/{otherAppid} (dismiss; 204, 404 PAIR_NOT_OPEN) JWT, AUDITOR
POST /api/library/games/{appid}/link-requests ({otherAppid}, both in the caller's library; 204, 400 SAME_GAME, 404 GAME_NOT_FOUND, 409 ALREADY_LINKED / PAIR_DISMISSED, 429 TOO_MANY_REQUESTS) JWT
PUT /api/library/games/{appid}/status (status only, keeps rating, notes, priority) JWT
GET /api/library/questions, /api/library/questions/count ({results: [{appid, name, imageUrl, question, status, playtimeMinutes, normallyMinutes, lastActivity}]}, {count}) JWT
POST /api/library/questions/{appid} ({question, yes, rating?}; 200 {next}, 400 INVALID_ANSWER, 404 GAME_NOT_FOUND, 409 QUESTION_GONE) JWT
PUT /api/library/games/{appid}/finished-hint ({dismissed: ENDING|PLAYTIME}, "Not yet"; 204) JWT
PUT /api/library/games/{appid}/hidden ({hidden}; Steam and manual games) JWT
POST /api/library/imports ({source: TEXT|CSV|SEARCH|GALAXY|GROUVEE|HLTB|PLAYNITE|BOOKMARKLET, rows: [{title, platform, playtimeMinutes, status, igdbId}]}, 1–1000 rows; 202 {id} in QUEUED, 409 IMPORT_RUNNING, 503 IGDB_OFF) JWT
GET /api/library/imports ({imports}, the last 20), /api/library/imports/{id} (status, progress counts, ahead while it waits in a queue, attempts; the rows with their candidates while READY) JWT
POST /api/library/imports/{id}/save ({rows: [{lineNo, igdbId, platform, status, playtimeMinutes}]}; 202, then poll until SAVED; 409 IMPORT_NOT_READY) JWT
POST /api/library/imports/{id}/retry (a FAILED import; 202, 409 IMPORT_NOT_FAILED / IMPORT_RUNNING) JWT
POST /api/library/psn-account ({npsso}; 200 {enabled, linked, linkedAt, expiresAt, reviewImportId} and a sync starts, 400 NPSSO_INVALID, 502 PSN_ERROR, 503 PSN_OFF) JWT
GET, DELETE /api/library/psn-account ({enabled, linked, linkedAt, expiresAt, reviewImportId}: the PSN import waiting for review, else null), unlink (204, deletes the stored token) JWT
GET, DELETE /api/library/imports/{id}/undo-check ({games, changed}), /api/library/imports/{id}/games (undo; 204, 409 IMPORT_NOT_SAVED) JWT
POST /api/library/sync JWT
GET /api/library/games/{appid} (detail; slice 3: achievements[] also hold trophies with kind STEAM|PSN, setId, trophyType, group, rarity; new achievementSets[] = {id, kind, platform, name, total, unlocked, perfect, visible, listed}, Steam first, then PS5 before PS4; hasAchievements = any set; the top-level counts stay Steam only) JWT
POST /api/library/broken-images ({appid, kind}, kind COVER or ACHIEVEMENT_ICONS; 204) JWT
GET /api/library/backlog?limit=&released= ({games} in the user's order; limit 1-20, none = all) JWT
PUT /api/library/backlog ({appids} = the whole order; 204) JWT
GET /api/library/picker?maxMinutes=&mood=&exclude=&category= ({categories:[{source, games}]}; each game has a reason object; category returns that row only) JWT
GET /api/library/stats?groups= (OVERVIEW, PILE, TOP_GAMES, TASTE, HABITS, FINISHED, TREND, PLATFORMS; none = all) JWT
GET /api/library/stats/games?key=&arg=&page= (the games behind a stats number; 400 BAD_ARG) JWT
GET /api/library/achievements/summary (Steam achievements and PlayStation trophies; one entry per game, platinum = perfect) JWT
GET /api/library/sync-status, /summary, /activity, /wishlist, /steam-profile, /games-count (canSync: a Steam ID or a PlayStation link; sources: the sources of the user's games) JWT
GET /api/public/profiles/library profile ticket
GET /api/public/profiles/library/stats/games?key=&arg=&page= (403 NOT_SHARED, 400 BAD_ARG) profile ticket
GET /api/public/profiles/rewards ({equipped}, the worn looks) profile ticket
GET /api/stats/public (+ games per platform, top genres, finished per year) public
GET /api/stats/owners?appid=&key=&arg=&page= (who owns a game in the set behind a community number, key default OWNED: listed display names only, listedTotal, privateCount, includesYou, not cached, 400 BAD_ARG) login
GET /api/stats/public/games?key=&arg=&page= (the games behind a landing number, per game libraries and minutes; 400 BAD_ARG) public
GET /api/follows ({following: [{id, displayName}], followers: [{id, displayName}]}, id is the follow row; a follower with a private profile has displayName null) login
PUT, DELETE /api/follows/by-name/{displayName} (follow, unfollow; 204; PUT: 404 PROFILE_NOT_FOUND for an unknown, private, blocked or deleted name, 400 FOLLOW_SELF, 400 FOLLOW_LIMIT) login
DELETE /api/follows/{id} (unfollow, or remove a follower; only rows of the caller change; 204) login
GET /api/inbox ({results: [{id, kind, displayName, newGames, achievements, statusChanges, wishlistAdds, createdAt, read}], unread}, newest 50, kind DIGEST or VISIBILITY) login
PUT /api/inbox/read (marks all read; 204) login
POST, PUT /internal/accounts/deleted ({userIds}; POST adds, PUT replaces the whole set; 204) service token
POST, PUT /internal/accounts/states ({accounts: [{userId, steamId, displayName, blocked, roleChangedAt, listed, publicProfile, profileSections}]}, displayName, roleChangedAt, listed and publicProfile optional (listed: Auth lists the user by name, kept in listed_user, null keeps it; publicProfile: the profile is public, kept as a row in public_user with profileSections, null keeps row and sections, false ends the follows of the user with one VISIBILITY note each; profileSections null = all shown), displayName at most 100 characters, kept in account_name for the audit log, roleChangedAt kept in account_role_change; POST updates, PUT replaces the whole set; 204) service token
POST /internal/accounts/purge ({userIds}), /internal/accounts/prune ({knownUserIds}, not empty) service token
GET /api/version ({commit, builtAt} of the running image, from the CI build; dev locally) public

Tech stack

  • Java 25, Spring Boot 4.1 (Web MVC, Security, Data JPA, Validation, Cache with Caffeine), virtual threads
  • PostgreSQL 17, Flyway migrations
  • Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com
  • JJWT with Ed25519 public key
  • Spring Boot Actuator + Micrometer (Prometheus)
  • Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit
  • CI: Forgejo Actions, multi-stage Docker image

Architecture

A modular monolith. Each module under core_service.modules is a vertical slice that could move to its own service later.

flowchart LR
    Browser["Frontend in the browser"]
    subgraph CoreService["ToBePlayed-Core"]
        Security["configuration.security<br/>JwtAuthenticationFilter"]
        subgraph Library["library"]
            LibApi["api"] --> LibService["service"] --> LibRepo["repo"]
        end
        Stats["stats"]
        Version["version"]
        Account["account"]
        Follow["follow"]
        Inbox["inbox"]
        Rewards["rewards"]
        Steam["steam_adapter"]
        Igdb["igdb_adapter"]
        Speedrun["speedrun_adapter"]
        Psn["psn_adapter"]
        LibService --> Steam
        LibService --> Igdb
        LibService --> Speedrun
        LibService --> Psn
    end
    Browser -->|"tbp_access cookie"| Security
    Rewards --> Db
    Security -->|"JWKS by kid"| Auth["ToBePlayed-Auth"]
    Steam --> SteamApi["Steam Web API"]
    Steam -->|"HEAD"| SteamCdn["Steam CDN"]
    Igdb --> IgdbApi["IGDB API"]
    Igdb -->|"OAuth token"| Twitch["Twitch"]
    Speedrun --> SpeedrunApi["speedrun.com API"]
    Psn -->|"NPSSO sign-in, played titles, trophy sets and lists"| PsnApi["PlayStation API (unofficial)"]
    LibRepo -->|"own tables; picker also follow, public_user, account_name, blocked_user, deleted_user; inbox_note (PSN expiry notes)"| Db[("PostgreSQL tbp_core")]
    Stats -->|"listed_user_game, counted_user_game, game_mood, coming_game, user_game_status_change, deleted_user, blocked_user, listed_user, account_name, game_alias"| Db
    Follow -->|"follow, follow_digest, inbox_note, public_user, sync_change, user_game_status_change, account_name"| Db
    Inbox --> Db
    Auth -->|"service token<br/>/internal/accounts"| Account
    Account -->|"per-user rows, deleted_user, blocked_user, listed_user, public_user, follow, account_name, user_seen_at"| Db
Module Content
library Sync (Steam and PSN), backlog, manual games, imports, the PlayStation link, playtime history, wishlist, achievements, public profile data
stats Public numbers for the landing page
version GET /api/version: commit and build time of the running image
follow Follows (follow), the digest cursor per followed user (follow_digest) and FollowDigestService: one note per follower and batch, from sync_change and the hand rows of user_game_status_change
inbox inbox_note: the notes of a user, GET /api/inbox, the 90-day purge
account Driven by Auth: tombstones in deleted_user, purge and orphan prune of the per-user rows; blocks in blocked_user, listing in listed_user (users Auth shows by name in owner lists, #26), public profiles and their sections in public_user (#27), role changes in account_role_change, display names in account_name, Steam ID changes stop the nightly sync
steam_adapter Steam Web API client, availability check of Steam CDN pictures
igdb_adapter IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search
speedrun_adapter speedrun.com client: game search by name, verified runs
psn_adapter PlayStation client (unofficial): NPSSO to tokens, refresh, played titles, trophy sets and lists

follow and inbox read account_name, public_user, blocked_user, deleted_user, sync_result, sync_change, user_game and user_game_status_change in SQL; the sync tells follow through shared.event.LibraryActivityEvent.

One shared schema. All modules use one database tbp_core, and some read or write another module's tables in SQL, which ArchUnit cannot see: stats reads listed_user, account_name, game_alias, listed_user_game, counted_user_game, steam_game, game_mood, coming_game, user_game_status_change, deleted_user and blocked_user; account deletes the per-user rows of the library tables (AccountCleanupRepository; a new library table with user_id goes into its list) and writes user_seen_at and last_synced_at of user_steam_sync (AccountStateRepository); library reads deleted_user and blocked_user, and for the picker follow, public_user and account_name; rewards reads user_game_set (a set pays once). A module that moves to its own service takes these reads along as calls. The listed rule ("still owned, not hidden") is UserGameRepository.LISTED for JPQL and the view listed_user_game for native SQL; V24 says what a migration of user_game must do with the view. The numbers that count games once per linked set read counted_user_game (V28), built on listed_user_game: a migration that changes a column of user_game drops this view first, then listed_user_game.

Inside a module: api (controllers, DTOs), service (business logic), domain (JPA entities) and repo (repositories). src/test/java/core_service/architecture/ArchitectureTest.java checks these rules on every build:

  • Modules do not depend on each other. Only library may use steam_adapter, igdb_adapter, speedrun_adapter and psn_adapter.
  • No cycles between modules, shared and configuration.
  • api never uses repo, and repo never uses api.
  • domain never uses api, service, repo or an adapter: the import stores IGDB hits as the own record IgdbGame, so a change of the adapter's IgdbApiService.Game cannot break stored imports.
  • service never uses api, except the adapters' response DTOs and a fixed list of 8 services that still build their module's response DTOs. The list only gets shorter.
  • Every class in repo ends with Repository.
  • shared never uses modules, and modules never use configuration.
  • A *Service class in a module's service package has @Service. A *Controller class in api has @RestController, @ControllerAdvice or @RestControllerAdvice.

Sync flow

  1. GET /api/library/games, /games-count and /picker call LibrarySyncService.syncIfStale. It starts a sync when the last one is older than 24 hours, older than 15 minutes after a failure, or was for another Steam ID. POST /api/library/sync starts one at once, but at most every 60 seconds (else 429 with Retry-After).
  2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A request without a steamId claim (the user unlinked the Steam account in Auth) takes the user out of it. Afterwards it writes every user's pile size of the day to backlog_snapshot, also for users without Steam, and dates finished games by the unlock of their ending achievement (FinishedEstimateService, user_game.finished_at_estimate).
  3. LibrarySyncWorker runs on the background pool. user_steam_sync lets only one run per user start, and none for a user in deleted_user or blocked_user.
  4. The worker loads the wishlist and the owned games from Steam. It updates steam_game and user_game, adds the playtime growth of the day to user_game_playtime_day, and soft-deletes Steam games the user no longer owns. The growth counts at most Steam's playtime of the last two weeks, so a game seen for the first time brings no old playtime, and one game gets at most 24 hours per day. A BACKLOG game with new playtime becomes PLAYING, unless the user chose its status. When Steam returns no games, the library stays as it is.
  5. TimeToBeatService and MoodService ask IGDB for missing or old time-to-beat, mood and genre data, for Steam games and manual games. Hidden games only get their playtime updated.
  6. After the commit, LibrarySyncedEvent starts AchievementSyncService and SpeedrunService (backlog games only) in the background. The achievement sync also fetches Steam's global unlock rates with each schema, and sets FINISHED for a game whose ending achievement is unlocked while the user never chose its status.

The PSN step (PsnSyncStep, then PsnTrophyStep, see "PlayStation link") runs after the Steam fetch and before the transaction; its ADDED, PLAYTIME and trophy ACHIEVEMENTS changes join the same sync_result.

The frontend polls GET /api/library/sync-status until the run ends.

Followers (#27): when the achievement run ends, AchievementSyncService publishes LibraryActivityEvent, and FollowDigestService writes one inbox note per follower from the new sync result (ADDED, ACHIEVEMENTS, STATUS and WISHLIST rows of sync_change) and the hand changes since the cursor. Kinds that the profile hides (public_user.sections) are left out, and the cursor moves anyway. A job every 10 minutes reports the hand changes alone once a user was quiet for 10 minutes; it never takes a sync result, which may still wait for its achievements.

sequenceDiagram
    participant W as LibrarySyncWorker
    participant P as PsnSyncStep + PsnTrophyStep
    participant A as AchievementSyncService
    participant F as FollowDigestService
    participant D as Db
    W->>P: run (titles, trophy summaries, changed lists)
    P-->>W: ADDED, PLAYTIME, ACHIEVEMENTS (trophy gains)
    W->>D: sync_result, sync_change (ADDED, STATUS, WISHLIST, PSN changes)
    W-->>A: LibrarySyncedEvent
    A->>D: sync_change (ACHIEVEMENTS)
    A-->>F: LibraryActivityEvent
    F->>D: lock follow_digest, count sync + hand changes
    F->>D: inbox_note per follower (public, not blocked, not deleted)
    F->>D: move cursor

At 01:30 ImageRepairService repairs the pictures reported through POST /api/library/broken-images (table broken_image, one row per game and kind). Reports only count for Steam games the user owns.

Background work

Background work runs on two pools (AsyncConfig; every @Async names one, ArchUnit checks it):

  • The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with a warning, and the next view or sync asks again. Known limit: a dropped sync still answers 202; the user starts it again.
  • The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page views' threads. A task is small (a user or a few appids), and none is dropped.

Imports have their own two threads, and game_import is their queue. Scheduled jobs (nightly sync, image repair, import watchdog) run on the scheduler.

flowchart LR
    View["page view, add, merge"] -->|"stale games only"| Pool["default pool<br/>4 threads, queue 100"]
    Sync["sync request"] --> Pool
    Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
    View -->|"speedrun"| FollowUp
    Pool --> Apis["Steam, IGDB, speedrun.com"]
    FollowUp --> Apis
    Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
    Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"]

Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks live in memory: a second instance would ask Steam and IGDB twice, and the database would stay consistent (sync claim, upserts). A restart drops what waits in the pools; it is all refresh work and is asked again. Imports survive a restart.

Corrections by an auditor

An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag, genres and time to beat for every user (game_correction, V20). The value is also written into the cached row (steam_game, game_speedrun, game_mood), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name), the image repair (image), the IGDB add (name, image), the mood, speedrun and time to beat upserts (SQL). SteamGame has no setter for name and image: applyCorrection is the only Java writer, a refresh uses the guarded updates in SteamGameRepository. "Back to source" (DELETE) writes back the source value from the first correction and lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP. The time to beat is one field for both times (normallyMinutes, completelyMinutes, each empty or 1 to 60000, completely not below normally) written into game_time_to_beat, so pile hours and stats use it; the IGDB hastily time is not correctable. Every correction and every reset writes admin_audit (GAME_CORRECT, GAME_CORRECTION_RESET). GET /api/library/catalog lets an auditor find and correct a game they do not own. Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the source value once (its guard sees the state from before the save); the auditor saves again. save(new SteamGame(...)) on a stored key also overwrites a corrected name; the sync and the IGDB add do it only after findById found no row.

Merging duplicates

One game can have two keys: a Steam appid and minus its IGDB id, or two IGDB ids. An AUDITOR merges them for every user (GameMergeService, V21). The key off Steam always goes into the other one; two Steam appids are refused, since the sync would bring both back. In one transaction:

  • A user with only the old row: the row moves to the new key with its status history (FK ON UPDATE CASCADE).
  • A user with both rows: one row stays on the new key. Status, rating, notes and priority come from the row whose current status the user chose (both: the one chosen last); a gap is filled from the other row. Source, platform and playtime stay, unless the Steam copy is no longer owned. Both status histories stay; the merge itself logs no status change.
  • The shared rows of the old key (catalog row, moods, speedrun, time to beat, corrections, broken images) are deleted. So are its queue pairs, its requests and its game_edition row; its set row goes to the kept key (see "Editions of one game").
  • game_alias points the old key to the new one (kept flat), so a later add of the old IGDB game lands on the new key. admin_audit gets one GAME_MERGE row: who, both keys, both names and how many rows moved or were combined.

Merges made since V29 can be undone for 30 days (POST /api/library/merges/{mergeId}/undo, AUDITOR). The merge saves a copy in its own transaction: game_merge_undo (the deleted shared rows as JSON per table, the link set after the merge, the flattened aliases) and game_merge_undo_user (per user: both user_game rows as JSON, the history ids and the reward ledger ids the merge moved). The copy holds notes, so it goes with the account. A nightly job (app.library.merge-undo-purge-cron) deletes copies older than 30 days; older merges answer 410.

stateDiagram-v2
    [*] --> MERGED: POST /games/{appid}/merge (copy saved)
    MERGED --> UNDONE: POST /merges/{id}/undo (30 days, newest first)
    MERGED --> FINAL: nightly purge after 30 days

The undo, in one transaction (the newest merge of a kept key goes first, else 409 NEWER_MERGE):

  • Shared rows (catalog row, moods, speedrun, time to beat, corrections, broken images, game_edition) come back; game_alias of the old key is removed and the aliases the merge flattened point to the old key again.
  • A user with only the old row: the row moves back if it is still on the new key, still added by hand (no Steam takeover) and the user has no old row. It keeps its current status, rating and notes.
  • A user with both rows: both rows are back if the kept row still exists. If the user did not change status, rating, notes or priority of the kept row since the merge (compared with a copy taken right after the merge), each row gets its own old values. If they did, both rows get the current values. The kept row gets its Steam ownership back when the merge had replaced it and nothing took the row over since. The old status history goes back to the old key; the log entries the undo itself causes are removed.
  • Ledger rows the merge moved go back to the old key (so does what a moved user earned on the row after the merge), only when the user has the old row again. Nothing is paid twice. A game the user hid after the merge stays hidden on the restored rows.
  • Links: when the kept key's set is still as the merge left it, the old sets come back with their set_ids; if links changed since, they stay and the old key comes back unlinked (the audit detail says "links kept").
  • Users who deleted their account are not restored. admin_audit gets a GAME_MERGE_UNDO row; the GAME_MERGE row stays.

GET /api/library/games/{appid} follows game_alias when the user has no row on the asked key, and answers with the kept appid (the frontend then replaces the URL). After an undo the alias is gone. The game search shows games already in ToBePlayed first, with their catalog name and image, and games other users added by hand that IGDB did not return; inCatalog marks both (only those can be merged).

Editions of one game

IGDB tells which library keys are one game: game_edition (V28) holds a "same game" key per library key, refreshed like game_mood (stale check, nightly sync, add, detail view; app.library.edition-ttl-days). The key is version_parent for an edition, else parent_game when game_type is 9 (remaster) or 11 (port), else the game's own IGDB id; remakes (8) keep their own id. null when IGDB does not know the game.

  • game_link(appid, set_id): the keys that are one game, for every user. A set is every row with one set_id. An unlink takes one key out; a set of one is dropped. A merge puts the kept key in the merged key's set; when both have a set, the sets join.
  • Auto link (V42): GameEditionService.linkSameGames links every pair of keys with the same same_game_id, after each editions refresh and once at start for the games already there (linkExisting). No auditor, so no audit row. A pair in DISMISSED (an auditor dismissal or an unlink) is never linked again, also not through a third edition: a link is skipped when a dismissed pair lies between the two sets. The link is global: a wrong IGDB key groups the game for every user until an auditor unlinks it.
  • Split (DELETE /api/library/games/{appid}/set): user_game_split(user_id, appid) makes the key stand alone for this user, only while another listed edition is in the user's set. No sync or link touches it, so it stays split. The view user_game_set(user_id, appid, set_id) is game_link for the user's rows minus the split keys; every reader of a user's set (counted_user_game, the editions of the detail, the source filter, the questions, rewards) reads it. There is no undo of a split yet.
  • One status per set (V42 triggers, like V14, so no code path can forget it): a change of status, status_set_by_user, rating, notes or priority on one row is copied to the other rows of the user's set, also hidden and removed ones (user_game_set_share; pg_trigger_depth() stops the echo). When rows join a set (a new row, a new link, a merge of sets), align_user_game_set applies the join rule: the winner is the counted edition (listed first, then status rank, playtime, older first_seen_at, lower appid); its status wins, status_set_by_user is true if any row has it, rating and priority fall back to the first row that has one, and the distinct notes are joined with a blank line, cut to 2000. A row that changes is first copied to user_game_set_backup (the first copy wins; kept until the account is deleted). The status log of a join is no hand change (by_user false), and a row that takes a finished status takes the set's first finish time, so "finished per year" does not move.
  • game_link_pair is the auditor queue (OPEN, DISMISSED, LINKED). IGDB pairs are inserted when the queue is read. A row in any state keeps the pair out for good. An unlink dismisses the pairs of that key with the former members. game_link_request holds who asked for a pair (the queue shows only the count). A user asks for a pair of two games of their own library; at most 20 open requests per user (a per-user advisory lock makes the cap hold for parallel calls), a dismissed pair answers 409 PAIR_DISMISSED.
  • Pile, stats, snapshots and the dashboard count a set once (counted_user_game, see "Stats"); the library filter hideDuplicates=true hides the lesser editions. Pile Points pay a set once (see "Pile Points"), and the MONTH bonus reads backlog_snapshot, which counts a set once.
  • The game detail has linked and editions (the user's other listed games of the set); a game row has editions (size of the set in the user's library, null below 2).
flowchart LR
    Edition["game_edition (IGDB same_game_id)"] -->|"linkSameGames"| Link["game_link (global sets)"]
    Auditor["auditor link / unlink"] --> Link
    Link --> Set["user_game_set"]
    Split["user_game_split (user splits)"] -->|"minus"| Set
    Set --> Counted["counted_user_game"]
    Set -->|"join: align_user_game_set"| Row["user_game (status, rating, notes, priority)"]
    Row -->|"update: user_game_set_share"| Row
    Row -.->|"old values"| Backup["user_game_set_backup"]
    Set --> Rewards["rewards: a set pays once"]

Deleted accounts

Auth drives it (see Auth's readme). POST /internal/accounts/deleted adds a tombstone in deleted_user: the user gets no more syncs and leaves the landing stats and the pile snapshots at once. Auth's nightly job sends the whole flagged set (PUT), then purge for users flagged 30 days ago (only tombstoned ones), then prune with all its user ids, which removes the rows of users Auth does not know (refused for more than app.account-cleanup.max-orphans, default 5). app.account-cleanup.dry-run (default true) only counts the rows. The tombstone also ends the user's follows in both directions without a note and removes the user's public_user and follow_digest rows; the purge removes the user's inbox notes and the notes about the user. The purge also removes the user's account_name and listed_user rows, link requests (game_link_request), splits (user_game_split) and set backups (user_game_set_backup). The tombstone also deletes the user's stored PlayStation token (user_psn_account) at once.

Blocked users and Steam ID changes

Auth sends a user's state (POST /internal/accounts/states) after a Steam link, an unlink, a Steam ID another account took by a verified link, a display name change, a block, an unblock and a change of the owner-list opt-in (listed, #26), of the public profile or of the profile tiles (publicProfile, profileSections, #27); its nightly job sends every user (PUT). A blocked user is in blocked_user: no nightly sync, no new sync run, not in the landing stats. JwtAuthenticationFilter refuses a blocked user's token and a token issued before the user's last role change (account_role_change, sent as roleChangedAt), through the cache loginStates (30 s, evicted by every state call). An Auth that does not send roleChangedAt changes only the blocks. A new Steam ID or no Steam ID takes the user out of the nightly sync; the next visit syncs the ID of the token (Core's copy changes only then, so that sync sees the change and drops the old account's games). A visit with an access token that still has the old ID (max 15 min) can sync it once more and put the user back on the nightly list; the nightly PUT at 04:15 takes the user off again, so at most one nightly sync (23:55) runs with the old ID. Unblock and the nightly PUT lift the block.

Manual games next to Steam games

A game the user adds by hand comes from IGDB (ManualGameService). Steam games and manual games share steam_game and user_game and one key, the column appid (see GameKeys):

  • appid > 0: the Steam appid. A manual game gets it when IGDB knows the game on Steam.
  • appid < 0: minus the IGDB id, for a game that is not on Steam. Steam appids are always positive, so the two never meet.

user_game.source (GameSource) says who owns the row. The sync never changes or removes a MANUAL row, with one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes STEAM, and status, rating and notes stay. PSN rows (a PlayStation import, see below) live the same way. Only MANUAL and PSN rows can be edited (PUT /api/library/games/{appid}/manual) or deleted (DELETE /api/library/games/{appid}). Time to beat uses the Steam appid for positive keys and the IGDB id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two (see "Merging duplicates").

Imports

Many games at once (V22, GameImportService): a list, a CSV or several picks of the game search become one game_import with a game_import_row per line. game_import is also the queue of GameImportWorker, which has two queues, each on one thread for all users:

  • matching: IGDB searches with typo help. One thread, so an import never takes the whole IGDB budget from the interactive search; other users' searches may wait for the limiter meanwhile. Up to 5 candidates per line are stored whole (JSONB), so the save needs no IGDB call, and exact marks a best one with the line's name.
  • saving: adds the chosen lines the way the manual add does (same keys and duplicate check) with user_game.import_id in the same transaction, then loads their time to beat and moods. A save never waits behind another user's big matching.

Each line is stored when done, so a stopped try goes on where it stopped. A worker sets progress_at when it takes an import and after every line. A watchdog (every minute) stops a try without progress for 5 minutes (app.library.import-stall-minutes): the import goes to the end of its queue and the queue gets a new thread. A restart counts as a stopped try too. After 3 tries the import is FAILED; the user can try it again. One import per user can be in the matching queue (a unique partial index).

stateDiagram-v2
    [*] --> QUEUED: POST /imports
    QUEUED --> MATCHING: matching worker takes it
    MATCHING --> READY: every line matched
    READY --> SAVING: POST /save (the review)
    SAVING --> SAVED: games added, time to beat and moods loaded
    SAVED --> UNDONE: DELETE /games (undo)
    MATCHING --> QUEUED: no progress for 5 min or restart
    SAVING --> SAVING: no progress for 5 min or restart (end of the queue)
    MATCHING --> FAILED: 3rd stopped try
    SAVING --> FAILED: 3rd stopped try
    FAILED --> QUEUED: POST /retry (stopped while matching)
    FAILED --> SAVING: POST /retry (stopped while saving)
    FAILED --> UNDONE: DELETE /games (the games added before it failed)

The undo deletes the games of the import that are still MANUAL or PSN (a game the Steam sync took over stays). A removed row the import brings back gets no import_id: it holds rating, notes and history from before, so the undo keeps it; undo-check counts those with a rating, notes, priority or a status change after the save.

modules/psn_adapter calls PlayStation's unofficial API (the public client of the PlayStation app, as the psn-api project ships it; all values in app.psn.*). PsnAccountService in library holds the use cases:

  • Link (POST /api/library/psn-account): the NPSSO (the user's npsso cookie from playstation.com) goes to /authorize (the redirect is not followed: Redirect.NEVER), the code in its Location header to /token. Only the refresh token is stored, in user_psn_account, sealed with AES-256-GCM (SecretBox, key APP_PSN_TOKEN_KEY, base64 of 32 bytes). The NPSSO, the code and the access token are never stored; no log line, answer or toString of a token type holds a secret. No key = the feature is off (enabled: false); a key of the wrong length stops Core at start. A changed key makes stored links unreadable: the sync skips PlayStation and the user links again.
  • Expiry: the refresh token lives about 10 days (refresh_expires_at). A refresh that returns the same refresh token keeps the stored expiry; a new one stores its own. The Settings page shows the date. The inbox gets one note PSN_EXPIRING when the link ends within app.psn.expiry-warning-days (2) and one note PSN_EXPIRED when it has ended or PlayStation refuses the refresh token (also an unreadable token). user_psn_account.expiry_note keeps what was written, so a note comes once per link; a new link resets it. The library module writes inbox_note by SQL, as the follow module does. An ended link makes the sync skip PlayStation until the user pastes a new NPSSO.
  • Sync: linking starts a sync (LibrarySyncService.startAfterLink). Inside the force-sync cooldown or while a sync runs, one sync per user is scheduled for the end of the cooldown (in memory, lost on restart; the next login sync covers it). After that the PSN step (PsnSyncStep) runs in every library sync (login, nightly, manual) after the Steam fetch. One refresh call, then gamelist/v2 (200 per page, at most 1000 titles). PsnTitles cleans the names (no ™/®, no PS4/PS5 suffix) and joins the PS4 and PS5 versions of one concept.id into one line (name and platform of the most played version, playtime summed, newest lastPlayedDateTime). user_psn_title remembers each title by its psn_key (c<conceptId>, else t<titleId>) as LINKED (with the library key) or DISMISSED:
    • A new title (not in the table, not in an open PSN import) is searched in IGDB. Same name and a version on the line's platform: ManualGameService.addGame as PSN, no review, an ADDED change. The same game under another spelling or platform that the user owns is only linked. Everything else goes to the review: a READY PSN import with the matched rows is created, or the rows are appended to the one that waits (reviewImportId). The review page is the import page; saving a PSN import links the checked lines and dismisses the unchecked ones for good.
    • A LINKED title updates its row (source = PSN, not removed): playtime, last_played_at, PLAYTIME changes and the playtime history; playtime_2weeks is the sum of the history of the last 14 days. A game the user deleted stays LINKED and is not added again. A game also owned on Steam stays one row (the Steam row wins).
    • The first run of a user (no user_psn_title yet) is a baseline like the first Steam sync: its adds are a count in sync_result.imported_games, not a list. PSN changes share the sync_result of the Steam part, so the banner and the follow digest need no change.
    • PlayStation and IGDB are called outside the sync transaction; the step catches every error, so PlayStation never fails the Steam part. A user with a PlayStation link and no Steam ID syncs too (user_steam_sync.steam_id is nullable): the Steam part is skipped, the same result, banner and digest apply. The client cannot start a PSN import with its own rows.
  • Limits: one limiter psnApi for the whole server (1 request / 4 s, taken per HTTP request). Titles bought but never started are not listed by PlayStation. A PSN row pays no rewards (no Steam proof), and a merge undo does not move it back. The nightly loop is sequential.
  • Trophies (V40, PsnTrophyStep, called at the end of PsnSyncStep.run, outside the sync transaction, never throws). It reuses the access token of the title part (FetchResult.Titles hides it in toString). Own tables psn_trophy_set, psn_trophy, user_psn_trophy_set, user_psn_trophy and the cache psn_title_trophy_set, so the Steam tables and the rewards (Steam only) stay untouched.
    • Summary: 1-2 calls trophy/v1/users/me/trophyTitles give every set with defined and earned counts, trophySetVersion and lastUpdatedDateTime. Hidden (hiddenFlag) sets, PS3/Vita-only sets and sets of unknown or DISMISSED titles are not stored. A summary row is saved for every mapped set. Gains (ACHIEVEMENTS changes) come from the earned total only; the first time a set is stored is a baseline. Only a complete summary (sets returned = the API's totalItemCount) removes the user's stored sets that are no longer in it (another account was linked); a partial one, after a broken page, removes nothing. Unlink deletes all of the user's trophy rows.
    • Mapping, two paths, the log line PSN trophies user=… mapped by title id=N by name=M shows which one worked. (1) Exact: titles/trophyTitles?npTitleIds= (5 ids per call) answers title id to set; cached globally, "no set" is asked again after 30 days. Not confirmed against the real service: a 4xx or an empty answer falls through to (2) Name + platform, same account only: the clean name equals that of exactly one linked title and its platform is in the set's platform list. Two games with the same name: the set stays unmapped. The set counts for the merge target of the game.
    • Lists only when changed: the global trophy list when the stored list_version differs from trophySetVersion; the user's earned list (also gives the rarity) when list_updated_at differs from lastUpdatedDateTime; a set with 0 earned needs no earned call.
    • Budget app.psn.trophy-calls-per-run (30, about 2 min at 1 request / 4 s; the sync counts as stale after 5 min) covers title id lookups and lists. Newest sets first; the rest follow in later syncs (the detail page then says the list loads with the next sync). Three errors in a row stop the list calls, the summaries are still saved.
    • Presentation: perfect() = platinum or all earned; one entry per set, one per game for the counts (perfect set, else best percent). A hidden trophy that is not earned has no name and no description. Stored image URLs must be https://. A sync costs 2 calls (token, titles) plus the summary and the changed lists.

Run locally

Local development without keys (profile local)

The whole stack runs on your machine with no API keys and no calls to Steam, Twitch/IGDB or speedrun.com. Needs Java 25, Node 24 and Docker. Clone ToBePlayed-Auth, ToBePlayed-Core and ToBePlayed-Frontend next to each other, then start them in this order (Core also starts without Auth and finds it later):

  1. Auth: in ToBePlayed-Auth, ./mvnw spring-boot:run -Dspring-boot.run.profiles=local (IDE: active profile local). Port 8080.
  2. Core: in ToBePlayed-Core, ./mvnw spring-boot:run -Dspring-boot.run.profiles=local. Port 8090.
  3. Frontend: in ToBePlayed-Frontend, npm install, then npm run dev, and open http://localhost:5173 (localhost, not 127.0.0.1, so the browser sends the login cookies to every local port). The development values must be VITE_API_URL=http://localhost:8080 and VITE_CORE_API_URL=http://localhost:8090.
  4. Log in with dev@localhost / password123. On the dashboard, start a Steam sync: the library fills with 8 games, achievements, time to beat, moods, a wishlist and one speedrun game from recorded answers.

What the local profile does:

  • Spring Boot's Docker Compose support starts each service's compose.yaml: PostgreSQL for Auth (port 5432) and for Core (port 5433), and in Core a WireMock container (port 8089, compose profile local) that answers in place of Steam, Twitch/IGDB and speedrun.com from ToBePlayed-Core/local/wiremock.
  • Auth makes a new signing key at each start (LocalKeysEnvironmentPostProcessor, only in this profile), so no key is in the repo. After an Auth restart (also a devtools restart), log in again.
  • Never use local on a server: it adds an admin with a public password. Auth refuses local together with prod.
  • Auth adds the dev user (Flyway callback in db/local): role PLATFORM_ADMIN, so user, auditor and admin pages can all be tried ("View as" in the menu switches down), and a linked Steam ID that the recorded Steam answers belong to. The invite code for more users is local.
  • Core checks the login with Auth's local JWKS, like in production.
  • The browser still loads game pictures from the Steam and IGDB image servers; offline they stay gray.
  • "Sign in through Steam" still opens the real Steam page (no key needed). The dev user is already linked.
  • A request to WireMock that has no recording answers 404 and shows up in Core's log.
  • PlayStation: paste local-npsso in the PlayStation card of the settings (any other value is refused). The link starts a sync: Ghost of Yōtei and Mega Man Legacy Collection match exactly and are added by themselves. Ghost of Tsushima (PS4 and PS5 version as one line, "DIRECTOR'S CUT" in the name) and Monster Hunter Wilds (IGDB's recording does not know it) wait in the import review (/import). The local profile has a fixed token key. The trophy stubs (Ghost of Yōtei PS5 by title id, Mega Man PS4 by name, one PS3 and one hidden set) are fake NPWR9… ids in local/wiremock.
  • To see an IGDB link suggestion: add Hollow Knight by hand (game search, PS5). It shares IGDB's "same game" key with the Steam copy, so the link queue (auditor) lists the pair with the IGDB badge.

To change the recorded answers, edit the JSON in local/wiremock/mappings (which request) and local/wiremock/__files (the answer), then restart Core (it restarts the container). LocalMocksParseTest checks that every answer still fits the adapters' response types. http://localhost:8089/__admin/requests lists what Core asked.

With the real services (profile dev)

Needs Java 25, Docker and your own keys.

./mvnw spring-boot:run -Dspring-boot.run.profiles=dev

Spring Boot's Docker Compose support starts PostgreSQL from compose.yaml, Flyway applies the migrations, and the service listens on http://localhost:8090. Health: /actuator/health, metrics: /actuator/prometheus.

In the dev profile a request without an Authorization header gets a dummy user, so the API is usable without a running Auth service.

Configuration comes from environment variables:

Variable Purpose
JWT_JWKS_URL Auth's /.well-known/jwks.json
JWT_PUBLIC_KEY Ed25519 public key (Base64 of the X.509 DER bytes), used until the first JWKS load. Set this, JWT_JWKS_URL, or both
STEAM_API_KEY Steam Web API key
IGDB_CLIENT_ID, IGDB_CLIENT_SECRET Twitch app for IGDB (optional)
APP_PSN_TOKEN_KEY Key for the stored PlayStation tokens, openssl rand -base64 32 (optional; empty = PlayStation link off)
DB_PASSWORD Local PostgreSQL password

Tests

./mvnw clean verify

Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers. In integration tests background work runs on the calling thread, so a test reads the result at once. AbstractMockedAdaptersTest mocks Steam, IGDB and speedrun.com, AbstractMockedLibraryServicesTest the sync and the refreshes of the picker and stats pages; each is one shared Spring context. Race tests wait until the other connection waits for the row lock (awaitLockWaiter), not for a fixed time.

Roadmap

  • Sale alerts for wishlist games.
  • Move the adapter DTOs out of api, then turn on strict module independence in ArchUnit and drop the adapter exception of the "service never uses api" rule.

License

All rights reserved. The source is public for reading only.

S
Description
No description provided
Readme
2.1 MiB
0 Stars 3 Watchers 0 Forks
Languages
Java 99.4%
PLpgSQL 0.5%