ToBePlayed-Core/readme.md

To Be Played – Core Service

Business service of To Be Played, a Steam backlog tracker. It syncs the user's Steam library, stores the backlog (status, rating, notes, priority), tracks playtime per day, syncs the wishlist and adds time-to-beat data from IGDB.

To Be Played is split into three repositories:

Repository	Role
ToBePlayed-Auth	Users, login, settings, signs JWTs
ToBePlayed-Core (this one)	Steam library, backlog, wishlist, stats; verifies JWTs
ToBePlayed-Frontend	React single-page app

Features

• Steam sync: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly sync. Resilience4j retries and rate-limits every Steam call; Caffeine caches game counts and player summaries.
• Backlog: status (BACKLOG, PLAYING, FINISHED, COMPLETED = 100 %, DROPPED), rating 1–5, notes, priority, filters and sorting. "Pile of Shame" summary with a random suggestion of what to play.
• Picker: up to 10 random backlog games for a time budget (IGDB normal time to beat) and a mood, HIGH priority first, then MEDIUM, then the rest. Games already shown can be excluded for a reroll. Moods STORY, COOP, CHILL, ACTION, SCARY come from IGDB genres, themes and game modes; SPEEDRUN from speedrun.com.
• Speedrun games: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns the speedrun.com link as speedrunUrl. speedrun.com has no Steam or IGDB link and no key, so one queue works for all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days.
• Achievements: after every library sync a background job fetches the achievement schema of each owned game and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5 games closest to 100 %.
• Playtime history: one row per game and day, for the activity view and the "most played in the last two weeks" dashboard banner.
• Finished from achievements or playtime: EndingAchievement finds the one achievement of a game that means "story end" (English text rule, Steam's global unlock rate between 1 and 80 %, the easiest variant wins, more than 3 candidates = a game without one end). When the user unlocks it, never chose the status and never answered "Not yet", the achievement sync sets FINISHED, but only for a certain ending: its text says so and no other achievement of the game could be the end. A status the user chose is never changed; such games get a hint on the game page instead. A second, weaker hint compares the playtime with IGDB's normal time to beat (at least 90 %), only for games IGDB does not mark as endless (sandbox, survival, 4X, simulator, MMO). "Not yet" hides a hint per game; a dismissed playtime hint still lets the ending hint through.
• Hide games: a user can hide a Steam game (e.g. a playtest or a tool). It leaves every list and number: library, picker, Pile of Shame, activity, achievement summary, public profile and landing page stats. The sync still updates its playtime, but no achievements, time to beat, moods or speedrun data are fetched for it. The library lists hidden games with hidden=true.
• Stats: UserStatsService builds the numbers of the stats page per group, so every page asks only for the groups it shows: totals and statuses; the pile (time to beat, weeks "at your pace" of the last 90 days, years of 40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the status history); the Pile of Shame trend (90 days of backlog_snapshot, plus "this month: added / done", where the first sync of a Steam account is no "added", see user_steam_sync.history_since). Every number counts only listed games.
• Status history: a database trigger logs every status change of a library game in user_game_status_change (old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older changes are not known.
• Wishlist: synced from Steam with store price and discount.
• Broken image repair: the frontend reports a Steam cover or achievement icon that does not load. At 01:30 (Europe/Berlin) a job checks up to 200 reports. A cover that loads again is left alone; else the store's hashed header picture (IStoreBrowseService/GetItems with assets) goes to steam_game.image_url. When a few achievement icons do not load, the schema comes fresh from Steam; an empty answer keeps the stored one. A report that cannot be repaired is tried again on the next report, at most 3 times, then after 30 days.
• Time to beat: story, extras and completionist hours from the IGDB API. Optional: without credentials the feature is off and the rest works.
• Public profile data: totals, status counts, games in progress and finished games with rating, perfect games and average completion. Readable only with a profile ticket issued by Auth. Never returns notes, priority or the Steam ID. stats holds only the stat groups the user shares (the ticket's profileStats claim, set in Auth); none by default. A section the user hides (the ticket's sections claim: SUMMARY, STATUS, PLAYING, FINISHED, STATS) is null and runs no query; a ticket without the claim shows all.

Authentication

Core never issues tokens. It verifies Auth's EdDSA (Ed25519) JWTs with public keys only. It loads them from Auth's JWKS by the token's kid and loads again when a new kid shows up (at most every 10 seconds). Loaded keys stay until Core restarts. JWT_PUBLIC_KEY is used only until the first JWKS load works. The token comes from Auth's tbp_access cookie or a Bearer header, whichever verifies first. Core reads userId, steamId and the active role (missing or unknown = USER) from the claims and grants that role and every lower one (USER < AUDITOR < PLATFORM_ADMIN). POST, PUT and DELETE need an X-TBP-Client or Authorization header (CSRF guard), and CORS allows only the frontend origin. A token with a typ claim (for example a profile ticket) is rejected on normal endpoints; the profile ticket is accepted only by GET /api/public/profiles/library in the X-Profile-Ticket header. /internal/** has its own filter chain: it accepts only a service token of Auth (typ=service, with exp, sent as Bearer), never a login, also not in the dev profile. Traefik exposes the path, so the signed token is its only guard.

Every service method a controller calls carries one role marker, on the method or on its class (the method wins): @RequiresRole(Role.X), @Public (no login, e.g. landing stats; the public profile checks its ticket) or @Internal (only Auth's service token). A Spring Security method interceptor checks it while a request is served; scheduled jobs and @Async work run without a user and are not checked. A denied call answers 401 without a login, else 403. The URL rules in SecurityConfig stay as the outer guard. ArchUnit fails the build for an unmarked method, two markers, a controller calling an @Async method (the check would never run), and a service method calling another service's method that needs more.

API

Method	Path	Auth
GET	/api/library/games, /api/library/games/{appid}	JWT
PUT	/api/library/games/{appid}/backlog	JWT
PUT, DELETE	/api/library/games/{appid}/corrections/{field} ({value}; name, image, speedrun, moods, endless, genres; 204, 400 INVALID_CORRECTION)	JWT, AUDITOR
POST	/api/library/games/{appid}/merge ({otherAppid}; 200 {appid} = the key that stays, 400 SAME_GAME / STEAM_GAMES, 404 GAME_NOT_FOUND)	JWT, AUDITOR
PUT	/api/library/games/{appid}/status (status only, keeps rating, notes, priority)	JWT
PUT	/api/library/games/{appid}/finished-hint ({dismissed: ENDING|PLAYTIME}, "Not yet"; 204)	JWT
PUT	/api/library/games/{appid}/hidden ({hidden}; 409 MANUAL_GAME for a game added by hand)	JWT
POST	/api/library/sync	JWT
POST	/api/library/broken-images ({appid, kind}, kind COVER or ACHIEVEMENT_ICONS; 204)	JWT
GET	/api/library/picker?maxMinutes=&mood=&count=&exclude=	JWT
GET	/api/library/stats?groups= (OVERVIEW, PILE, TOP_GAMES, TASTE, HABITS, FINISHED, TREND; none = all)	JWT
GET	/api/library/achievements/summary	JWT
GET	/api/library/sync-status, /summary, /activity, /wishlist, /steam-profile, /games-count	JWT
GET	/api/public/profiles/library	profile ticket
GET	/api/stats/public	public
POST, PUT	/internal/accounts/deleted ({userIds}; POST adds, PUT replaces the whole set; 204)	service token
POST	/internal/accounts/purge ({userIds}), /internal/accounts/prune ({knownUserIds}, not empty)	service token
GET	/api/version ({commit, builtAt} of the running image, from the CI build; dev locally)	public

Tech stack

• Java 25, Spring Boot 4.1 (Web MVC, Security, Data JPA, Validation, Cache with Caffeine), virtual threads
• PostgreSQL 17, Flyway migrations
• Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com
• JJWT with Ed25519 public key
• Spring Boot Actuator + Micrometer (Prometheus)
• Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit
• CI: Forgejo Actions, multi-stage Docker image

Architecture

A modular monolith. Each module under core_service.modules is a vertical slice that could move to its own service later.

flowchart LR
    Browser["Frontend in the browser"]
    subgraph CoreService["ToBePlayed-Core"]
        Security["configuration.security<br/>JwtAuthenticationFilter"]
        subgraph Library["library"]
            LibApi["api"] --> LibService["service"] --> LibRepo["repo"]
        end
        Stats["stats"]
        Version["version"]
        Account["account"]
        Steam["steam_adapter"]
        Igdb["igdb_adapter"]
        Speedrun["speedrun_adapter"]
        LibService --> Steam
        LibService --> Igdb
        LibService --> Speedrun
    end
    Browser -->|"tbp_access cookie"| Security
    Security -->|"JWKS by kid"| Auth["ToBePlayed-Auth"]
    Steam --> SteamApi["Steam Web API"]
    Steam -->|"HEAD"| SteamCdn["Steam CDN"]
    Igdb --> IgdbApi["IGDB API"]
    Igdb -->|"OAuth token"| Twitch["Twitch"]
    Speedrun --> SpeedrunApi["speedrun.com API"]
    LibRepo --> Db[("PostgreSQL tbp_core")]
    Stats --> Db
    Auth -->|"service token<br/>/internal/accounts"| Account
    Account --> Db

Module	Content
library	Sync, backlog, manual games, playtime history, wishlist, achievements, public profile data
stats	Public numbers for the landing page
version	GET /api/version: commit and build time of the running image
account	Deleted accounts, driven by Auth: tombstones in deleted_user, purge and orphan prune of the per-user rows
steam_adapter	Steam Web API client, availability check of Steam CDN pictures
igdb_adapter	IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search
speedrun_adapter	speedrun.com client: game search by name, verified runs

Inside a module: api (controllers, DTOs), service (business logic), domain (JPA entities) and repo (repositories). src/test/java/core_service/architecture/ArchitectureTest.java checks these rules on every build:

• Modules do not depend on each other. Only library may use steam_adapter, igdb_adapter and speedrun_adapter.
• No cycles between modules, shared and configuration.
• api never uses repo, and repo never uses api.
• Every class in repo ends with Repository.
• shared never uses modules, and modules never use configuration.
• A *Service class in a module's service package has @Service. A *Controller class in api has @RestController, @ControllerAdvice or @RestControllerAdvice.

Sync flow

1. GET /api/library/games, /games-count and /picker call LibrarySyncService.syncIfStale. It starts a sync when the last one is older than 24 hours, older than 15 minutes after a failure, or was for another Steam ID. POST /api/library/sync starts one at once, but at most every 60 seconds (else 429 with Retry-After).
2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A request without a steamId claim (the user unlinked the Steam account in Auth) takes the user out of it. Afterwards it writes every user's pile size of the day to backlog_snapshot, also for users without Steam, and dates finished games by the unlock of their ending achievement (FinishedEstimateService, user_game.finished_at_estimate).
3. LibrarySyncWorker runs in the background. user_steam_sync lets only one run per user start, and none for a user in deleted_user.
4. The worker loads the wishlist and the owned games from Steam. It updates steam_game and user_game, adds the playtime growth of the day to user_game_playtime_day, and soft-deletes Steam games the user no longer owns. The growth counts at most Steam's playtime of the last two weeks, so a game seen for the first time brings no old playtime, and one game gets at most 24 hours per day. A BACKLOG game with new playtime becomes PLAYING, unless the user chose its status. When Steam returns no games, the library stays as it is.
5. TimeToBeatService and MoodService ask IGDB for missing or old time-to-beat, mood and genre data, for Steam games and manual games. Hidden games only get their playtime updated.
6. After the commit, LibrarySyncedEvent starts AchievementSyncService and SpeedrunService (backlog games only) in the background. The achievement sync also fetches Steam's global unlock rates with each schema, and sets FINISHED for a game whose ending achievement is unlocked while the user never chose its status.

The frontend polls GET /api/library/sync-status until the run ends.

At 01:30 ImageRepairService repairs the pictures reported through POST /api/library/broken-images (table broken_image, one row per game and kind). Reports only count for Steam games the user owns.

Corrections by an auditor

An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag and genres for every user (game_correction, V20). The value is also written into the cached row (steam_game, game_speedrun, game_mood), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name), the image repair (image), the IGDB add (name, image), the mood and speedrun upserts (SQL). A new writer of these fields must do the same. "Back to source" (DELETE) writes back the source value from the first correction and lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP. Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the source value once (its guard sees the state from before the save); the auditor saves again.

Merging duplicates

One game can have two keys: a Steam appid and minus its IGDB id, or two IGDB ids. An AUDITOR merges them for every user (GameMergeService, V21). The key off Steam always goes into the other one; two Steam appids are refused, since the sync would bring both back. In one transaction:

• A user with only the old row: the row moves to the new key with its status history (FK ON UPDATE CASCADE).
• A user with both rows: one row stays on the new key. Status, rating, notes and priority come from the row whose current status the user chose (both: the one chosen last); a gap is filled from the other row. Source, platform and playtime stay, unless the Steam copy is no longer owned. Both status histories stay; the merge itself logs no status change.
• The shared rows of the old key (catalog row, moods, speedrun, time to beat, corrections, broken images) are deleted.
• game_alias points the old key to the new one (kept flat), so a later add of the old IGDB game lands on the new key. admin_audit gets one GAME_MERGE row: who, both keys, both names and how many rows moved or were combined.

There is no undo. The game search shows games already in ToBePlayed first, with their catalog name and image, and games other users added by hand that IGDB did not return; inCatalog marks both (only those can be merged).

Deleted accounts

Auth drives it (see Auth's readme). POST /internal/accounts/deleted adds a tombstone in deleted_user: the user gets no more syncs and leaves the landing stats and the pile snapshots at once. Auth's nightly job sends the whole flagged set (PUT), then purge for users flagged 30 days ago (only tombstoned ones), then prune with all its user ids, which removes the rows of users Auth does not know (refused for more than app.account-cleanup.max-orphans, default 5). app.account-cleanup.dry-run (default true) only counts the rows.

Manual games next to Steam games

A game the user adds by hand comes from IGDB (ManualGameService). Steam games and manual games share steam_game and user_game and one key, the column appid (see GameKeys):

• appid > 0: the Steam appid. A manual game gets it when IGDB knows the game on Steam.
• appid < 0: minus the IGDB id, for a game that is not on Steam. Steam appids are always positive, so the two never meet.

user_game.source (GameSource) says who owns the row. The sync never changes or removes a MANUAL row, with one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes STEAM, and status, rating and notes stay. Only MANUAL rows can be edited (PUT /api/library/games/{appid}/manual) or deleted (DELETE /api/library/games/{appid}). Time to beat uses the Steam appid for positive keys and the IGDB id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two (see "Merging duplicates").

Run locally

Local development without keys (profile local)

The whole stack runs on your machine with no API keys and no calls to Steam, Twitch/IGDB or speedrun.com. Needs Java 25, Node 24 and Docker. Clone ToBePlayed-Auth, ToBePlayed-Core and ToBePlayed-Frontend next to each other, then start them in this order (Core also starts without Auth and finds it later):

1. Auth: in ToBePlayed-Auth, ./mvnw spring-boot:run -Dspring-boot.run.profiles=local (IDE: active profile local). Port 8080.
2. Core: in ToBePlayed-Core, ./mvnw spring-boot:run -Dspring-boot.run.profiles=local. Port 8090.
3. Frontend: in ToBePlayed-Frontend, npm install, then npm run dev, and open http://localhost:5173 (localhost, not 127.0.0.1, so the browser sends the login cookies to every local port). The development values must be VITE_API_URL=http://localhost:8080 and VITE_CORE_API_URL=http://localhost:8090.
4. Log in with dev@localhost / password123. On the dashboard, start a Steam sync: the library fills with 8 games, achievements, time to beat, moods, a wishlist and one speedrun game from recorded answers.

What the local profile does:

• Spring Boot's Docker Compose support starts each service's compose.yaml: PostgreSQL for Auth (port 5432) and for Core (port 5433), and in Core a WireMock container (port 8089, compose profile local) that answers in place of Steam, Twitch/IGDB and speedrun.com from ToBePlayed-Core/local/wiremock.
• Auth makes a new signing key at each start (LocalKeysEnvironmentPostProcessor, only in this profile), so no key is in the repo. After an Auth restart (also a devtools restart), log in again.
• Never use local on a server: it adds an admin with a public password. Auth refuses local together with prod.
• Auth adds the dev user (Flyway callback in db/local): role PLATFORM_ADMIN, so user, auditor and admin pages can all be tried ("View as" in the menu switches down), and a linked Steam ID that the recorded Steam answers belong to. The invite code for more users is local.
• Core checks the login with Auth's local JWKS, like in production.
• The browser still loads game pictures from the Steam and IGDB image servers; offline they stay gray.
• "Sign in through Steam" still opens the real Steam page (no key needed). The dev user is already linked.
• A request to WireMock that has no recording answers 404 and shows up in Core's log.

To change the recorded answers, edit the JSON in local/wiremock/mappings (which request) and local/wiremock/__files (the answer), then restart Core (it restarts the container). LocalMocksParseTest checks that every answer still fits the adapters' response types. http://localhost:8089/__admin/requests lists what Core asked.

With the real services (profile dev)

Needs Java 25, Docker and your own keys.

./mvnw spring-boot:run -Dspring-boot.run.profiles=dev

Spring Boot's Docker Compose support starts PostgreSQL from compose.yaml, Flyway applies the migrations, and the service listens on http://localhost:8090. Health: /actuator/health, metrics: /actuator/prometheus.

In the dev profile a request without an Authorization header gets a dummy user, so the API is usable without a running Auth service.

Configuration comes from environment variables:

Variable	Purpose
JWT_JWKS_URL	Auth's /.well-known/jwks.json
JWT_PUBLIC_KEY	Ed25519 public key (Base64 of the X.509 DER bytes), used until the first JWKS load. Set this, JWT_JWKS_URL, or both
STEAM_API_KEY	Steam Web API key
IGDB_CLIENT_ID, IGDB_CLIENT_SECRET	Twitch app for IGDB (optional)
DB_PASSWORD	Local PostgreSQL password

Tests

./mvnw clean verify

Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers.

Roadmap

• Sale alerts for wishlist games.
• Move steam_adapter DTOs out of api, then turn on strict module independence in ArchUnit.

License

All rights reserved. The source is public for reading only.