lobo a389bdd0c7
Build, Test & Push Core Service / quality-check (push) Successful in 49s
Build, Test & Push Core Service / docker (push) Successful in 1m20s
fix(sync): stop the nightly sync when the Steam ID is gone
After an unlink in Auth, or when a verified link took the ID away, the
nightly job kept syncing the old Steam ID for up to 14 days. A
games-count request without a steamId claim now clears user_seen_at.
steam_id stays, so a later link with another ID still soft-deletes the
old games.
2026-09-26 11:20:04 +02:00

To Be Played – Core Service

Business service of To Be Played, a Steam backlog tracker. It syncs the user's Steam library, stores the backlog (status, rating, notes, priority), tracks playtime per day, syncs the wishlist and adds time-to-beat data from IGDB.

To Be Played is split into three repositories:

Repository Role
ToBePlayed-Auth Users, login, settings, signs JWTs
ToBePlayed-Core (this one) Steam library, backlog, wishlist, stats; verifies JWTs
ToBePlayed-Frontend React single-page app

Features

  • Steam sync: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly sync. Resilience4j retries and rate-limits every Steam call; Caffeine caches game counts and player summaries.
  • Backlog: status (BACKLOG, PLAYING, FINISHED, COMPLETED = 100 %, DROPPED), rating 1–5, notes, priority, filters and sorting. "Pile of Shame" summary with a random suggestion of what to play.
  • Picker: up to 10 random backlog games for a time budget (IGDB normal time to beat) and a mood, HIGH priority first, then MEDIUM, then the rest. Games already shown can be excluded for a reroll. Moods STORY, COOP, CHILL, ACTION, SCARY come from IGDB genres, themes and game modes; SPEEDRUN from speedrun.com.
  • Speedrun games: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns the speedrun.com link as speedrunUrl. speedrun.com has no Steam or IGDB link and no key, so one queue works for all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days.
  • Achievements: after every library sync a background job fetches the achievement schema of each owned game and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5 games closest to 100 %.
  • Playtime history: one row per game and day, for the activity view and the "most played in the last two weeks" dashboard banner.
  • Wishlist: synced from Steam with store price and discount.
  • Time to beat: story, extras and completionist hours from the IGDB API. Optional: without credentials the feature is off and the rest works.
  • Public profile data: totals, status counts, games in progress and finished games with rating, perfect games and average completion. Readable only with a profile ticket issued by Auth. Never returns notes, priority or the Steam ID.

Authentication

Core never issues tokens. It verifies Auth's EdDSA (Ed25519) JWTs with public keys only. It loads them from Auth's JWKS by the token's kid and loads again when a new kid shows up (at most every 10 seconds). Loaded keys stay until Core restarts. JWT_PUBLIC_KEY is used only until the first JWKS load works. The token comes from Auth's tbp_access cookie or a Bearer header, whichever verifies first. Core reads userId and steamId from the claims. POST, PUT and DELETE need an X-TBP-Client or Authorization header (CSRF guard), and CORS allows only the frontend origin. A token with a typ claim (for example a profile ticket) is rejected on normal endpoints; the profile ticket is accepted only by GET /api/public/profiles/library in the X-Profile-Ticket header.

API

Method Path Auth
GET /api/library/games, /api/library/games/{appid} JWT
PUT /api/library/games/{appid}/backlog JWT
PUT /api/library/games/{appid}/status (status only, keeps rating, notes, priority) JWT
POST /api/library/sync JWT
GET /api/library/picker?maxMinutes=&mood=&count=&exclude= JWT
GET /api/library/achievements/summary JWT
GET /api/library/sync-status, /summary, /activity, /wishlist, /steam-profile, /games-count JWT
GET /api/public/profiles/library profile ticket
GET /api/stats/public public

Tech stack

  • Java 25, Spring Boot 4.1 (Web MVC, Security, Data JPA, Validation, Cache with Caffeine), virtual threads
  • PostgreSQL 17, Flyway migrations
  • Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com
  • JJWT with Ed25519 public key
  • Spring Boot Actuator + Micrometer (Prometheus)
  • Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit
  • CI: Forgejo Actions, multi-stage Docker image

Architecture

A modular monolith. Each module under core_service.modules is a vertical slice that could move to its own service later.

flowchart LR
    Browser["Frontend in the browser"]
    subgraph CoreService["ToBePlayed-Core"]
        Security["configuration.security<br/>JwtAuthenticationFilter"]
        subgraph Library["library"]
            LibApi["api"] --> LibService["service"] --> LibRepo["repo"]
        end
        Stats["stats"]
        Steam["steam_adapter"]
        Igdb["igdb_adapter"]
        Speedrun["speedrun_adapter"]
        LibService --> Steam
        LibService --> Igdb
        LibService --> Speedrun
    end
    Browser -->|"tbp_access cookie"| Security
    Security -->|"JWKS by kid"| Auth["ToBePlayed-Auth"]
    Steam --> SteamApi["Steam Web API"]
    Igdb --> IgdbApi["IGDB API"]
    Igdb -->|"OAuth token"| Twitch["Twitch"]
    Speedrun --> SpeedrunApi["speedrun.com API"]
    LibRepo --> Db[("PostgreSQL tbp_core")]
    Stats --> Db
Module Content
library Sync, backlog, manual games, playtime history, wishlist, achievements, public profile data
stats Public numbers for the landing page
steam_adapter Steam Web API client
igdb_adapter IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search
speedrun_adapter speedrun.com client: game search by name, verified runs

Inside a module: api (controllers, DTOs), service (business logic), domain (JPA entities) and repo (repositories). src/test/java/core_service/architecture/ArchitectureTest.java checks these rules on every build:

  • Modules do not depend on each other. Only library may use steam_adapter, igdb_adapter and speedrun_adapter.
  • No cycles between modules, shared and configuration.
  • api never uses repo, and repo never uses api.
  • Every class in repo ends with Repository.
  • shared never uses modules, and modules never use configuration.
  • A *Service class in a module's service package has @Service. A *Controller class in api has @RestController, @ControllerAdvice or @RestControllerAdvice.

Sync flow

  1. GET /api/library/games, /games-count and /picker call LibrarySyncService.syncIfStale. It starts a sync when the last one is older than 24 hours, older than 15 minutes after a failure, or was for another Steam ID. POST /api/library/sync starts one at once, but at most every 60 seconds (else 429 with Retry-After).
  2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A request without a steamId claim (the user unlinked the Steam account in Auth) takes the user out of it.
  3. LibrarySyncWorker runs in the background. user_steam_sync lets only one run per user start.
  4. The worker loads the wishlist and the owned games from Steam. It updates steam_game and user_game, adds the playtime growth of the day to user_game_playtime_day, and soft-deletes Steam games the user no longer owns. When Steam returns no games, the library stays as it is.
  5. TimeToBeatService and MoodService ask IGDB for missing or old time-to-beat and mood data, for Steam games and manual games.
  6. After the commit, LibrarySyncedEvent starts AchievementSyncService and SpeedrunService (backlog games only) in the background.

The frontend polls GET /api/library/sync-status until the run ends.

Manual games next to Steam games

A game the user adds by hand comes from IGDB (ManualGameService). Steam games and manual games share steam_game and user_game and one key, the column appid (see GameKeys):

  • appid > 0: the Steam appid. A manual game gets it when IGDB knows the game on Steam.
  • appid < 0: minus the IGDB id, for a game that is not on Steam. Steam appids are always positive, so the two never meet.

user_game.source (GameSource) says who owns the row. The sync never changes or removes a MANUAL row, with one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes STEAM, and status, rating and notes stay. Only MANUAL rows can be edited (PUT /api/library/games/{appid}/manual) or deleted (DELETE /api/library/games/{appid}). Time to beat uses the Steam appid for positive keys and the IGDB id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get a second row when IGDB learns its Steam appid later and the user owns it on Steam.

Run locally

Needs Java 25 and Docker.

./mvnw spring-boot:run -Dspring-boot.run.profiles=dev

Spring Boot's Docker Compose support starts PostgreSQL from compose.yaml, Flyway applies the migrations, and the service listens on http://localhost:8090. Health: /actuator/health, metrics: /actuator/prometheus.

In the dev profile a request without an Authorization header gets a dummy user, so the API is usable without a running Auth service.

Configuration comes from environment variables:

Variable Purpose
JWT_JWKS_URL Auth's /.well-known/jwks.json
JWT_PUBLIC_KEY Ed25519 public key (Base64 of the X.509 DER bytes), used until the first JWKS load. Set this, JWT_JWKS_URL, or both
STEAM_API_KEY Steam Web API key
IGDB_CLIENT_ID, IGDB_CLIENT_SECRET Twitch app for IGDB (optional)
DB_PASSWORD Local PostgreSQL password

Tests

./mvnw clean verify

Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers.

Roadmap

  • Sale alerts for wishlist games.
  • Move steam_adapter DTOs out of api, then turn on strict module independence in ArchUnit.

License

All rights reserved. The source is public for reading only.

S
Description
No description provided
Readme
2.1 MiB
0 Stars 3 Watchers 0 Forks
Languages
Java 99.4%
PLpgSQL 0.5%