Compare commits

..
24 Commits
Author SHA1 Message Date
lobo e11fae8e31 feat(follow): add player search with rate limit (#57)
Build, Test & Push Core Service / quality-check (push) Successful in 2m19s
Build, Test & Push Core Service / docker (push) Successful in 1m34s
Find public profiles by name: GET /api/follows/players?q=.
Queries under 2 characters find nothing; more than 30 searches
per minute get 429. LIKE escaping is shared with the library search.
2026-10-21 14:54:48 +02:00
lobo c42526bc16 feat(library): sort by name, last played and time to beat (#56)
Sort by name ignores upper and lower case.
Games never played, or without a time to beat, come last.
2026-10-21 11:09:44 +02:00
lobo 4fbf572f32 perf(datasource): reuse query plans on core connections (#49)
Build, Test & Push Core Service / quality-check (push) Successful in 2m11s
Build, Test & Push Core Service / docker (push) Successful in 1m29s
Planning was 94% of Dashboard DB time, and prod never reused a plan.
Force generic plans and prepare on first use. Burst: 443 -> 229 ms.
2026-10-20 11:41:31 +02:00
lobo 6f2d7b4885 fix(library): book psn play on its last played day (#53)
Build, Test & Push Core Service / quality-check (push) Successful in 2m35s
Build, Test & Push Core Service / docker (push) Successful in 1m29s
A newly linked PSN title booked its whole history on the sync day.
Now only new play counts, on its last played day. A day is capped
at 24 h. V49 deletes the wrong old day rows.
2026-10-20 09:06:53 +02:00
lobo d3c9c7558c feat(follow): add feed link data and game endpoints (#52, #51)
Build, Test & Push Core Service / quality-check (push) Successful in 2m34s
Build, Test & Push Core Service / docker (push) Successful in 1m52s
Feed notes get profileName and appid; follows get lastActivityAt.
New GET /api/library/games/{appid}/preview: catalog data, any game.
New GET /api/public/profiles/library/games/{appid}: one profile game.
2026-10-14 16:23:11 +02:00
lobo ec66bd5ead chore: remove unused dev compose and prometheus dependency
Build, Test & Push Core Service / quality-check (push) Successful in 3m9s
Build, Test & Push Core Service / docker (push) Successful in 2m4s
Prometheus is not in use. Only /actuator/health is exposed.
2026-10-14 15:22:20 +02:00
lobo 54946ef0eb docs(readme): sync features, endpoints and config with code
Build, Test & Push Core Service / quality-check (push) Successful in 2m38s
Build, Test & Push Core Service / docker (push) Successful in 2m3s
The readme had drifted from the code. Update sync rules, thread pools,
API tables, env vars and the CI setup.
2026-10-14 14:51:40 +02:00
lobo dba4fedfa1 test(wiremock): add 30-game library and 6 achievement sets
Build, Test & Push Core Service / quality-check (push) Successful in 2m32s
Build, Test & Push Core Service / docker (push) Successful in 1m32s
Local WireMock library grows from 8 to 30 games.
Six games get schema, achievement and percentage stubs.
LocalMocksParseTest checks each stub appid is an owned game.
2026-10-13 12:03:56 +02:00
lobo cdfbb4a863 fix(follow): keep sync status rows for the sync digest
A hand-only digest used to move the cursor past all status rows.
It now stops at the last by_user change, so the sync digest still
gets its status detail lines.
2026-10-13 11:15:10 +02:00
lobo c6bfea1976 feat(inbox): add details for digest notes
Digest notes now keep the lines behind their counts in
inbox_note_detail (V48). GET /api/inbox/{id}/details shows them
while the reader still follows the subject and it is public.
2026-10-13 09:17:41 +02:00
lobo 03469f9eb5 perf(library): add dashboard-games and stored game detail
Build, Test & Push Core Service / quality-check (push) Successful in 2m26s
Build, Test & Push Core Service / docker (push) Successful in 1m32s
Add GET /api/library/dashboard-games for the top and playing tiles
in one call. It has no count query.
GET /api/library/games/{appid}?refresh=false answers from stored
data and sets achievementsStale. It does not wait for Steam.
entryPlatforms reads the view once in a materialized CTE:
planning ~33 ms -> 15 ms.
Test CORS Access-Control-Max-Age 3600 on preflight.

Refs #47
2026-10-12 09:25:49 +02:00
lobo b1aaec0615 fix(speedrun): pause queue on 429/420 instead of dropping it
speedrun.com returns 429 or 420 when we ask too fast. These were
counted as errors: 3 in a row dropped both queues. Now the thread
pauses and retries the same game. Rate: 1 request per 2 s. Refs #48
2026-10-12 08:25:43 +02:00
lobo 9efed0ab1d feat(speedrun): match speedrun.com games only when sure
Build, Test & Push Core Service / quality-check (push) Successful in 2m32s
Build, Test & Push Core Service / docker (push) Successful in 1m29s
Loose name matches gave wrong links. Now a link needs one sure hit:
same name and within 1 year of IGDB, or one hit when no year is known.
V47 marks old matches (not corrected ones) for one re-check. Refs #8
2026-10-11 17:23:32 +02:00
lobo bb195a1f79 test(library): compare trend move times as Instants
Sort the parsed Instant values, not the raw strings. String order
is wrong when fraction lengths differ. Refs #45.
2026-10-11 16:46:03 +02:00
lobo 0f56f5bb0f feat(library): add ALL_UNLOCKED sync change for last achievement
Build, Test & Push Core Service / quality-check (push) Successful in 2m30s
Build, Test & Push Core Service / docker (push) Successful in 1m35s
A sync that unlocks the last achievement of a game not COMPLETED
adds an ALL_UNLOCKED change. The overview hides it once the game is
COMPLETED, hidden or removed. Refs #21
2026-10-11 16:22:49 +02:00
lobo 1de8acef2e fix(library): merge manual game into its Steam row on sync
Steam sync added a second row for a game the user added by hand (#9).
The manual row moves to the Steam appid and keeps its data, such as
status, notes, sets and PSN links. Manual rows on other platforms stay
and join the Steam game's set.
2026-10-11 14:55:09 +02:00
lobo 59d6b3394d fix(rewards): check last achievement unlock for manual COMPLETED
Refs #44. Check the last achievement unlock, not the first status change.
Old 100% games no longer pass. New reason COMPLETION_TOO_OLD.
Add earlySince to the games and wallet GET responses.
2026-10-11 13:51:34 +02:00
lobo 1a9a4e912c fix(rewards): let manual finish open COMPLETED for Steam edition
A paid FINISHED_MANUAL on a set edition blocked COMPLETED for the
Steam game forever. It now opens COMPLETED, but only when the Steam
game's first finish is on or after app.rewards.early-since. Refs #35
2026-10-11 13:07:26 +02:00
lobo 6215bb0dec fix(library): count trend never played by set total
Nightly snapshot and live trend point use the set total, as stats do.
Newest point now equals stats "never started". Old rows keep old value.
Refs #43
2026-10-11 12:11:47 +02:00
lobo 8f884040d8 fix(stats): count each set once by its set playtime
Played, never started, under one hour and oldest unplayed used the
counted edition's own minutes, so a set played only on PS5 showed as
never started. They and their drill-down lists now use the set total
(user_game_entry_playtime). Refs #38. Nightly snapshot (#43) unchanged.
2026-10-11 11:52:45 +02:00
lobo 0550f3fe9c feat(configuration): add request timing log
Build, Test & Push Core Service / quality-check (push) Successful in 2m28s
Build, Test & Push Core Service / docker (push) Successful in 1m34s
One INFO line per API request: method, handler pattern, status, ms.
No ids or query strings in the line. Refs #40.
2026-10-11 11:05:18 +02:00
lobo e18b799501 perf(library): bind owned-games subqueries to user
Subqueries in OWNED_WHERE had no userId filter, so Postgres hashed
rows of all users. Skip the source filter when all sources are set.
Refs #40.
2026-10-11 11:05:18 +02:00
lobo 4d49b0f023 feat(library): soft delete removed games and add restore
Build, Test & Push Core Service / quality-check (push) Successful in 1m46s
Build, Test & Push Core Service / docker (push) Successful in 1m26s
Removed games could not come back (#39). Remove now sets removed_at
and restore clears it. V46 brings back PSN games removed before.
2026-10-11 09:52:11 +02:00
lobo 696e06e7fe fix(test): delete user_game_playtime_day rows in seed
Build, Test & Push Core Service / quality-check (push) Successful in 1m45s
Build, Test & Push Core Service / docker (push) Successful in 1m38s
RewardControllerTest leaves user_game_playtime_day rows for user 1,
affecting paceWeeks calculation. Delete them in seed() to ensure
clean test isolation.
2026-10-09 19:32:35 +02:00
161 changed files with 6989 additions and 438 deletions

No files matched your search

-22
View File
@@ -1,22 +0,0 @@
# docker-compose.dev.yml
services:
# ─────────────────────────────────────────────
# LOCAL DATABASE FOR DEVELOPMENT
# ─────────────────────────────────────────────
tbp-db-local:
image: postgres:17-alpine
container_name: tbp_db_local
restart: unless-stopped
ports:
# Map the container port 5432 to your local machine port 5432
- "5432:5432"
environment:
- POSTGRES_DB=tbp_core
- POSTGRES_USER=tbp_user
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
# Use a local volume to persist data across container restarts
- tbp_local_db_data:/var/lib/postgresql/data
volumes:
tbp_local_db_data:
@@ -0,0 +1,30 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_B1",
"percent": 81.0
},
{
"name": "ACH_B2",
"percent": 57.3
},
{
"name": "ACH_B3",
"percent": 39.8
},
{
"name": "ACH_B4",
"percent": 26.5
},
{
"name": "ACH_B5",
"percent": 23.6
},
{
"name": "ACH_B6",
"percent": 18.2
}
]
}
}
@@ -0,0 +1,38 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_A1",
"percent": 88.2
},
{
"name": "ACH_A2",
"percent": 52.7
},
{
"name": "ACH_A3",
"percent": 74.1
},
{
"name": "ACH_A4",
"percent": 31.4
},
{
"name": "ACH_A5",
"percent": 44.9
},
{
"name": "ACH_A6",
"percent": 36.0
},
{
"name": "ACH_A7",
"percent": 28.3
},
{
"name": "ACH_A8",
"percent": 4.6
}
]
}
}
@@ -0,0 +1,30 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_F1",
"percent": 91.2
},
{
"name": "ACH_F2",
"percent": 64.8
},
{
"name": "ACH_F3",
"percent": 48.5
},
{
"name": "ACH_F4",
"percent": 37.1
},
{
"name": "ACH_F5",
"percent": 22.4
},
{
"name": "ACH_F6",
"percent": 11.0
}
]
}
}
@@ -0,0 +1,30 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_E1",
"percent": 83.4
},
{
"name": "ACH_E2",
"percent": 62.0
},
{
"name": "ACH_E3",
"percent": 44.3
},
{
"name": "ACH_E4",
"percent": 31.9
},
{
"name": "ACH_E5",
"percent": 27.5
},
{
"name": "ACH_E6",
"percent": 9.7
}
]
}
}
@@ -0,0 +1,30 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_C1",
"percent": 78.5
},
{
"name": "ACH_C2",
"percent": 47.2
},
{
"name": "ACH_C3",
"percent": 33.6
},
{
"name": "ACH_C4",
"percent": 35.4
},
{
"name": "ACH_C5",
"percent": 41.0
},
{
"name": "ACH_C6",
"percent": 12.9
}
]
}
}
@@ -0,0 +1,30 @@
{
"achievementpercentages": {
"achievements": [
{
"name": "ACH_D1",
"percent": 70.3
},
{
"name": "ACH_D2",
"percent": 25.8
},
{
"name": "ACH_D3",
"percent": 30.2
},
{
"name": "ACH_D4",
"percent": 15.6
},
{
"name": "ACH_D5",
"percent": 12.1
},
{
"name": "ACH_D6",
"percent": 8.4
}
]
}
}
+205 -7
View File
@@ -1,6 +1,6 @@
{ {
"response": { "response": {
"game_count": 8, "game_count": 30,
"games": [ "games": [
{ {
"appid": 620, "appid": 620,
@@ -8,7 +8,8 @@
"playtime_forever": 1450, "playtime_forever": 1450,
"playtime_2weeks": null, "playtime_2weeks": null,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1787300000
}, },
{ {
"appid": 1145360, "appid": 1145360,
@@ -16,7 +17,8 @@
"playtime_forever": 3200, "playtime_forever": 3200,
"playtime_2weeks": 240, "playtime_2weeks": 240,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1790500000
}, },
{ {
"appid": 413150, "appid": 413150,
@@ -24,7 +26,8 @@
"playtime_forever": 5400, "playtime_forever": 5400,
"playtime_2weeks": null, "playtime_2weeks": null,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1788900000
}, },
{ {
"appid": 427520, "appid": 427520,
@@ -32,7 +35,8 @@
"playtime_forever": 12000, "playtime_forever": 12000,
"playtime_2weeks": 600, "playtime_2weeks": 600,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1790900000
}, },
{ {
"appid": 504230, "appid": 504230,
@@ -40,7 +44,8 @@
"playtime_forever": 800, "playtime_forever": 800,
"playtime_2weeks": null, "playtime_2weeks": null,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1789200000
}, },
{ {
"appid": 367520, "appid": 367520,
@@ -56,7 +61,8 @@
"playtime_forever": 780, "playtime_forever": 780,
"playtime_2weeks": null, "playtime_2weeks": null,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true,
"rtime_last_played": 1786900000
}, },
{ {
"appid": 892970, "appid": 892970,
@@ -64,6 +70,198 @@
"playtime_forever": 90, "playtime_forever": 90,
"playtime_2weeks": 90, "playtime_2weeks": 90,
"img_icon_url": "", "img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790700000
},
{
"appid": 1245620,
"name": "ELDEN RING",
"playtime_forever": 6100,
"playtime_2weeks": 900,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790000000
},
{
"appid": 1091500,
"name": "Cyberpunk 2077",
"playtime_forever": 4300,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1787900000
},
{
"appid": 292030,
"name": "The Witcher 3: Wild Hunt",
"playtime_forever": 5200,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1789800000
},
{
"appid": 1086940,
"name": "Baldur's Gate 3",
"playtime_forever": 2800,
"playtime_2weeks": 300,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790150000
},
{
"appid": 730,
"name": "Counter-Strike 2",
"playtime_forever": 1900,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1788600000
},
{
"appid": 570,
"name": "Dota 2",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 1174180,
"name": "Red Dead Redemption 2",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 105600,
"name": "Terraria",
"playtime_forever": 2600,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1789300000
},
{
"appid": 294100,
"name": "RimWorld",
"playtime_forever": 3900,
"playtime_2weeks": 120,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790300000
},
{
"appid": 632470,
"name": "Disco Elysium - The Final Cut",
"playtime_forever": 1500,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1788100000
},
{
"appid": 548430,
"name": "Deep Rock Galactic",
"playtime_forever": 700,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790000000
},
{
"appid": 646570,
"name": "Slay the Spire",
"playtime_forever": 2200,
"playtime_2weeks": 60,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790450000
},
{
"appid": 2379780,
"name": "Balatro",
"playtime_forever": 1100,
"playtime_2weeks": 420,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790600000
},
{
"appid": 1794680,
"name": "Vampire Survivors",
"playtime_forever": 450,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790700000
},
{
"appid": 268910,
"name": "Cuphead",
"playtime_forever": 900,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1787600000
},
{
"appid": 814380,
"name": "Sekiro: Shadows Die Twice",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 374320,
"name": "DARK SOULS III",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 1057090,
"name": "Ori and the Will of the Wisps",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 275850,
"name": "No Man's Sky",
"playtime_forever": 120,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1790200000
},
{
"appid": 526870,
"name": "Satisfactory",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true
},
{
"appid": 1332010,
"name": "Stray",
"playtime_forever": 330,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true,
"rtime_last_played": 1789000000
},
{
"appid": 108600,
"name": "Project Zomboid",
"playtime_forever": 0,
"playtime_2weeks": null,
"img_icon_url": "",
"has_community_visible_stats": true "has_community_visible_stats": true
} }
] ]
@@ -0,0 +1,39 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "Cyberpunk 2077",
"success": true,
"achievements": [
{
"apiname": "ACH_B1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_B2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_B3",
"achieved": 1,
"unlocktime": 1786518400
},
{
"apiname": "ACH_B4",
"achieved": 1,
"unlocktime": 1786777600
},
{
"apiname": "ACH_B5",
"achieved": 1,
"unlocktime": 1787036800
},
{
"apiname": "ACH_B6",
"achieved": 0,
"unlocktime": 0
}
]
}
}
@@ -0,0 +1,49 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "ELDEN RING",
"success": true,
"achievements": [
{
"apiname": "ACH_A1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_A2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_A3",
"achieved": 1,
"unlocktime": 1786518400
},
{
"apiname": "ACH_A4",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_A5",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_A6",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_A7",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_A8",
"achieved": 0,
"unlocktime": 0
}
]
}
}
@@ -0,0 +1,39 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "Stray",
"success": true,
"achievements": [
{
"apiname": "ACH_F1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_F2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_F3",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_F4",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_F5",
"achieved": 0,
"unlocktime": 0
},
{
"apiname": "ACH_F6",
"achieved": 0,
"unlocktime": 0
}
]
}
}
@@ -0,0 +1,39 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "Cuphead",
"success": true,
"achievements": [
{
"apiname": "ACH_E1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_E2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_E3",
"achieved": 1,
"unlocktime": 1786518400
},
{
"apiname": "ACH_E4",
"achieved": 1,
"unlocktime": 1786777600
},
{
"apiname": "ACH_E5",
"achieved": 1,
"unlocktime": 1787036800
},
{
"apiname": "ACH_E6",
"achieved": 0,
"unlocktime": 0
}
]
}
}
@@ -0,0 +1,39 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "Disco Elysium - The Final Cut",
"success": true,
"achievements": [
{
"apiname": "ACH_C1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_C2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_C3",
"achieved": 1,
"unlocktime": 1786518400
},
{
"apiname": "ACH_C4",
"achieved": 1,
"unlocktime": 1786777600
},
{
"apiname": "ACH_C5",
"achieved": 1,
"unlocktime": 1787036800
},
{
"apiname": "ACH_C6",
"achieved": 0,
"unlocktime": 0
}
]
}
}
@@ -0,0 +1,39 @@
{
"playerstats": {
"steamID": "76561198000000001",
"gameName": "Slay the Spire",
"success": true,
"achievements": [
{
"apiname": "ACH_D1",
"achieved": 1,
"unlocktime": 1786000000
},
{
"apiname": "ACH_D2",
"achieved": 1,
"unlocktime": 1786259200
},
{
"apiname": "ACH_D3",
"achieved": 1,
"unlocktime": 1786518400
},
{
"apiname": "ACH_D4",
"achieved": 1,
"unlocktime": 1786777600
},
{
"apiname": "ACH_D5",
"achieved": 1,
"unlocktime": 1787036800
},
{
"apiname": "ACH_D6",
"achieved": 1,
"unlocktime": 1787296000
}
]
}
}
@@ -0,0 +1,58 @@
{
"game": {
"gameName": "Cyberpunk 2077",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_B1",
"displayName": "Street Kid",
"description": "Get through the opening job.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
},
{
"name": "ACH_B2",
"displayName": "Chrome Fan",
"description": "Install five implants.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
},
{
"name": "ACH_B3",
"displayName": "Fast Driver",
"description": "Win a street race.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
},
{
"name": "ACH_B4",
"displayName": "Netrunner",
"description": "Hack ten enemies in one fight.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
},
{
"name": "ACH_B5",
"displayName": "Final Run",
"description": "Complete the main story.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
},
{
"name": "ACH_B6",
"displayName": "Rich Merc",
"description": "Hold a big pile of eddies.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1091500/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,74 @@
{
"game": {
"gameName": "ELDEN RING",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_A1",
"displayName": "Rune Bearer",
"description": "Pick up your first rune from the grace.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A2",
"displayName": "Margit Down",
"description": "Defeat the warden of the gate.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A3",
"displayName": "Roundtable Guest",
"description": "Visit the hold between worlds.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A4",
"displayName": "Smith Friend",
"description": "Upgrade a weapon to a high level.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A5",
"displayName": "Mounted Hunter",
"description": "Slay a foe from horseback.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A6",
"displayName": "Great Rune",
"description": "Restore a great rune.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A7",
"displayName": "Deep Delver",
"description": "Explore a deep underground city.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
},
{
"name": "ACH_A8",
"displayName": "Lord of Lands",
"description": "Gather every legendary talisman.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1245620/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,58 @@
{
"game": {
"gameName": "Stray",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_F1",
"displayName": "Cat Nap",
"description": "Sleep in a sunny spot.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
},
{
"name": "ACH_F2",
"displayName": "Rooftop Walker",
"description": "Cross the roofs of the old slum.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
},
{
"name": "ACH_F3",
"displayName": "Robot Friend",
"description": "Make a friend of a small robot.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
},
{
"name": "ACH_F4",
"displayName": "Meow Choir",
"description": "Meow in front of ten robots.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
},
{
"name": "ACH_F5",
"displayName": "Fish Lover",
"description": "Find a cold bowl of fish.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
},
{
"name": "ACH_F6",
"displayName": "Secret Paw",
"description": "Open every hidden door.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/1332010/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,58 @@
{
"game": {
"gameName": "Cuphead",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_E1",
"displayName": "Contract Signed",
"description": "Take the deal at the casino.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
},
{
"name": "ACH_E2",
"displayName": "Run and Gun",
"description": "Pass a platform stage.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
},
{
"name": "ACH_E3",
"displayName": "Boss Rush",
"description": "Defeat five bosses.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
},
{
"name": "ACH_E4",
"displayName": "Devil Beaten",
"description": "Complete the main story.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
},
{
"name": "ACH_E5",
"displayName": "Coin Collector",
"description": "Find every coin of a world.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
},
{
"name": "ACH_E6",
"displayName": "Perfect Parry",
"description": "Parry twelve pink shots in a row.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/268910/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,58 @@
{
"game": {
"gameName": "Disco Elysium - The Final Cut",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_C1",
"displayName": "Fresh Case",
"description": "Wake up in the hostel room.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
},
{
"name": "ACH_C2",
"displayName": "Good Listener",
"description": "Talk to every skill voice.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
},
{
"name": "ACH_C3",
"displayName": "Thought Cabinet",
"description": "Fill all slots of the cabinet.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
},
{
"name": "ACH_C4",
"displayName": "Case Closed",
"description": "Finish the main story.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
},
{
"name": "ACH_C5",
"displayName": "Best Friend",
"description": "Win the trust of your partner.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
},
{
"name": "ACH_C6",
"displayName": "Sweet Tunes",
"description": "Hear every song on the radio.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/632470/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,58 @@
{
"game": {
"gameName": "Slay the Spire",
"gameVersion": "1",
"availableGameStats": {
"achievements": [
{
"name": "ACH_D1",
"displayName": "Spire Climber",
"description": "Reach the top of the first act.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
},
{
"name": "ACH_D2",
"displayName": "Card Hoarder",
"description": "Own forty cards in one run.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
},
{
"name": "ACH_D3",
"displayName": "Relic Fan",
"description": "Hold ten relics in one run.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
},
{
"name": "ACH_D4",
"displayName": "Silent Hunter",
"description": "Win a run as the second hero.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
},
{
"name": "ACH_D5",
"displayName": "Cold Heart",
"description": "Win a run as the third hero.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
},
{
"name": "ACH_D6",
"displayName": "Pure Deck",
"description": "Win a run with a tiny deck.",
"hidden": 0,
"icon": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg",
"icongray": "https://cdn.cloudflare.steamstatic.com/steam/apps/646570/capsule_184x69.jpg"
}
]
}
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "1091500"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-1091500.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "1245620"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-1245620.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "1332010"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-1332010.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "268910"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-268910.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "632470"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-632470.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetGlobalAchievementPercentagesForApp/v2/",
"queryParameters": {
"gameid": {
"equalTo": "646570"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-global-percentages-646570.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "1091500"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-1091500.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "1245620"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-1245620.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "1332010"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-1332010.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "268910"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-268910.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "632470"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-632470.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetPlayerAchievements/v1/",
"queryParameters": {
"appid": {
"equalTo": "646570"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-player-achievements-646570.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "1091500"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-1091500.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "1245620"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-1245620.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "1332010"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-1332010.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "268910"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-268910.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "632470"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-632470.json"
}
}
@@ -0,0 +1,18 @@
{
"request": {
"method": "GET",
"urlPath": "/steam/ISteamUserStats/GetSchemaForGame/v2/",
"queryParameters": {
"appid": {
"equalTo": "646570"
}
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"bodyFileName": "steam-schema-646570.json"
}
}
-5
View File
@@ -89,11 +89,6 @@
<groupId>org.springframework.boot</groupId> <groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId> <artifactId>spring-boot-starter-actuator</artifactId>
</dependency> </dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<scope>runtime</scope>
</dependency>
<!-- testing --> <!-- testing -->
<dependency> <dependency>
+147 -60
View File
@@ -1,22 +1,27 @@
# To Be Played – Core Service # To Be Played – Core Service
Business service of **To Be Played**, a Steam backlog tracker. It syncs the user's Steam library, stores the backlog Business service of **To Be Played**, a backlog tracker for your Steam and PlayStation libraries, plus games added
(status, rating, notes, priority), tracks playtime per day, syncs the wishlist and adds time-to-beat data from IGDB. by hand or imported from GOG Galaxy, Playnite, Grouvee and HowLongToBeat files. It syncs the user's Steam library and
the linked PlayStation account, stores the backlog (status, rating, notes, priority), tracks playtime per day, syncs
the wishlist and adds time-to-beat data from IGDB.
To Be Played is split into three repositories: To Be Played is split into four repositories:
| Repository | Role | | Repository | Role |
|---|---| |---|---|
| ToBePlayed-Auth | Users, login, settings, signs JWTs | | ToBePlayed-Auth | Users, login, settings, signs JWTs |
| **ToBePlayed-Core** (this one) | Steam library, backlog, wishlist, stats; verifies JWTs | | **ToBePlayed-Core** (this one) | Steam library, backlog, wishlist, stats; verifies JWTs |
| ToBePlayed-Frontend | React single-page app | | ToBePlayed-Frontend | React single-page app |
| ToBePlayed-Deployment | Docker Compose stack, env template, ROADMAP.md (history), plans |
## Features ## Features
- **Steam sync**: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly - **Steam sync**: owned games, playtime and achievements through the Steam Web API. Manual sync plus a nightly
sync. Resilience4j rate-limits every Steam call and tries the owned games up to three times (never again on a 4xx; sync. Resilience4j rate-limits every Steam call (2 per second) and retries only where a repeat is safe: owned games,
the wishlist and the calls a page waits for once); Caffeine caches game counts and player summaries. A sync commits its claim (`RUNNING`, with achievement schema and player achievements up to 3 times (`steamApi`, never on a 4xx); player summary and wishlist
`sync_started_at`), its writes and its finish apart, and calls Steam and IGDB outside any transaction: a second once with an empty fallback (`steamApiOnce`); store, release and global-percentage calls not at all. Caffeine caches
player summaries (`steamPlayerSummary`), IGDB searches and the landing numbers, not game counts. A sync commits its
claim (`RUNNING`, with `sync_started_at`), its writes and its finish apart, and calls Steam and IGDB outside any transaction: a second
sync sees `RUNNING` and skips, a failed one stores `FAILED`. A run older than `app.library.running-stale-minutes` sync sees `RUNNING` and skips, a failed one stores `FAILED`. A run older than `app.library.running-stale-minutes`
can be taken over; its own finish then changes nothing. can be taken over; its own finish then changes nothing.
- **Backlog**: status (`BACKLOG`, `PLAYING`, `FINISHED`, `COMPLETED` = 100 %, `DROPPED`), rating 1–5, notes, priority, - **Backlog**: status (`BACKLOG`, `PLAYING`, `FINISHED`, `COMPLETED` = 100 %, `DROPPED`), rating 1–5, notes, priority,
@@ -40,12 +45,35 @@ To Be Played is split into three repositories:
`SERIES`, `studioName` for `STUDIO`, `friendName`, `friendFinished` and `friendPlaytimeMinutes` for `FRIENDS` or `firstSeenAt` for `LONGEST`). `SERIES`, `studioName` for `STUDIO`, `friendName`, `friendFinished` and `friendPlaytimeMinutes` for `FRIENDS` or `firstSeenAt` for `LONGEST`).
Games already shown can be excluded for a reroll. Moods `STORY`, `COOP`, Games already shown can be excluded for a reroll. Moods `STORY`, `COOP`,
`CHILL`, `ACTION`, `SCARY` come from IGDB genres, themes and game modes; `SPEEDRUN` from speedrun.com. `CHILL`, `ACTION`, `SCARY` come from IGDB genres, themes and game modes; `SPEEDRUN` from speedrun.com.
```mermaid
flowchart LR
NEXT_UP --> Rows
SERIES --> Rows
SIMILAR --> Rows
STUDIO --> Rows
FRIENDS --> Rows
LONGEST --> Rows
RANDOM --> Rows
Rows["up to 3 games per row (ROW_SIZE), empty rows left out"] --> Filters["budget + exclude: every row; mood: RANDOM only"]
Filters --> Out["games + reason"]
```
- **Follow users (#27)**: a logged-in user follows people with a public profile by display name (`/api/follows`, at most - **Follow users (#27)**: a logged-in user follows people with a public profile by display name (`/api/follows`, at most
500, never an id in an answer) and gets one inbox note per followed user and batch (`/api/inbox`): a sync with its 500, never an id in an answer; `GET /api/follows/players?q=` finds public profiles by name, rate-limited) and gets
achievement run, or the hand changes of a quiet 10 minutes. A note counts new games, achievement unlocks, status one inbox note per followed user and batch (`/api/inbox`): a sync with its achievement run, or the hand changes of a
changes and Steam wishlist adds, and leaves out what the followed user's profile hides (`public_user.sections`). quiet 10 minutes. A note counts new games, achievement unlocks, status changes and Steam wishlist adds, and leaves
out what the followed user's profile hides (`public_user.sections`).
Follows end for good, with a note, when the followed user goes private, and without a note when the user is blocked or Follows end for good, with a note, when the followed user goes private, and without a note when the user is blocked or
deleted. The nightly job at 03:15 deletes notes older than 90 days (`app.inbox.keep-days`); the list shows the newest 50. deleted. Each note keeps one copy of the lines behind its counts (`inbox_note_detail`: games, achievement or trophy
names, status moves collapsed per game, wishlist adds; at most 20 achievements per game, the newest unlocks stand for
the new ones). `GET /api/inbox/{id}/details` shows them only while the reader still follows the subject, the subject
is public, else `404`. Rows of a hidden part or a hidden game are left out. The nightly job at 03:15 deletes notes
older than 90 days (`app.inbox.keep-days`); the list shows the newest 50. A note also carries `profileName` (the
subject's current name while the profile is public, else null), a detail line its `appid`, and each followed person
in `GET /api/follows` `lastActivityAt` (the newest digest note the caller got about them): the Frontend links the feed
to the game and to the profile view of that game. `GET /api/library/games/{appid}/preview` gives the catalog data of
any game (for "not in your library yet"), `GET /api/public/profiles/library/games/{appid}` one listed game of a public
profile, filtered by the ticket's sections; a hidden, removed or not owned game answers the same `404`.
- **Backlog order**: every listed `BACKLOG` game (not removed, not hidden) has a row in `backlog_order (user_id, - **Backlog order**: every listed `BACKLOG` game (not removed, not hidden) has a row in `backlog_order (user_id,
appid, position)`, kept by a database trigger on `user_game` like the status history: a game entering the backlog appid, position)`, kept by a database trigger on `user_game` like the status history: a game entering the backlog
goes to the bottom, a game leaving it loses its row. V34 seeded the first order (priority HIGH, MEDIUM, rest, then goes to the bottom, a game leaving it loses its row. V34 seeded the first order (priority HIGH, MEDIUM, rest, then
@@ -57,27 +85,22 @@ To Be Played is split into three repositories:
sum). It keeps its status and shows `coming` / `comingOn`. Date: IGDB `first_release_date`, else Steam's store sum). It keeps its status and shows `coming` / `comingOn`. Date: IGDB `first_release_date`, else Steam's store
(`GetItems` with `include_release`), else unknown (= released). View `coming_game`, table `game_release`; a (`GetItems` with `include_release`), else unknown (= released). View `coming_game`, table `game_release`; a
year-only IGDB date stays a placeholder until the real date comes (coming rows refresh daily). year-only IGDB date stays a placeholder until the real date comes (coming rows refresh daily).
- **Speedrun games**: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (case,
```mermaid punctuation, ™ and roman numerals ignored) and a release year at most one year off, when IGDB knows it, with at
flowchart LR least one verified run. An edition that finds nothing tries its IGDB parent game with the parent's year, else its
A["anchor: random of top 5 played"] --> S[similar] name without an edition suffix. Several same-name games and no year to decide: no link. The game detail returns
S --> M["PickMix: rotate 4 sources, dedupe, random fallback"]
L[longest waiting] --> M
R[random] --> M
N["next up: random of top 5 in order"] --> M
M --> O["games + reason"]
```
- **Speedrun games**: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and
a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns
the speedrun.com link as `speedrunUrl`. speedrun.com has no Steam or IGDB link and no key, so one queue works for the speedrun.com link as `speedrunUrl`. speedrun.com has no Steam or IGDB link and no key, so one queue works for
all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days. all users, with up to 3 requests per game and 1 request every 2 seconds; a result is kept 30 days. A rate limit answer
(429 or 420) pauses the queue (its Retry-After, else 1 min doubling up to 30 min) and asks the same game again; the
queue is never dropped for it. Migration V47 asks
every stored match again once at boot, behind user requests; links set by an auditor are never touched.
- **Achievements**: after every library sync a background job fetches the achievement schema of each owned game - **Achievements**: after every library sync a background job fetches the achievement schema of each owned game
and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half
of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never
overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5 overwrites stored data. Summary: unlocked total, perfect games, average completion over started games, the 5
games closest to 100 %. games closest to 100 %.
- **Playtime history**: one row per game and day, for the activity view and the "most played in the last two weeks" - **Playtime history**: one row per game and day (a day counts at most 24 h in the stats and the chart), for the
dashboard banner. activity view and the "most played in the last two weeks" dashboard banner.
- **Finished from achievements or playtime**: `EndingAchievement` finds the one achievement of a game that means - **Finished from achievements or playtime**: `EndingAchievement` finds the one achievement of a game that means
"story end" (English text rule, Steam's global unlock rate between 1 and 80 %, the easiest variant wins, more than "story end" (English text rule, Steam's global unlock rate between 1 and 80 %, the easiest variant wins, more than
3 candidates = a game without one end). When the user unlocks it, never chose the status and never answered 3 candidates = a game without one end). When the user unlocks it, never chose the status and never answered
@@ -122,8 +145,11 @@ flowchart LR
genres, moods, perfect games and the longest day list one entry per set, with the playtime of all its editions genres, moods, perfect games and the longest day list one entry per set, with the playtime of all its editions
(`user_game_entry_member`, V43; its list shows that sum as `minutes`). Every playtime number (most played, genres, (`user_game_entry_member`, V43; its list shows that sum as `minutes`). Every playtime number (most played, genres,
moods, the total playtime and moon trips, the drill-down lists) uses the set total, the view moods, the total playtime and moon trips, the drill-down lists) uses the set total, the view
`user_game_entry_playtime` (V45), the same as the library card, sort and public profile. Platforms stay per edition: `user_game_entry_playtime` (V45), the same as the library card, sort and public profile. So do "played", "never
a Steam + PlayStation set counts on both platforms. started" and "under one hour": a set counts once, judged by its set total (also the dashboard "never played" and the
oldest unplayed game) and the trend's never played (`backlog_snapshot`); rows written before #43 keep the old value,
they cannot be rebuilt.
Platforms stay per edition: a Steam + PlayStation set counts on both platforms.
- **Drill-down**: every countable number opens the games behind it. A `StatKey` (library), `CommunityKey` (landing page) - **Drill-down**: every countable number opens the games behind it. A `StatKey` (library), `CommunityKey` (landing page)
or a Pile Points track/bonus row names a number; its SQL "game set" is the same text the number counts, so the or a Pile Points track/bonus row names a number; its SQL "game set" is the same text the number counts, so the
list and the number cannot drift. 40 games per page (at most page 250), a fixed order per key. A profile visitor list and the number cannot drift. 40 games per page (at most page 250), a fixed order per key. A profile visitor
@@ -140,8 +166,9 @@ flowchart LR
proof of fresh play (ending achievement or playtime around the finish, or half of the IGDB time played since the finish). Without proof a Steam finish pays 10 early points (`FINISHED_EARLY`), only for finishes on or after proof of fresh play (ending achievement or playtime around the finish, or half of the IGDB time played since the finish). Without proof a Steam finish pays 10 early points (`FINISHED_EARLY`), only for finishes on or after
`app.rewards.early-since` (older unproven finishes pay only with proof); when proof arrives later, `FINISHED_TOPUP` pays the rest up to the full amount, once per game or set. PlayStation days never prove: a game linked to a PlayStation title pays by its own rules, only `app.rewards.early-since` (older unproven finishes pay only with proof); when proof arrives later, `FINISHED_TOPUP` pays the rest up to the full amount, once per game or set. PlayStation days never prove: a game linked to a PlayStation title pays by its own rules, only
PlayStation rows pay nothing. The month bonus needs a finished game in that month. Games added by hand pay 15 for PlayStation rows pay nothing. The month bonus needs a finished game in that month. Games added by hand pay 15 for
a finish. Manual finishes and early finishes share the cap of 3 per ISO week. `COMPLETED` needs a `FINISHED` or `FINISHED_TOPUP` row. a finish. Manual finishes and early finishes share the cap of 3 per ISO week. `COMPLETED` needs a `FINISHED`, `FINISHED_TOPUP` or `FINISHED_MANUAL` row on the game or its set, so a manual edition's finish opens it for the Steam edition, but only when the Steam game's last achievement
`GET /api/rewards/expected?appids=` (read-only, at most 200) answers the full finish value per game for the Questions cards. `GET /api/rewards/games/{appid}` (game page open, after every status save) settles, then answers the paid rows of the game and its set (`bookedNow` = written by this call) and one `NoPointsReason` per missing kind (`STARTED`, `FINISHED`, `COMPLETED`); the codes live in `RewardRules`, the Frontend only maps them to texts. The settle takes a per-user advisory lock. V26 also pays a one-time starter grant unlock (100 % reached) is on or after `app.rewards.early-since` (else `COMPLETION_TOO_OLD`); a manual game itself never gets it.
`GET /api/rewards/expected?appids=` (read-only, at most 200) answers the full finish value per game for the Questions cards. `GET /api/rewards/games/{appid}` (game page open, after every status save) settles, then answers the paid rows of the game and its set (`bookedNow` = written by this call) and one `NoPointsReason` per missing kind (`STARTED`, `FINISHED`, `COMPLETED`); the codes live in `RewardRules`, the Frontend only maps them to texts; `earlySince` (ISO date, `app.rewards.early-since`) fills the date in the texts. The settle takes a per-user advisory lock. V26 also pays a one-time starter grant
for finishes from before the status log. Shop: a whitelist of 57 looks (`ShopItem`; 45 buyable and 12 unlock-only) in eight slots, including animated completed effects, page backgrounds (CSS, SVG and two canvas engines in the Frontend), for finishes from before the status log. Shop: a whitelist of 57 looks (`ShopItem`; 45 buyable and 12 unlock-only) in eight slots, including animated completed effects, page backgrounds (CSS, SVG and two canvas engines in the Frontend),
banner backgrounds (V27 widens `reward_equipped.slot`) and three buyable skins (full themes, slot `SKIN`). The unlock-only banner backgrounds (V27 widens `reward_equipped.slot`) and three buyable skins (full themes, slot `SKIN`). The unlock-only
items (price 0, never sold): the settle books an `UNLOCK` row (ref = item code, 0 points) once a items (price 0, never sold): the settle books an `UNLOCK` row (ref = item code, 0 points) once a
@@ -165,8 +192,8 @@ flowchart LR
header picture (`IStoreBrowseService/GetItems` with assets) goes to `steam_game.image_url`. When a few achievement header picture (`IStoreBrowseService/GetItems` with assets) goes to `steam_game.image_url`. When a few achievement
icons do not load, the schema comes fresh from Steam; an empty answer keeps the stored one. A report that cannot be repaired is tried again on the next report, at most 3 times, icons do not load, the schema comes fresh from Steam; an empty answer keeps the stored one. A report that cannot be repaired is tried again on the next report, at most 3 times,
then after 30 days. then after 30 days.
- **Time to beat**: story, extras and completionist hours from the IGDB API. Optional: without credentials the - **Time to beat**: IGDB's `hastily`, `normally` and `completely` minutes (`game_time_to_beat`, V6). The Frontend labels
feature is off and the rest works. them story / extras / 100 %. Optional: without credentials the feature is off and the rest works.
- **PlayStation link** (optional, unofficial API): the user pastes an NPSSO once, Core keeps only the encrypted refresh - **PlayStation link** (optional, unofficial API): the user pastes an NPSSO once, Core keeps only the encrypted refresh
token (about 10 days). Every library sync then adds the played PS4/PS5 titles that match IGDB exactly, keeps the unsure token (about 10 days). Every library sync then adds the played PS4/PS5 titles that match IGDB exactly, keeps the unsure
ones for the import review, and updates playtime and last played date. The same sync reads the PS4/PS5 **trophy ones for the import review, and updates playtime and last played date. The same sync reads the PS4/PS5 **trophy
@@ -191,7 +218,7 @@ The token comes from Auth's `tbp_access` cookie or a `Bearer` header, whichever
(`USER < AUDITOR < PLATFORM_ADMIN`). A token without `aud` is refused. The claims are pinned in (`USER < AUDITOR < PLATFORM_ADMIN`). A token without `aud` is refused. The claims are pinned in
`src/test/resources/contract/auth-core-contract.json`, the same file as in Auth; the tests build tokens only from it. `src/test/resources/contract/auth-core-contract.json`, the same file as in Auth; the tests build tokens only from it.
`POST`, `PUT` and `DELETE` need an `X-TBP-Client` or `Authorization` header (CSRF guard), and CORS allows only the `POST`, `PUT` and `DELETE` need an `X-TBP-Client` or `Authorization` header (CSRF guard), and CORS allows only the
frontend origin. frontend origin; preflight answers carry `Access-Control-Max-Age: 3600` (pinned by `ClientHeaderFilterTest`).
A token for another audience (for example a profile ticket) is rejected on normal endpoints; the profile ticket is A token for another audience (for example a profile ticket) is rejected on normal endpoints; the profile ticket is
accepted only by `GET /api/public/profiles/library` in the `X-Profile-Ticket` header. accepted only by `GET /api/public/profiles/library` in the `X-Profile-Ticket` header.
Every error answer of a controller has the body `{code, message}` (a 401 or 403 of the security filters has none); a refused request is `400 VALIDATION_FAILED` with Every error answer of a controller has the body `{code, message}` (a 401 or 403 of the security filters has none); a refused request is `400 VALIDATION_FAILED` with
@@ -221,14 +248,17 @@ ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/aut
| Method | Path | Auth | | Method | Path | Auth |
|---|---|---| |---|---|---|
| `GET` | `/api/rewards/wallet` (balance, catalog, owned, equipped, last 50 ledger rows with the game name, unlock `progress` per track) | JWT | | `GET` | `/api/rewards/wallet` (balance, catalog, owned, equipped, last 50 ledger rows with the game name, unlock `progress` per track, `earlySince` for the rules text) | JWT |
| `GET` | `/api/rewards/expected?appids=` (`{appid: points}`, the full finish value, 0 when paid; read-only, no settle; `400 TOO_MANY` over 200 appids) | JWT | | `GET` | `/api/rewards/expected?appids=` (`{appid: points}`, the full finish value, 0 when paid; read-only, no settle; `400 TOO_MANY` over 200 appids) | JWT |
| `POST` | `/api/rewards/purchases` (`{item}`; `200` wallet, `409 ALREADY_OWNED` / `NOT_ENOUGH_POINTS` / `NOT_FOR_SALE`) | JWT | | `POST` | `/api/rewards/purchases` (`{item}`; `200` wallet, `409 ALREADY_OWNED` / `NOT_ENOUGH_POINTS` / `NOT_FOR_SALE`) | JWT |
| `GET` | `/api/rewards/games/{appid}` (settles, then the paid rows of the game and its set, `bookedNow`, one reason per missing kind; `404 GAME_NOT_FOUND`) | JWT |
| `GET` | `/api/rewards/progress/{track}/games?page=`, `/api/rewards/history/{kind}/{ref}/games?page=` (the games behind a track or a `MONTH` / `STREAK` bonus; `400 NOT_A_GAME_LIST`) | JWT | | `GET` | `/api/rewards/progress/{track}/games?page=`, `/api/rewards/history/{kind}/{ref}/games?page=` (the games behind a track or a `MONTH` / `STREAK` bonus; `400 NOT_A_GAME_LIST`) | JWT |
| `PUT` | `/api/rewards/equipped/{slot}` (`{item}`, null = take off; `200` wallet, `409 NOT_OWNED`, `400 WRONG_SLOT`) | JWT | | `PUT` | `/api/rewards/equipped/{slot}` (`{item}`, null = take off; `200` wallet, `409 NOT_OWNED`, `400 WRONG_SLOT`) | JWT |
| `GET` | `/api/library/games` (one row per set, the counted edition, with `platforms[]` of all its editions: Steam, PlayStation, then manual, and `totalPlaytimeMinutes` (all platforms; `playtimeMinutes` stays the counted edition's own); `name` matches any edition; `source=STEAM,PSN` keeps rows with a platform of that source, a PlayStation title on a Steam row counts as `PSN`; ignored with `hidden=true`, which lists every hidden row; `400` on an unknown source; an old `hideDuplicates` param is ignored), `/api/library/games/{appid}` | JWT | | `GET` | `/api/library/games` (`sort` = `playtimeMinutes`, `firstSeenAt`, `lastSeenAt`, `rating`, `priority`, `name`, `lastPlayedAt` or `timeToBeat` plus `,asc` or `,desc`; a game never played or without a time to beat comes last; one row per set, the counted edition, with `platforms[]` of all its editions: Steam, PlayStation, then manual, and `totalPlaytimeMinutes` (all platforms; `playtimeMinutes` stays the counted edition's own); `name` matches any edition; `source=STEAM,PSN` keeps rows with a platform of that source, a PlayStation title on a Steam row counts as `PSN`; ignored with `hidden=true`, which lists every hidden row; `400` on an unknown source; an old `hideDuplicates` param is ignored) | JWT |
| `GET` | `/api/library/dashboard-games?top=10&playing=8` (`{top, playing}`: the Dashboard tiles in one call, listed entries by set playtime, most first; `playing` only status PLAYING; same rows and order as `/games` with `sort=playtimeMinutes,desc` / `status=PLAYING`, no count query; limits clamped to 1..20; calls `syncIfStale` like `/games`) | JWT |
| `PATCH` | `/api/library/games/{appid}/backlog` (only the sent fields; `null` clears one) | JWT | | `PATCH` | `/api/library/games/{appid}/backlog` (only the sent fields; `null` clears one) | JWT |
| `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`, `time_to_beat` = `{normallyMinutes, completelyMinutes}`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR | | `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`, `time_to_beat` = `{normallyMinutes, completelyMinutes}`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR |
| `GET` | `/api/library/games/{appid}/preview` (`{appid, name, imageUrl, genres, timeToBeat}`: catalog data of any game, a merged key answers with the kept key; `404 GAME_NOT_FOUND`) | JWT |
| `GET` | `/api/library/catalog?q=&page=&size=` (every game some user still has, by name part, `PageDto` of `{appid, name, imageUrl, platforms}`; `q` at most 100 characters; size 20, 40 or 80) | JWT, AUDITOR | | `GET` | `/api/library/catalog?q=&page=&size=` (every game some user still has, by name part, `PageDto` of `{appid, name, imageUrl, platforms}`; `q` at most 100 characters; size 20, 40 or 80) | JWT, AUDITOR |
| `GET` | `/api/library/catalog/{appid}` (what the correction dialog reads: name, image, speedrun, moods, endless, genres, timeToBeat, corrections; `404 GAME_NOT_FOUND`) | JWT, AUDITOR | | `GET` | `/api/library/catalog/{appid}` (what the correction dialog reads: name, image, speedrun, moods, endless, genres, timeToBeat, corrections; `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
| `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR | | `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
@@ -243,6 +273,14 @@ ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/aut
| `GET` | `/api/library/questions`, `/api/library/questions/count` (`{results: [{appid, name, imageUrl, question, status, playtimeMinutes, normallyMinutes, lastActivity}]}`, `{count}`) | JWT | | `GET` | `/api/library/questions`, `/api/library/questions/count` (`{results: [{appid, name, imageUrl, question, status, playtimeMinutes, normallyMinutes, lastActivity}]}`, `{count}`) | JWT |
| `POST` | `/api/library/questions/{appid}` (`{question, yes, rating?}`; `200 {next}`, `400 INVALID_ANSWER`, `404 GAME_NOT_FOUND`, `409 QUESTION_GONE`) | JWT | | `POST` | `/api/library/questions/{appid}` (`{question, yes, rating?}`; `200 {next}`, `400 INVALID_ANSWER`, `404 GAME_NOT_FOUND`, `409 QUESTION_GONE`) | JWT |
| `PUT` | `/api/library/games/{appid}/finished-hint` (`{dismissed: ENDING\|PLAYTIME}`, "Not yet"; `204`) | JWT | | `PUT` | `/api/library/games/{appid}/finished-hint` (`{dismissed: ENDING\|PLAYTIME}`, "Not yet"; `204`) | JWT |
| `GET` | `/api/library/removed-games` (`{results: [{appid, name, imageUrl, source, platform, removedAt}]}`: the caller's removed `MANUAL` and `PSN` games, newest first) | JWT |
| `POST` | `/api/library/games/{appid}/restore` (a removed game comes back with its old data; `204`, also when it is already back; `404 GAME_NOT_FOUND`, `409 STEAM_GAME`) | JWT |
| `GET` | `/api/library/game-search?q=` (IGDB search for "add a game", `q` 2-100 characters; `{available, results: [{appid, inLibrary, ...}]}`; `502 IGDB_FAILED`) | JWT |
| `POST` | `/api/library/games` (`{igdbId, platform, status, playtimeMinutes?}`; `201 {appid}`, `400 UNKNOWN_GAME`, `409 ALREADY_IN_LIBRARY`, `503 IGDB_OFF`) | JWT |
| `PUT` | `/api/library/games/{appid}/manual` (`{platform, playtimeMinutes}`, MANUAL and PSN rows, a PSN row keeps its playtime; `204`, `404 GAME_NOT_FOUND`, `409 STEAM_GAME`) | JWT |
| `DELETE` | `/api/library/games/{appid}` (soft remove, same answers) | JWT |
| `GET` | `/api/library/sync-changes` (`{syncedAt, startedBy, seen, importedGames, changes[]}` of the last sync; `204` when none) | JWT |
| `PUT` | `/api/library/sync-changes/seen` (`{syncedAt}`; idempotent; `204`) | JWT |
| `PUT` | `/api/library/games/{appid}/hidden` (`{hidden}`; Steam and manual games) | JWT | | `PUT` | `/api/library/games/{appid}/hidden` (`{hidden}`; Steam and manual games) | JWT |
| `POST` | `/api/library/imports` (`{source: TEXT\|CSV\|SEARCH\|GALAXY\|GROUVEE\|HLTB\|PLAYNITE\|BOOKMARKLET, rows: [{title, platform, playtimeMinutes, status, igdbId}]}`, 1–1000 rows; `202 {id}` in `QUEUED`, `409 IMPORT_RUNNING`, `503 IGDB_OFF`) | JWT | | `POST` | `/api/library/imports` (`{source: TEXT\|CSV\|SEARCH\|GALAXY\|GROUVEE\|HLTB\|PLAYNITE\|BOOKMARKLET, rows: [{title, platform, playtimeMinutes, status, igdbId}]}`, 1–1000 rows; `202 {id}` in `QUEUED`, `409 IMPORT_RUNNING`, `503 IGDB_OFF`) | JWT |
| `GET` | `/api/library/imports` (`{imports}`, the last 20), `/api/library/imports/{id}` (status, progress counts, `ahead` while it waits in a queue, `attempts`; the rows with their candidates while `READY`) | JWT | | `GET` | `/api/library/imports` (`{imports}`, the last 20), `/api/library/imports/{id}` (status, progress counts, `ahead` while it waits in a queue, `attempts`; the rows with their candidates while `READY`) | JWT |
@@ -252,7 +290,7 @@ ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/aut
| `GET`, `DELETE` | `/api/library/psn-account` (`{enabled, linked, linkedAt, expiresAt, reviewImportId}`: the PSN import waiting for review, else null), unlink (`204`, deletes the stored token) | JWT | | `GET`, `DELETE` | `/api/library/psn-account` (`{enabled, linked, linkedAt, expiresAt, reviewImportId}`: the PSN import waiting for review, else null), unlink (`204`, deletes the stored token) | JWT |
| `GET`, `DELETE` | `/api/library/imports/{id}/undo-check` (`{games, changed}`), `/api/library/imports/{id}/games` (undo; `204`, `409 IMPORT_NOT_SAVED`) | JWT | | `GET`, `DELETE` | `/api/library/imports/{id}/undo-check` (`{games, changed}`), `/api/library/imports/{id}/games` (undo; `204`, `409 IMPORT_NOT_SAVED`) | JWT |
| `POST` | `/api/library/sync` | JWT | | `POST` | `/api/library/sync` | JWT |
| `GET` | `/api/library/games/{appid}` (detail; slice 3: `achievements[]` also hold trophies with `kind` `STEAM`\|`PSN`, `setId`, `trophyType`, `group`, `rarity`; new `achievementSets[]` = `{id, kind, platform, name, total, unlocked, perfect, visible, listed}`, Steam first, then PS5 before PS4; `hasAchievements` = any set; the top-level counts stay Steam only; `platforms[]` = `{source, platform, appid, playtimeMinutes, lastPlayedAt, achievementsUnlocked, achievementsTotal, perfect}` one line per platform of the entry, Steam, PS5, PS4, manual, each with its own progress, `appid` = the edition whose detail lists them; `totalPlaytimeMinutes` = their sum) | JWT | | `GET` | `/api/library/games/{appid}` (detail; `refresh=false` answers from stored data and sets `achievementsStale` when a due Steam refresh was skipped; `achievements[]` also hold trophies with `kind` `STEAM`\|`PSN`, `setId`, `trophyType`, `group`, `rarity`; new `achievementSets[]` = `{id, kind, platform, name, total, unlocked, perfect, visible, listed}`, Steam first, then PS5 before PS4; `hasAchievements` = any set; the top-level counts stay Steam only; `platforms[]` = `{source, platform, appid, playtimeMinutes, lastPlayedAt, achievementsUnlocked, achievementsTotal, perfect}` one line per platform of the entry, Steam, PS5, PS4, manual, each with its own progress, `appid` = the edition whose detail lists them; `totalPlaytimeMinutes` = their sum) | JWT |
| `POST` | `/api/library/broken-images` (`{appid, kind}`, kind `COVER` or `ACHIEVEMENT_ICONS`; `204`) | JWT | | `POST` | `/api/library/broken-images` (`{appid, kind}`, kind `COVER` or `ACHIEVEMENT_ICONS`; `204`) | JWT |
| `GET` | `/api/library/backlog?limit=&released=` (`{games}` in the user's order; `limit` 1-20, none = all) | JWT | | `GET` | `/api/library/backlog?limit=&released=` (`{games}` in the user's order; `limit` 1-20, none = all) | JWT |
| `PUT` | `/api/library/backlog` (`{appids}` = the whole order; `204`) | JWT | | `PUT` | `/api/library/backlog` (`{appids}` = the whole order; `204`) | JWT |
@@ -263,14 +301,17 @@ ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/aut
| `GET` | `/api/library/sync-status`, `/summary`, `/activity`, `/wishlist`, `/steam-profile`, `/games-count` (`canSync`: a Steam ID or a PlayStation link; `sources`: the sources of the user's games) | JWT | | `GET` | `/api/library/sync-status`, `/summary`, `/activity`, `/wishlist`, `/steam-profile`, `/games-count` (`canSync`: a Steam ID or a PlayStation link; `sources`: the sources of the user's games) | JWT |
| `GET` | `/api/public/profiles/library` | profile ticket | | `GET` | `/api/public/profiles/library` | profile ticket |
| `GET` | `/api/public/profiles/library/stats/games?key=&arg=&page=` (`403 NOT_SHARED`, `400 BAD_ARG`) | profile ticket | | `GET` | `/api/public/profiles/library/stats/games?key=&arg=&page=` (`403 NOT_SHARED`, `400 BAD_ARG`) | profile ticket |
| `GET` | `/api/public/profiles/library/games/{appid}` (`{appid, name, imageUrl, status, playtimeMinutes, achievementsUnlocked, achievementsTotal, completionPercent}`, a field of a hidden section is null; `404 GAME_NOT_FOUND` for a hidden, removed or not owned game or when every field is hidden, `401 INVALID_TICKET`) | profile ticket |
| `GET` | `/api/public/profiles/rewards` (`{equipped}`, the worn looks) | profile ticket | | `GET` | `/api/public/profiles/rewards` (`{equipped}`, the worn looks) | profile ticket |
| `GET` | `/api/stats/public` (+ games per platform, top genres, finished per year) | public | | `GET` | `/api/stats/public` (+ games per platform, top genres, finished per year) | public |
| `GET` | `/api/stats/owners?appid=&key=&arg=&page=` (who owns a game in the set behind a community number, `key` default `OWNED`: `listed` display names only, `listedTotal`, `privateCount`, `includesYou`, not cached, `400 BAD_ARG`) | login | | `GET` | `/api/stats/owners?appid=&key=&arg=&page=` (who owns a game in the set behind a community number, `key` default `OWNED`: `listed` display names only, `listedTotal`, `privateCount`, `includesYou`, not cached, `400 BAD_ARG`) | login |
| `GET` | `/api/stats/public/games?key=&arg=&page=` (the games behind a landing number, per game `libraries` and `minutes`; `400 BAD_ARG`) | public | | `GET` | `/api/stats/public/games?key=&arg=&page=` (the games behind a landing number, per game `libraries` and `minutes`; `400 BAD_ARG`) | public |
| `GET` | `/api/follows` (`{following: [{id, displayName}], followers: [{id, displayName}]}`, `id` is the follow row; a follower with a private profile has `displayName` null) | login | | `GET` | `/api/follows` (`{following: [{id, displayName, lastActivityAt}], followers: [{id, displayName, lastActivityAt}]}`, `id` is the follow row; a follower with a private profile has `displayName` null; `lastActivityAt` = the newest digest note the caller got about that person, null for followers and without a note) | login |
| `PUT`, `DELETE` | `/api/follows/by-name/{displayName}` (follow, unfollow; `204`; PUT: `404 PROFILE_NOT_FOUND` for an unknown, private, blocked or deleted name, `400 FOLLOW_SELF`, `400 FOLLOW_LIMIT`) | login | | `PUT`, `DELETE` | `/api/follows/by-name/{displayName}` (follow, unfollow; `204`; PUT: `404 PROFILE_NOT_FOUND` for an unknown, private, blocked or deleted name, `400 FOLLOW_SELF`, `400 FOLLOW_LIMIT`) | login |
| `DELETE` | `/api/follows/{id}` (unfollow, or remove a follower; only rows of the caller change; `204`) | login | | `DELETE` | `/api/follows/{id}` (unfollow, or remove a follower; only rows of the caller change; `204`) | login |
| `GET` | `/api/inbox` (`{results: [{id, kind, displayName, newGames, achievements, statusChanges, wishlistAdds, createdAt, read}], unread}`, newest 50, `kind` `DIGEST` or `VISIBILITY`) | login | | `GET` | `/api/follows/players?q=` (`{results: [{displayName}]}`: public profiles whose name contains `q`, case-insensitive, not blocked or deleted, at most 20, names that start with `q` first, the caller's own name included; `q` under 2 characters answers `[]`; `429 TOO_MANY_SEARCHES` beyond 30 searches per user and minute, `app.follow.search-per-minute`) | login |
| `GET` | `/api/inbox` (`{results: [{id, kind, displayName, profileName, newGames, achievements, statusChanges, wishlistAdds, createdAt, read, details}], unread}`, newest 50, `kind` `DIGEST` or `VISIBILITY`) | login |
| `GET` | `/api/inbox/{id}/details` (`{results: [{appid, kind, gameName, itemName, fromStatus, toStatus}]}`, `kind` `NEW_GAME`, `ACHIEVEMENT`, `STATUS` or `WISHLIST`; `404` for a note that is not the caller's or not open any more) | login |
| `PUT` | `/api/inbox/read` (marks all read; `204`) | login | | `PUT` | `/api/inbox/read` (marks all read; `204`) | login |
| `POST`, `PUT` | `/internal/accounts/deleted` (`{userIds}`; POST adds, PUT replaces the whole set; `204`) | service token | | `POST`, `PUT` | `/internal/accounts/deleted` (`{userIds}`; POST adds, PUT replaces the whole set; `204`) | service token |
| `POST`, `PUT` | `/internal/accounts/states` (`{accounts: [{userId, steamId, displayName, blocked, roleChangedAt, listed, publicProfile, profileSections}]}`, `displayName`, `roleChangedAt`, `listed` and `publicProfile` optional (`listed`: Auth lists the user by name, kept in `listed_user`, `null` keeps it; `publicProfile`: the profile is public, kept as a row in `public_user` with `profileSections`, `null` keeps row and sections, `false` ends the follows of the user with one `VISIBILITY` note each; `profileSections` `null` = all shown), `displayName` at most 100 characters, kept in `account_name` for the audit log, `roleChangedAt` kept in `account_role_change`; POST updates, PUT replaces the whole set; `204`) | service token | | `POST`, `PUT` | `/internal/accounts/states` (`{accounts: [{userId, steamId, displayName, blocked, roleChangedAt, listed, publicProfile, profileSections}]}`, `displayName`, `roleChangedAt`, `listed` and `publicProfile` optional (`listed`: Auth lists the user by name, kept in `listed_user`, `null` keeps it; `publicProfile`: the profile is public, kept as a row in `public_user` with `profileSections`, `null` keeps row and sections, `false` ends the follows of the user with one `VISIBILITY` note each; `profileSections` `null` = all shown), `displayName` at most 100 characters, kept in `account_name` for the audit log, `roleChangedAt` kept in `account_role_change`; POST updates, PUT replaces the whole set; `204`) | service token |
@@ -283,9 +324,9 @@ ArchUnit role rules (`RoleAnnotationRules`) and `src/test/resources/contract/aut
- PostgreSQL 17, Flyway migrations - PostgreSQL 17, Flyway migrations
- Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com - Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com
- JJWT with Ed25519 public key - JJWT with Ed25519 public key
- Spring Boot Actuator + Micrometer (Prometheus) - Spring Boot Actuator (`/actuator/health` only)
- Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit - Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit
- CI: Forgejo Actions, multi-stage Docker image - CI: Gitea Actions (`.gitea/workflows/publish-core.yml`), multi-stage Docker image
## Architecture ## Architecture
@@ -326,8 +367,8 @@ flowchart LR
Psn -->|"NPSSO sign-in, played titles, trophy sets and lists"| PsnApi["PlayStation API (unofficial)"] Psn -->|"NPSSO sign-in, played titles, trophy sets and lists"| PsnApi["PlayStation API (unofficial)"]
LibRepo -->|"own tables; picker also follow, public_user, account_name, blocked_user, deleted_user; inbox_note (PSN expiry notes)"| Db[("PostgreSQL tbp_core")] LibRepo -->|"own tables; picker also follow, public_user, account_name, blocked_user, deleted_user; inbox_note (PSN expiry notes)"| Db[("PostgreSQL tbp_core")]
Stats -->|"listed_user_game, counted_user_game, game_mood, coming_game, user_game_status_change, deleted_user, blocked_user, listed_user, account_name, game_alias"| Db Stats -->|"listed_user_game, counted_user_game, game_mood, coming_game, user_game_status_change, deleted_user, blocked_user, listed_user, account_name, game_alias"| Db
Follow -->|"follow, follow_digest, inbox_note, public_user, sync_change, user_game_status_change, account_name"| Db Follow -->|"follow, follow_digest, inbox_note, inbox_note_detail, public_user, sync_change, user_game_status_change, user_game_achievement, user_psn_trophy, wishlist_game, account_name"| Db
Inbox --> Db Inbox -->|"inbox_note, inbox_note_detail, follow, public_user, user_game, account_name, blocked_user, deleted_user"| Db
Auth -->|"service token<br/>/internal/accounts"| Account Auth -->|"service token<br/>/internal/accounts"| Account
Account -->|"per-user rows, deleted_user, blocked_user, listed_user, public_user, follow, account_name, user_seen_at"| Db Account -->|"per-user rows, deleted_user, blocked_user, listed_user, public_user, follow, account_name, user_seen_at"| Db
``` ```
@@ -338,7 +379,7 @@ flowchart LR
| `stats` | Public numbers for the landing page | | `stats` | Public numbers for the landing page |
| `version` | `GET /api/version`: commit and build time of the running image | | `version` | `GET /api/version`: commit and build time of the running image |
| `follow` | Follows (`follow`), the digest cursor per followed user (`follow_digest`) and `FollowDigestService`: one note per follower and batch, from `sync_change` and the hand rows of `user_game_status_change` | | `follow` | Follows (`follow`), the digest cursor per followed user (`follow_digest`) and `FollowDigestService`: one note per follower and batch, from `sync_change` and the hand rows of `user_game_status_change` |
| `inbox` | `inbox_note`: the notes of a user, `GET /api/inbox`, the 90-day purge | | `inbox` | `inbox_note`, `inbox_note_detail`: the notes of a user, `GET /api/inbox`, `GET /api/inbox/{id}/details`, the 90-day purge |
| `account` | Driven by Auth: tombstones in `deleted_user`, purge and orphan prune of the per-user rows; blocks in `blocked_user`, listing in `listed_user` (users Auth shows by name in owner lists, #26), public profiles and their sections in `public_user` (#27), role changes in `account_role_change`, display names in `account_name`, Steam ID changes stop the nightly sync | | `account` | Driven by Auth: tombstones in `deleted_user`, purge and orphan prune of the per-user rows; blocks in `blocked_user`, listing in `listed_user` (users Auth shows by name in owner lists, #26), public profiles and their sections in `public_user` (#27), role changes in `account_role_change`, display names in `account_name`, Steam ID changes stop the nightly sync |
| `steam_adapter` | Steam Web API client, availability check of Steam CDN pictures | | `steam_adapter` | Steam Web API client, availability check of Steam CDN pictures |
| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search | | `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search |
@@ -380,9 +421,11 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
### Sync flow ### Sync flow
1. `GET /api/library/games`, `/games-count` and `/picker` call `LibrarySyncService.syncIfStale`. It starts a sync 1. `GET /api/library/games`, `/dashboard-games`, `/games-count` and `/picker` call `LibrarySyncService.syncIfStale`. It starts a sync
when the last one is older than 24 hours, older than 15 minutes after a failure, or was for another Steam ID. when the last success is older than 20 hours (`app.library.sync-ttl-hours`) and the last try older than 60 minutes
`POST /api/library/sync` starts one at once, but at most every 60 seconds (else `429` with `Retry-After`). (`login-retry-minutes`, also after a failure), or when it was for another Steam ID.
`POST /api/library/sync` starts one at once, but at most every 60 seconds (`force-sync-cooldown-seconds`),
else `429` with `Retry-After`.
2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A 2. A nightly job (23:55, Europe/Berlin) syncs every account used in the last 14 days, one after the other. A
request without a `steamId` claim (the user unlinked the Steam account in Auth) takes the user out of it. request without a `steamId` claim (the user unlinked the Steam account in Auth) takes the user out of it.
Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and
@@ -396,6 +439,12 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
no old playtime, and one game gets at most 24 hours per day. Steam's `rtime_last_played` (unix seconds, 0 = never) no old playtime, and one game gets at most 24 hours per day. Steam's `rtime_last_played` (unix seconds, 0 = never)
is stored in `user_game.last_played_at`; a missing or zero value keeps the old one. A BACKLOG game with new playtime becomes PLAYING, unless the user chose its status. When Steam returns no is stored in `user_game.last_played_at`; a missing or zero value keeps the old one. A BACKLOG game with new playtime becomes PLAYING, unless the user chose its status. When Steam returns no
games, the library stays as it is. games, the library stays as it is.
A game the user added by hand for PC, Steam, Mac or Linux (or with no platform), whose IGDB game is the IGDB
game of a Steam appid the user owns for the first time (`game_edition`), becomes that Steam row. The row moves
from minus its IGDB id to the appid and keeps status, rating, notes, priority, status history, backlog
position, questions, set rows, PlayStation links and paid game rewards. Playtime and last played come from
Steam. A hand-added game on another platform stays and joins the Steam game's set. A user with such a game
gets the IGDB edition refresh before the save, so the mapping is known in time.
5. `TimeToBeatService` and `MoodService` ask IGDB for missing or old time-to-beat, mood and genre data, for Steam 5. `TimeToBeatService` and `MoodService` ask IGDB for missing or old time-to-beat, mood and genre data, for Steam
games and manual games. Hidden games only get their playtime updated. games and manual games. Hidden games only get their playtime updated.
6. After the commit, `LibrarySyncedEvent` starts `AchievementSyncService` and `SpeedrunService` (backlog games only) 6. After the commit, `LibrarySyncedEvent` starts `AchievementSyncService` and `SpeedrunService` (backlog games only)
@@ -414,6 +463,8 @@ and `WISHLIST` rows of `sync_change`) and the hand changes since the cursor. Kin
are left out, and the cursor moves anyway. A job every 10 minutes reports the hand changes alone once a user was quiet for 10 are left out, and the cursor moves anyway. A job every 10 minutes reports the hand changes alone once a user was quiet for 10
minutes; it never takes a sync result, which may still wait for its achievements. minutes; it never takes a sync result, which may still wait for its achievements.
`ALL_UNLOCKED`: the sync unlocked the last Steam achievement of a game that is not COMPLETED; Core never sets COMPLETED itself, the user confirms it in the sync details.
```mermaid ```mermaid
sequenceDiagram sequenceDiagram
participant W as LibrarySyncWorker participant W as LibrarySyncWorker
@@ -426,7 +477,7 @@ sequenceDiagram
P-->>W: ADDED, PLAYTIME, ACHIEVEMENTS (trophy gains) P-->>W: ADDED, PLAYTIME, ACHIEVEMENTS (trophy gains)
W->>D: sync_result, sync_change (ADDED, STATUS, WISHLIST, PSN changes) W->>D: sync_result, sync_change (ADDED, STATUS, WISHLIST, PSN changes)
W-->>A: LibrarySyncedEvent W-->>A: LibrarySyncedEvent
A->>D: sync_change (ACHIEVEMENTS) A->>D: sync_change (ACHIEVEMENTS, ALL_UNLOCKED)
A-->>F: LibraryActivityEvent A-->>F: LibraryActivityEvent
F->>D: lock follow_digest, count sync + hand changes F->>D: lock follow_digest, count sync + hand changes
F->>D: inbox_note per follower (public, not blocked, not deleted) F->>D: inbox_note per follower (public, not blocked, not deleted)
@@ -438,18 +489,20 @@ At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/l
### Background work ### Background work
Background work runs on two pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it): Background work runs on three pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it):
- The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view - The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view
starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with
a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user
starts it again. starts it again.
- The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun - The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and the speedrun
refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page refreshes (`@Async`). They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take
views' threads. A task is small (a user or a few appids), and none is dropped. the page views' threads. A task is small (a user or a few appids), and none is dropped.
- The speedrun drainer (`speedrunExecutor`, 1 thread) works the speedrun.com queue (`SpeedrunService`).
Imports have their own two threads, and `game_import` is their queue. Scheduled jobs (nightly sync, image repair, Imports run in two lanes, `game-import-match` and `game-import-save`, 1 thread each, queue 1; `game_import` is their
import watchdog) run on the scheduler. queue. Scheduled jobs run on the scheduler: nightly sync 23:55, image repair 01:30, rewards settle 02:30, inbox purge
03:15, merge-undo purge 03:45, import watchdog every minute, credit backfill every 30 s, follow digest every 10 min.
```mermaid ```mermaid
flowchart LR flowchart LR
@@ -457,10 +510,12 @@ flowchart LR
Sync["sync request"] --> Pool Sync["sync request"] --> Pool
Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"] Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
View -->|"speedrun"| FollowUp View -->|"speedrun"| FollowUp
FollowUp --> Drainer["speedrunExecutor<br/>1 thread"]
Pool --> Apis["Steam, IGDB, speedrun.com"] Pool --> Apis["Steam, IGDB, speedrun.com"]
FollowUp --> Apis FollowUp --> Apis
Drainer --> Apis
Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"] Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"] Cron["scheduler"] --> Jobs["nightly sync, image repair, rewards settle, inbox purge, merge-undo purge, import watchdog, credit backfill, follow digest"]
``` ```
Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks
@@ -647,11 +702,21 @@ A game the user adds by hand comes from IGDB (`ManualGameService`). Steam games
`user_game.source` (`GameSource`) says who owns the row. The sync never changes or removes a `MANUAL` row, with `user_game.source` (`GameSource`) says who owns the row. The sync never changes or removes a `MANUAL` row, with
one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes one exception: when the user later owns the game on Steam under the same appid, the sync takes the row over: it becomes
`STEAM`, and status, rating and notes stay. `PSN` rows (a PlayStation import, see below) live the same way. Only `MANUAL` and `PSN` rows can be edited (`PUT /api/library/games/{appid}/manual`) `STEAM`, and status, rating and notes stay. `PSN` rows (a PlayStation import, see below) live the same way. Only `MANUAL` and `PSN` rows can be edited (`PUT /api/library/games/{appid}/manual`)
or deleted (`DELETE /api/library/games/{appid}`). Time to beat uses the Steam appid for positive keys and the IGDB or removed (`DELETE /api/library/games/{appid}`, soft: it sets `removed_at`, the row keeps status, rating, notes
and history) and restored (`POST /api/library/games/{appid}/restore`, see the diagram). Time to beat uses the Steam appid for positive keys and the IGDB
id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get id for negative keys. Search and add check both keys of a game. A game added under minus its IGDB id can still get
a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two a second row when IGDB learns its Steam appid later and the user owns it on Steam; an auditor then merges the two
(see "Merging duplicates"). (see "Merging duplicates").
```mermaid
stateDiagram-v2
[*] --> Listed: add / sync
Listed --> Removed: DELETE /games/{appid} (MANUAL, PSN)
Removed --> Listed: POST /games/{appid}/restore or Undo
Removed: removed_at set, data kept
Removed: PSN sync skips the row
```
### Imports ### Imports
Many games at once (V22, `GameImportService`): a list, a CSV or several picks of the game search become one Many games at once (V22, `GameImportService`): a list, a CSV or several picks of the game search become one
@@ -723,8 +788,12 @@ project ships it; all values in `app.psn.*`). `PsnAccountService` in `library` h
- Every `LINKED` or `STEAM_OWNED` title also keeps its own platform, minutes and last played in `user_psn_title` (V44), - Every `LINKED` or `STEAM_OWNED` title also keeps its own platform, minutes and last played in `user_psn_title` (V44),
so the detail shows PlayStation play next to Steam's on a shared row. The Steam row's own playtime stays Steam's. so the detail shows PlayStation play next to Steam's on a shared row. The Steam row's own playtime stays Steam's.
- A `LINKED` title updates its row (`source = PSN`, not removed): playtime, `last_played_at`, `PLAYTIME` changes and the - A `LINKED` title updates its row (`source = PSN`, not removed): playtime, `last_played_at`, `PLAYTIME` changes and the
playtime history; `playtime_2weeks` is the sum of the history of the last 14 days. A game the user deleted stays playtime history; `playtime_2weeks` is the sum of the history of the last 14 days. The minutes a title grew since
`LINKED` and is not added again. A game also owned on Steam stays one row (the Steam row wins). its stored play are booked on its last played day (Berlin), only when its last played moved; a title linked in this
run is the start point and books nothing (V49 dropped the old wrong rows). A game the user removed stays
`LINKED` and the sync does not add it again; its row stays removed, so the sync skips it until the user restores it:
a restore takes the stored title play (`psn_title_play`) and the sync updates the row again. Flyway V46 gave games
removed before (soft delete) a removed `PSN` row and turned a `MANUAL` row re-added on a `LINKED` key back to `PSN`. A game also owned on Steam stays one row (the Steam row wins).
- The first run of a user (no `user_psn_title` yet) is a baseline like the first Steam sync: its adds are a count in - The first run of a user (no `user_psn_title` yet) is a baseline like the first Steam sync: its adds are a count in
`sync_result.imported_games`, not a list. PSN changes share the `sync_result` of the Steam part, so the banner and the `sync_result.imported_games`, not a list. PSN changes share the `sync_result` of the Steam part, so the banner and the
follow digest need no change. follow digest need no change.
@@ -775,7 +844,7 @@ other, then start them in this order (Core also starts without Auth and finds it
3. **Frontend**: in `ToBePlayed-Frontend`, `npm install`, then `npm run dev`, and open `http://localhost:5173` 3. **Frontend**: in `ToBePlayed-Frontend`, `npm install`, then `npm run dev`, and open `http://localhost:5173`
(`localhost`, not `127.0.0.1`, so the browser sends the login cookies to every local port). The development (`localhost`, not `127.0.0.1`, so the browser sends the login cookies to every local port). The development
values must be `VITE_API_URL=http://localhost:8080` and `VITE_CORE_API_URL=http://localhost:8090`. values must be `VITE_API_URL=http://localhost:8080` and `VITE_CORE_API_URL=http://localhost:8090`.
4. **Log in** with `dev@localhost` / `password123`. On the dashboard, start a Steam sync: the library fills with 8 4. **Log in** with `dev@localhost` / `password123`. On the dashboard, start a Steam sync: the library fills with 30
games, achievements, time to beat, moods, a wishlist and one speedrun game from recorded answers. games, achievements, time to beat, moods, a wishlist and one speedrun game from recorded answers.
What the `local` profile does: What the `local` profile does:
@@ -816,10 +885,11 @@ Needs Java 25, Docker and your own keys.
``` ```
Spring Boot's Docker Compose support starts PostgreSQL from `compose.yaml`, Flyway applies the migrations, and the Spring Boot's Docker Compose support starts PostgreSQL from `compose.yaml`, Flyway applies the migrations, and the
service listens on `http://localhost:8090`. Health: `/actuator/health`, metrics: `/actuator/prometheus`. service listens on `http://localhost:8090`. Health: `/actuator/health` (the only open actuator endpoint).
In the `dev` profile a request without an `Authorization` header gets a dummy user, so the API is usable without a In the `dev` profile a request without an `Authorization` header gets a dummy user, so the API is usable without a
running Auth service. running Auth service. There is no `application-dev.yml`: `dev` only swaps the JWT filter
(`DevJwtAuthenticationFilter`, `@Profile("dev")`); everything else is `application.yml`.
Configuration comes from environment variables: Configuration comes from environment variables:
@@ -830,7 +900,24 @@ Configuration comes from environment variables:
| `STEAM_API_KEY` | Steam Web API key | | `STEAM_API_KEY` | Steam Web API key |
| `IGDB_CLIENT_ID`, `IGDB_CLIENT_SECRET` | Twitch app for IGDB (optional) | | `IGDB_CLIENT_ID`, `IGDB_CLIENT_SECRET` | Twitch app for IGDB (optional) |
| `APP_PSN_TOKEN_KEY` | Key for the stored PlayStation tokens, `openssl rand -base64 32` (optional; empty = PlayStation link off) | | `APP_PSN_TOKEN_KEY` | Key for the stored PlayStation tokens, `openssl rand -base64 32` (optional; empty = PlayStation link off) |
| `DB_PASSWORD` | Local PostgreSQL password | | `APP_FRONTEND_URL` | CORS origin, default the production host |
| `APP_ACCOUNT_CLEANUP_DRY_RUN` | Default `true`: the purge and prune only count |
| `APP_BUILD_COMMIT`, `APP_BUILD_TIME` | Feed `GET /api/version`; defaults `dev` and empty. CI passes `GIT_SHA` and `BUILD_TIME` as Docker build args |
### Request timing log
`RequestTimingFilter` writes one INFO line to stdout per API request: method, handler path pattern, status and
duration, e.g. `GET /api/library/games 200 28 ms`. The pattern is `-` when no handler matched (e.g. a `401` before
routing). The line never holds ids, query strings, user data or tokens. `/actuator/**` is skipped. An exception
that leaves the filter chain is logged as `500`.
### Database connection settings
Every Core connection runs with `plan_cache_mode = force_generic_plan` and `prepareThreshold=1`
(`spring.datasource.hikari` in `application.yml`). Planning was 94% of the Dashboard DB time and prod never reused a
plan (few users, a new connection every 30 min). With a cached generic plan the Dashboard burst dropped from 443 to
229 ms on 2 CPUs (#49). Cost: the first burst on fresh connections costs about 100 ms more (cold start: 1101 -> 1205
ms), and a generic plan ignores parameter values.
## Tests ## Tests
@@ -8,7 +8,7 @@ import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
/** /**
* The two pools of {@link BackgroundPools}. A dropped task is a refresh of data with a TTL: the next view or sync * The pools of {@link BackgroundPools}. A dropped task is a refresh of data with a TTL: the next view or sync
* asks again. * asks again.
*/ */
@Configuration @Configuration
@@ -43,4 +43,14 @@ public class AsyncConfig {
executor.setThreadNamePrefix("sync-follow-up-"); executor.setThreadNamePrefix("sync-follow-up-");
return executor; return executor;
} }
// One thread: speedrun.com has one rate limit and the queue has one drainer. The queue holds only drain triggers.
@Bean(name = BackgroundPools.SPEEDRUN)
public ThreadPoolTaskExecutor speedrunExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(1);
executor.setMaxPoolSize(1);
executor.setThreadNamePrefix("speedrun-");
return executor;
}
} }
@@ -0,0 +1,54 @@
package core_service.configuration;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.jspecify.annotations.NonNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.servlet.HandlerMapping;
import java.io.IOException;
/**
* One INFO line per request: method, handler pattern, status, duration. The pattern (e.g. {@code /api/games/{appid}})
* stands for the path, so the line holds no ids, query, user or credentials. A request no handler took shows {@code -}.
*/
@Component
@Order(Ordered.HIGHEST_PRECEDENCE)
public class RequestTimingFilter extends OncePerRequestFilter {
private static final Logger LOG = LoggerFactory.getLogger(RequestTimingFilter.class);
@Override
protected void doFilterInternal(
@NonNull HttpServletRequest request,
@NonNull HttpServletResponse response,
@NonNull FilterChain filterChain
) throws ServletException, IOException {
long start = System.nanoTime();
int status = HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
try {
filterChain.doFilter(request, response);
status = response.getStatus();
} finally {
// An exception that leaves the chain ends as a 500; the container sets it only after this filter.
Object pattern = request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE);
LOG.info("{} {} {} {} ms", request.getMethod(), pattern != null ? pattern : "-", status,
(System.nanoTime() - start) / 1_000_000);
}
}
/**
* The health check calls the actuator every 30 s; its lines would flood the log.
*/
@Override
protected boolean shouldNotFilter(@NonNull HttpServletRequest request) {
return request.getRequestURI().startsWith("/actuator/");
}
}
@@ -1,14 +1,17 @@
package core_service.modules.follow.api; package core_service.modules.follow.api;
import core_service.modules.follow.api.dto.FollowsDto; import core_service.modules.follow.api.dto.FollowsDto;
import core_service.modules.follow.api.dto.PlayersDto;
import core_service.modules.follow.domain.FollowEntry; import core_service.modules.follow.domain.FollowEntry;
import core_service.modules.follow.domain.FollowResult; import core_service.modules.follow.domain.FollowResult;
import core_service.modules.follow.domain.Follows; import core_service.modules.follow.domain.Follows;
import core_service.modules.follow.domain.PlayerSearch;
import core_service.modules.follow.service.FollowService; import core_service.modules.follow.service.FollowService;
import core_service.shared.api.ErrorResponse; import core_service.shared.api.ErrorResponse;
import core_service.shared.security.UserPrincipal; import core_service.shared.security.UserPrincipal;
import jakarta.validation.constraints.Size; import jakarta.validation.constraints.Size;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity; import org.springframework.http.ResponseEntity;
import org.springframework.security.core.annotation.AuthenticationPrincipal; import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.DeleteMapping; import org.springframework.web.bind.annotation.DeleteMapping;
@@ -16,6 +19,7 @@ import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PutMapping; import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController; import org.springframework.web.bind.annotation.RestController;
import java.util.List; import java.util.List;
@@ -33,6 +37,20 @@ public class FollowController {
return ResponseEntity.ok(new FollowsDto(toDto(follows.following()), toDto(follows.followers()))); return ResponseEntity.ok(new FollowsDto(toDto(follows.following()), toDto(follows.followers())));
} }
/**
* The caller's own name is a normal hit.
*/
@GetMapping("/players")
public ResponseEntity<?> players(@AuthenticationPrincipal UserPrincipal principal,
@RequestParam("q") @Size(max = 100) String q) {
PlayerSearch search = followService.searchPlayers(principal.getIdAsLong(), q);
if (search.limited()) {
return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
.body(new ErrorResponse("TOO_MANY_SEARCHES", "Too many searches"));
}
return ResponseEntity.ok(new PlayersDto(search.names().stream().map(PlayersDto.Player::new).toList()));
}
@PutMapping("/by-name/{displayName}") @PutMapping("/by-name/{displayName}")
public ResponseEntity<?> follow(@AuthenticationPrincipal UserPrincipal principal, public ResponseEntity<?> follow(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("displayName") @Size(max = 100) String displayName) { @PathVariable("displayName") @Size(max = 100) String displayName) {
@@ -63,6 +81,6 @@ public class FollowController {
} }
private static List<FollowsDto.Person> toDto(List<FollowEntry> entries) { private static List<FollowsDto.Person> toDto(List<FollowEntry> entries) {
return entries.stream().map(entry -> new FollowsDto.Person(entry.id(), entry.displayName())).toList(); return entries.stream().map(entry -> new FollowsDto.Person(entry.id(), entry.displayName(), entry.lastActivityAt())).toList();
} }
} }
@@ -1,5 +1,6 @@
package core_service.modules.follow.api.dto; package core_service.modules.follow.api.dto;
import java.time.Instant;
import java.util.List; import java.util.List;
/** /**
@@ -8,9 +9,10 @@ import java.util.List;
public record FollowsDto(List<Person> following, List<Person> followers) { public record FollowsDto(List<Person> following, List<Person> followers) {
/** /**
* @param id the follow row, to end it; not a user id * @param id the follow row, to end it; not a user id
* @param displayName null for a follower with a private profile * @param displayName null for a follower with a private profile
* @param lastActivityAt following only: when the newest feed message about this person arrived; null without one
*/ */
public record Person(long id, String displayName) { public record Person(long id, String displayName, Instant lastActivityAt) {
} }
} }
@@ -0,0 +1,15 @@
package core_service.modules.follow.api.dto;
import java.util.List;
/**
* Public profiles found by name: names that start with the query first, then by name.
*/
public record PlayersDto(List<Player> results) {
/**
* @param displayName the key of the profile URL
*/
public record Player(String displayName) {
}
}
@@ -9,8 +9,8 @@ import java.util.List;
*/ */
public record Digest(int newGames, int achievements, int statusChanges, int wishlistAdds) { public record Digest(int newGames, int achievements, int statusChanges, int wishlistAdds) {
private static final String SUMMARY = "SUMMARY"; static final String SUMMARY = "SUMMARY";
private static final List<String> STATUS_SECTIONS = List.of("STATUS", "PLAYING", "FINISHED"); static final List<String> STATUS_SECTIONS = List.of("STATUS", "PLAYING", "FINISHED");
public boolean isEmpty() { public boolean isEmpty() {
return newGames == 0 && achievements == 0 && statusChanges == 0 && wishlistAdds == 0; return newGames == 0 && achievements == 0 && statusChanges == 0 && wishlistAdds == 0;
@@ -0,0 +1,28 @@
package core_service.modules.follow.domain;
/**
* One line behind a digest count. Names are snapshots. The kind is a plain string: the inbox module has its own enum.
*/
public record DigestItem(String kind, long appid, String gameName, String itemName, String fromStatus, String toStatus) {
public static final String NEW_GAME = "NEW_GAME";
public static final String ACHIEVEMENT = "ACHIEVEMENT";
public static final String STATUS = "STATUS";
public static final String WISHLIST = "WISHLIST";
public static DigestItem newGame(long appid, String gameName) {
return new DigestItem(NEW_GAME, appid, gameName, null, null, null);
}
public static DigestItem achievement(long appid, String gameName, String itemName) {
return new DigestItem(ACHIEVEMENT, appid, gameName, itemName, null, null);
}
public static DigestItem status(long appid, String gameName, String from, String to) {
return new DigestItem(STATUS, appid, gameName, null, from, to);
}
public static DigestItem wishlist(long appid, String gameName) {
return new DigestItem(WISHLIST, appid, gameName, null, null, null);
}
}
@@ -0,0 +1,66 @@
package core_service.modules.follow.domain;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
/**
* The lines behind a {@link Digest}, in the order they are shown.
*/
public record DigestItems(List<DigestItem> items) {
public DigestItems plus(DigestItems other) {
List<DigestItem> all = new ArrayList<>(items);
all.addAll(other.items);
return new DigestItems(all);
}
/**
* Same rule as {@link Digest#visibleOn}.
*
* @param sections the profile parts the user shows, null = all
*/
public DigestItems visibleOn(Collection<String> sections) {
if (sections == null) {
return this;
}
boolean summary = sections.contains(Digest.SUMMARY);
boolean status = Digest.STATUS_SECTIONS.stream().anyMatch(sections::contains);
return new DigestItems(items.stream().filter(item -> switch (item.kind()) {
case DigestItem.NEW_GAME, DigestItem.ACHIEVEMENT -> summary;
case DigestItem.STATUS -> status;
default -> true;
}).toList());
}
/**
* One STATUS line per game: the first old status to the last new one, dropped when they are equal. The line keeps
* the place of the first move of its game.
*/
public DigestItems collapseStatus() {
Map<Long, String> lastTo = new HashMap<>();
for (DigestItem item : items) {
if (DigestItem.STATUS.equals(item.kind())) {
lastTo.put(item.appid(), item.toStatus());
}
}
List<DigestItem> result = new ArrayList<>();
Set<Long> seen = new HashSet<>();
for (DigestItem item : items) {
if (!DigestItem.STATUS.equals(item.kind())) {
result.add(item);
} else if (seen.add(item.appid())) {
String to = lastTo.get(item.appid());
if (!Objects.equals(item.fromStatus(), to)) {
result.add(DigestItem.status(item.appid(), item.gameName(), item.fromStatus(), to));
}
}
}
return new DigestItems(result);
}
}
@@ -1,8 +1,11 @@
package core_service.modules.follow.domain; package core_service.modules.follow.domain;
import java.time.Instant;
/** /**
* @param id the follow row, not a user id * @param id the follow row, not a user id
* @param displayName null for a follower with a private profile * @param displayName null for a follower with a private profile
* @param lastActivityAt the newest digest note the caller got about this user; null for followers and without a note
*/ */
public record FollowEntry(long id, String displayName) { public record FollowEntry(long id, String displayName, Instant lastActivityAt) {
} }
@@ -0,0 +1,17 @@
package core_service.modules.follow.domain;
import java.util.List;
/**
* The answer of a player search: the names found, or "limited" when the caller searched too often.
*/
public record PlayerSearch(List<String> names, boolean limited) {
public static PlayerSearch tooMany() {
return new PlayerSearch(List.of(), true);
}
public static PlayerSearch of(List<String> names) {
return new PlayerSearch(names, false);
}
}
@@ -2,15 +2,19 @@ package core_service.modules.follow.repo;
import core_service.modules.follow.domain.Cursor; import core_service.modules.follow.domain.Cursor;
import core_service.modules.follow.domain.Digest; import core_service.modules.follow.domain.Digest;
import core_service.modules.follow.domain.DigestItem;
import core_service.modules.follow.domain.DigestItems;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; import org.springframework.jdbc.core.namedparam.MapSqlParameterSource;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.jdbc.core.namedparam.SqlParameterSource;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
import java.sql.Array; import java.sql.Array;
import java.sql.Timestamp; import java.sql.Timestamp;
import java.time.Instant; import java.time.Instant;
import java.util.Arrays; import java.util.Arrays;
import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -22,6 +26,8 @@ import java.util.Optional;
@RequiredArgsConstructor @RequiredArgsConstructor
public class FollowDigestRepository { public class FollowDigestRepository {
static final int MAX_ACHIEVEMENTS_PER_GAME = 20;
private final NamedParameterJdbcTemplate jdbc; private final NamedParameterJdbcTemplate jdbc;
/** /**
@@ -41,20 +47,20 @@ public class FollowDigestRepository {
} }
/** /**
* Hidden games never count. Wishlist adds are counted apart, see {@link #countWishlist}. * Hidden and removed games never count. Wishlist adds are counted apart, see {@link #countWishlist}.
*/ */
public Digest countSync(long userId) { public Digest countSync(long userId) {
return jdbc.queryForObject(""" return jdbc.queryForObject("""
select select
(select count(distinct c.appid) from sync_change c (select count(distinct c.appid) from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
where c.user_id = :userId and c.kind = 'ADDED' and ug.hidden_at is null) as new_games, where c.user_id = :userId and c.kind = 'ADDED' and ug.hidden_at is null and ug.removed_at is null) as new_games,
(select coalesce(sum(c.amount), 0) from sync_change c (select coalesce(sum(c.amount), 0) from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
where c.user_id = :userId and c.kind = 'ACHIEVEMENTS' and ug.hidden_at is null) as achievements, where c.user_id = :userId and c.kind = 'ACHIEVEMENTS' and ug.hidden_at is null and ug.removed_at is null) as achievements,
(select count(*) from sync_change c (select count(*) from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
where c.user_id = :userId and c.kind = 'STATUS' and ug.hidden_at is null) as status_changes""", where c.user_id = :userId and c.kind = 'STATUS' and ug.hidden_at is null and ug.removed_at is null) as status_changes""",
user(userId), user(userId),
(rs, i) -> new Digest(rs.getInt("new_games"), rs.getInt("achievements"), rs.getInt("status_changes"), 0)); (rs, i) -> new Digest(rs.getInt("new_games"), rs.getInt("achievements"), rs.getInt("status_changes"), 0));
} }
@@ -83,6 +89,17 @@ public class FollowDigestRepository {
return max != null ? max : 0; return max != null ? max : 0;
} }
/**
* The newest hand change after {@code afterId}, or {@code afterId} when there is none. A hand-only digest stops
* there, so the status rows of a sync stay for the digest of that sync.
*/
public long maxHandStatusChangeId(long userId, long afterId) {
Long max = jdbc.queryForObject("select max(id) from user_game_status_change "
+ "where user_id = :userId and by_user and id > :afterId",
user(userId).addValue("afterId", afterId), Long.class);
return max != null ? max : afterId;
}
/** /**
* The changes the user made by hand: no old status = a game added by hand, else a status change. * The changes the user made by hand: no old status = a game added by hand, else a status change.
*/ */
@@ -93,11 +110,95 @@ public class FollowDigestRepository {
from user_game_status_change c from user_game_status_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
where c.user_id = :userId and c.by_user and c.id > :afterId and c.id <= :upToId where c.user_id = :userId and c.by_user and c.id > :afterId and c.id <= :upToId
and ug.hidden_at is null""", and ug.hidden_at is null and ug.removed_at is null""",
user(userId).addValue("afterId", afterId).addValue("upToId", upToId), user(userId).addValue("afterId", afterId).addValue("upToId", upToId),
(rs, i) -> new Digest(rs.getInt("new_games"), 0, rs.getInt("status_changes"), 0)); (rs, i) -> new Digest(rs.getInt("new_games"), 0, rs.getInt("status_changes"), 0));
} }
/**
* The lines behind {@link #countSync}, same joins and filters. Achievement names are not stored by the sync, only
* how many: the newest unlocks of the game (Steam and PlayStation together) stand for the new ones, at most
* {@link #MAX_ACHIEVEMENTS_PER_GAME} per game.
*/
public DigestItems findSyncItems(long userId) {
List<DigestItem> items = new ArrayList<>(findSyncNewGames(userId));
items.addAll(findSyncAchievements(userId));
return new DigestItems(items);
}
private List<DigestItem> findSyncNewGames(long userId) {
return jdbc.query("""
select distinct c.appid, sg.name from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
left join steam_game sg on sg.appid = c.appid
where c.user_id = :userId and c.kind = 'ADDED' and ug.hidden_at is null and ug.removed_at is null
order by sg.name, c.appid""", user(userId), (rs, i) -> DigestItem.newGame(rs.getLong("appid"), rs.getString("name")));
}
private List<DigestItem> findSyncAchievements(long userId) {
return jdbc.query("""
with gained as (
select c.appid, sum(c.amount) as amount from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
where c.user_id = :userId and c.kind = 'ACHIEVEMENTS' and ug.hidden_at is null and ug.removed_at is null
group by c.appid),
unlocks as (
select a.appid, coalesce(s.display_name, a.api_name) as name, a.unlocked_at as at
from user_game_achievement a
left join game_achievement_schema s on s.appid = a.appid and s.api_name = a.api_name
where a.user_id = :userId and a.unlocked
union all
select st.appid, coalesce(t.name, 'Trophy ' || t.trophy_id), ut.earned_at
from user_psn_trophy ut
join user_psn_trophy_set st on st.user_id = ut.user_id and st.np_communication_id = ut.np_communication_id
join psn_trophy t on t.np_communication_id = ut.np_communication_id and t.trophy_id = ut.trophy_id
where ut.user_id = :userId),
ranked as (
select u.appid, u.name, row_number() over (partition by u.appid order by u.at desc nulls last, u.name) as rn
from unlocks u join gained g on g.appid = u.appid)
select r.appid, sg.name as game_name, r.name from ranked r
join gained g on g.appid = r.appid
left join steam_game sg on sg.appid = r.appid
where r.rn <= least(g.amount, :cap)
order by sg.name, r.appid, r.rn""", user(userId).addValue("cap", MAX_ACHIEVEMENTS_PER_GAME),
(rs, i) -> DigestItem.achievement(rs.getLong("appid"), rs.getString("game_name"), rs.getString("name")));
}
/**
* The lines behind {@link #countHand}, same filters. With {@code withSyncMoves} the status moves of the sync are
* read from the same log, so a game's moves by hand and by sync stay in time order for the collapse.
*/
public DigestItems findHandItems(long userId, long afterId, long upToId, boolean withSyncMoves) {
return new DigestItems(jdbc.query("""
select c.appid, sg.name, c.from_status, c.to_status
from user_game_status_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
left join steam_game sg on sg.appid = c.appid
where c.user_id = :userId and c.id > :afterId and c.id <= :upToId
and (c.by_user or (:withSyncMoves and c.from_status is not null))
and ug.hidden_at is null and ug.removed_at is null
order by c.id""",
user(userId).addValue("afterId", afterId).addValue("upToId", upToId).addValue("withSyncMoves", withSyncMoves),
(rs, i) -> rs.getString("from_status") == null
? DigestItem.newGame(rs.getLong("appid"), rs.getString("name"))
: DigestItem.status(rs.getLong("appid"), rs.getString("name"), rs.getString("from_status"),
rs.getString("to_status"))));
}
/**
* The lines behind {@link #countWishlist}, same id range.
*/
public DigestItems findWishlistItems(long userId, long afterId, long upToId) {
return new DigestItems(jdbc.query("""
select c.appid, coalesce(w.name, sg.name, 'App ' || c.appid) as name from sync_change c
left join wishlist_game w on w.user_id = c.user_id and w.appid = c.appid
left join steam_game sg on sg.appid = c.appid
where c.user_id = :userId and c.kind = 'WISHLIST' and c.id > :afterId and c.id <= :upToId
order by c.id""",
user(userId).addValue("afterId", afterId).addValue("upToId", upToId),
(rs, i) -> DigestItem.wishlist(rs.getLong("appid"), rs.getString("name"))));
}
/** /**
* @return null when there is no row or the user did not send sections: nothing is filtered * @return null when there is no row or the user did not send sections: nothing is filtered
*/ */
@@ -115,10 +216,10 @@ public class FollowDigestRepository {
* One note per follower, only while the followed user is public, not blocked and not deleted and the follower is * One note per follower, only while the followed user is public, not blocked and not deleted and the follower is
* not deleted. A name is stored as it is now. * not deleted. A name is stored as it is now.
* *
* @return how many notes were written * @return the ids of the notes written
*/ */
public int insertDigestNotes(long userId, Digest digest) { public List<Long> insertDigestNotes(long userId, Digest digest) {
return jdbc.update(""" return jdbc.query("""
insert into inbox_note (user_id, kind, subject_user_id, subject_name, new_games, achievements, insert into inbox_note (user_id, kind, subject_user_id, subject_name, new_games, achievements,
status_changes, wishlist_adds) status_changes, wishlist_adds)
select f.user_id, 'DIGEST', f.followed_user_id, select f.user_id, 'DIGEST', f.followed_user_id,
@@ -128,9 +229,32 @@ public class FollowDigestRepository {
where f.followed_user_id = :userId where f.followed_user_id = :userId
and exists (select 1 from public_user p where p.user_id = f.followed_user_id) and exists (select 1 from public_user p where p.user_id = f.followed_user_id)
and not exists (select 1 from blocked_user b where b.user_id = f.followed_user_id) and not exists (select 1 from blocked_user b where b.user_id = f.followed_user_id)
and not exists (select 1 from deleted_user d where d.user_id in (f.followed_user_id, f.user_id))""", and not exists (select 1 from deleted_user d where d.user_id in (f.followed_user_id, f.user_id))
returning id""",
user(userId).addValue("newGames", digest.newGames()).addValue("achievements", digest.achievements()) user(userId).addValue("newGames", digest.newGames()).addValue("achievements", digest.achievements())
.addValue("statusChanges", digest.statusChanges()).addValue("wishlistAdds", digest.wishlistAdds())); .addValue("statusChanges", digest.statusChanges()).addValue("wishlistAdds", digest.wishlistAdds()),
(rs, i) -> rs.getLong("id"));
}
/**
* One copy of the lines per note: the note owns them and they go with it.
*/
public void insertDigestItems(List<Long> noteIds, DigestItems items) {
if (noteIds.isEmpty() || items.items().isEmpty()) {
return;
}
List<SqlParameterSource> rows = new ArrayList<>();
for (long noteId : noteIds) {
for (DigestItem item : items.items()) {
rows.add(new MapSqlParameterSource("noteId", noteId).addValue("kind", item.kind())
.addValue("appid", item.appid()).addValue("gameName", item.gameName())
.addValue("itemName", item.itemName()).addValue("fromStatus", item.fromStatus())
.addValue("toStatus", item.toStatus()));
}
}
jdbc.batchUpdate("insert into inbox_note_detail (note_id, kind, appid, game_name, item_name, from_status, to_status) "
+ "values (:noteId, :kind, :appid, :gameName, :itemName, :fromStatus, :toStatus)",
rows.toArray(new SqlParameterSource[0]));
} }
public void moveCursor(long userId, Instant syncStartedAt, long statusChangeId, long wishlistChangeId) { public void moveCursor(long userId, Instant syncStartedAt, long statusChangeId, long wishlistChangeId) {
@@ -1,12 +1,14 @@
package core_service.modules.follow.repo; package core_service.modules.follow.repo;
import core_service.modules.follow.domain.FollowEntry; import core_service.modules.follow.domain.FollowEntry;
import core_service.shared.sql.LikePattern;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.jdbc.core.RowMapper; import org.springframework.jdbc.core.RowMapper;
import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; import org.springframework.jdbc.core.namedparam.MapSqlParameterSource;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
import java.sql.Timestamp;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -19,9 +21,13 @@ import java.util.Optional;
public class FollowRepository { public class FollowRepository {
static final int LIST_LIMIT = 500; static final int LIST_LIMIT = 500;
public static final int SEARCH_LIMIT = 20;
private static final RowMapper<FollowEntry> ENTRY = (rs, i) -> new FollowEntry(rs.getLong("id"), private static final RowMapper<FollowEntry> ENTRY = (rs, i) -> {
rs.getString("display_name")); Timestamp lastActivityAt = rs.getTimestamp("last_activity_at");
return new FollowEntry(rs.getLong("id"), rs.getString("display_name"),
lastActivityAt == null ? null : lastActivityAt.toInstant());
};
private final NamedParameterJdbcTemplate jdbc; private final NamedParameterJdbcTemplate jdbc;
@@ -41,6 +47,23 @@ public class FollowRepository {
return ids.size() == 1 ? Optional.of(ids.get(0)) : Optional.empty(); return ids.size() == 1 ? Optional.of(ids.get(0)) : Optional.empty();
} }
/**
* Public names that contain the query, case-insensitive; names that start with it come first. Same visibility
* rules as {@link #findFollowable}. Wildcards in the query match themselves.
*/
public List<String> searchPublicNames(String query, int limit) {
String escaped = LikePattern.escapeLower(query);
return jdbc.queryForList("""
select a.display_name from account_name a
join public_user p on p.user_id = a.user_id
where lower(a.display_name) like :pattern escape '\\'
and not exists (select 1 from blocked_user b where b.user_id = a.user_id)
and not exists (select 1 from deleted_user d where d.user_id = a.user_id)
order by (lower(a.display_name) like :prefix escape '\\') desc, lower(a.display_name)
limit :limit""", new MapSqlParameterSource("pattern", "%" + escaped + "%")
.addValue("prefix", escaped + "%").addValue("limit", limit), String.class);
}
/** /**
* Checks the visibility again in the same statement, so a user who went private in between gets no follower. * Checks the visibility again in the same statement, so a user who went private in between gets no follower.
* *
@@ -91,7 +114,11 @@ public class FollowRepository {
public List<FollowEntry> findFollowing(long userId) { public List<FollowEntry> findFollowing(long userId) {
return jdbc.query(""" return jdbc.query("""
select f.id, a.display_name from follow f select f.id, a.display_name,
(select max(n.created_at) from inbox_note n
where n.user_id = f.user_id and n.subject_user_id = f.followed_user_id
and n.kind = 'DIGEST') as last_activity_at
from follow f
left join account_name a on a.user_id = f.followed_user_id left join account_name a on a.user_id = f.followed_user_id
where f.user_id = :userId where f.user_id = :userId
order by lower(a.display_name), f.id order by lower(a.display_name), f.id
@@ -103,7 +130,8 @@ public class FollowRepository {
*/ */
public List<FollowEntry> findFollowers(long userId) { public List<FollowEntry> findFollowers(long userId) {
return jdbc.query(""" return jdbc.query("""
select f.id, case when p.user_id is not null then a.display_name end as display_name select f.id, case when p.user_id is not null then a.display_name end as display_name,
null::timestamptz as last_activity_at
from follow f from follow f
left join public_user p on p.user_id = f.user_id left join public_user p on p.user_id = f.user_id
left join account_name a on a.user_id = f.user_id left join account_name a on a.user_id = f.user_id
@@ -2,6 +2,7 @@ package core_service.modules.follow.service;
import core_service.modules.follow.domain.Cursor; import core_service.modules.follow.domain.Cursor;
import core_service.modules.follow.domain.Digest; import core_service.modules.follow.domain.Digest;
import core_service.modules.follow.domain.DigestItems;
import core_service.modules.follow.repo.FollowDigestRepository; import core_service.modules.follow.repo.FollowDigestRepository;
import core_service.shared.event.LibraryActivityEvent; import core_service.shared.event.LibraryActivityEvent;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@@ -19,7 +20,8 @@ import java.util.Optional;
/** /**
* One inbox note per follower and batch. A batch is a sync with its achievement run, plus the hand changes since the * One inbox note per follower and batch. A batch is a sync with its achievement run, plus the hand changes since the
* last batch. The cursor moves in every case, so what the profile hides is never reported later. * last batch. The cursor moves in every case, so what the profile hides is never reported later. Each note also gets
* the lines behind its counts (status moves collapsed per game).
*/ */
@Service @Service
@Slf4j @Slf4j
@@ -67,23 +69,31 @@ public class FollowDigestService {
return; return;
} }
Cursor cursor = locked.get(); Cursor cursor = locked.get();
long upTo = digestRepository.maxStatusChangeId(userId); long upTo = withSync ? digestRepository.maxStatusChangeId(userId)
: digestRepository.maxHandStatusChangeId(userId, cursor.statusChangeId());
Digest digest = digestRepository.countHand(userId, cursor.statusChangeId(), upTo); Digest digest = digestRepository.countHand(userId, cursor.statusChangeId(), upTo);
Instant syncStartedAt = cursor.syncStartedAt(); Instant syncStartedAt = cursor.syncStartedAt();
long wishlistUpTo = cursor.wishlistChangeId(); long wishlistUpTo = cursor.wishlistChangeId();
Optional<Instant> current = withSync ? digestRepository.findSyncStartedAt(userId) : Optional.empty();
boolean newSync = current.isPresent() && (syncStartedAt == null || current.get().isAfter(syncStartedAt));
DigestItems items = digestRepository.findHandItems(userId, cursor.statusChangeId(), upTo, newSync);
if (withSync) { if (withSync) {
Optional<Instant> current = digestRepository.findSyncStartedAt(userId); if (newSync) {
if (current.isPresent() && (syncStartedAt == null || current.get().isAfter(syncStartedAt))) {
digest = digest.plus(digestRepository.countSync(userId)); digest = digest.plus(digestRepository.countSync(userId));
items = items.plus(digestRepository.findSyncItems(userId));
syncStartedAt = current.get(); syncStartedAt = current.get();
} }
wishlistUpTo = digestRepository.maxWishlistChangeId(userId); wishlistUpTo = digestRepository.maxWishlistChangeId(userId);
digest = digest.plus(new Digest(0, 0, 0, digest = digest.plus(new Digest(0, 0, 0,
digestRepository.countWishlist(userId, cursor.wishlistChangeId(), wishlistUpTo))); digestRepository.countWishlist(userId, cursor.wishlistChangeId(), wishlistUpTo)));
items = items.plus(digestRepository.findWishlistItems(userId, cursor.wishlistChangeId(), wishlistUpTo));
} }
digest = digest.visibleOn(digestRepository.findPublicSections(userId)); List<String> sections = digestRepository.findPublicSections(userId);
digest = digest.visibleOn(sections);
items = items.visibleOn(sections).collapseStatus();
if (!digest.isEmpty()) { if (!digest.isEmpty()) {
digestRepository.insertDigestNotes(userId, digest); List<Long> noteIds = digestRepository.insertDigestNotes(userId, digest);
digestRepository.insertDigestItems(noteIds, items);
} }
digestRepository.moveCursor(userId, syncStartedAt, upTo, wishlistUpTo); digestRepository.moveCursor(userId, syncStartedAt, upTo, wishlistUpTo);
}); });
@@ -2,6 +2,7 @@ package core_service.modules.follow.service;
import core_service.modules.follow.domain.FollowResult; import core_service.modules.follow.domain.FollowResult;
import core_service.modules.follow.domain.Follows; import core_service.modules.follow.domain.Follows;
import core_service.modules.follow.domain.PlayerSearch;
import core_service.modules.follow.repo.FollowRepository; import core_service.modules.follow.repo.FollowRepository;
import core_service.shared.security.RequiresRole; import core_service.shared.security.RequiresRole;
import core_service.shared.security.Role; import core_service.shared.security.Role;
@@ -9,6 +10,7 @@ import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional; import org.springframework.transaction.annotation.Transactional;
import java.util.List;
import java.util.Optional; import java.util.Optional;
@Service @Service
@@ -17,8 +19,10 @@ import java.util.Optional;
public class FollowService { public class FollowService {
static final int MAX_FOLLOWING = 500; static final int MAX_FOLLOWING = 500;
static final int MIN_QUERY_LENGTH = 2;
private final FollowRepository followRepository; private final FollowRepository followRepository;
private final PlayerSearchLimiter limiter;
/** /**
* Unknown, private, blocked and deleted names all answer NOT_FOUND, so a name cannot be probed. * Unknown, private, blocked and deleted names all answer NOT_FOUND, so a name cannot be probed.
@@ -57,4 +61,20 @@ public class FollowService {
public Follows follows(long userId) { public Follows follows(long userId) {
return new Follows(followRepository.findFollowing(userId), followRepository.findFollowers(userId)); return new Follows(followRepository.findFollowing(userId), followRepository.findFollowers(userId));
} }
/**
* A query shorter than two characters finds nothing and does not count against the limit.
*/
@RequiresRole(Role.USER)
@Transactional(readOnly = true)
public PlayerSearch searchPlayers(long userId, String query) {
String stripped = query.strip();
if (stripped.length() < MIN_QUERY_LENGTH) {
return PlayerSearch.of(List.of());
}
if (!limiter.allow(userId)) {
return PlayerSearch.tooMany();
}
return PlayerSearch.of(followRepository.searchPublicNames(stripped, FollowRepository.SEARCH_LIMIT));
}
} }
@@ -0,0 +1,32 @@
package core_service.modules.follow.service;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RateLimiterConfig;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import java.time.Duration;
import java.util.concurrent.ConcurrentHashMap;
/**
* Searches per user and minute. One limiter per user who searched since boot stays in memory: small, and bounded by
* the number of users. The limit holds per instance.
*/
@Component
public class PlayerSearchLimiter {
private final RateLimiterConfig config;
private final ConcurrentHashMap<Long, RateLimiter> limiters = new ConcurrentHashMap<>();
public PlayerSearchLimiter(@Value("${app.follow.search-per-minute:30}") int perMinute) {
this.config = RateLimiterConfig.custom()
.limitForPeriod(perMinute)
.limitRefreshPeriod(Duration.ofMinutes(1))
.timeoutDuration(Duration.ZERO)
.build();
}
public boolean allow(long userId) {
return limiters.computeIfAbsent(userId, id -> RateLimiter.of("player-search-" + id, config)).acquirePermission();
}
}
@@ -30,6 +30,7 @@ import java.util.Map;
import java.util.Optional; import java.util.Optional;
import java.util.Set; import java.util.Set;
import java.util.TreeSet; import java.util.TreeSet;
import java.util.function.Function;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import java.util.stream.Stream; import java.util.stream.Stream;
@@ -77,6 +78,9 @@ public class IgdbApiService {
*/ */
public record Game(long igdbId, String name, Integer year, String imageUrl, List<String> platforms, Long steamAppid) {} public record Game(long igdbId, String name, Integer year, String imageUrl, List<String> platforms, Long steamAppid) {}
/** @param year null when IGDB knows no release date */
public record NameAndYear(String name, Integer year) {}
// IGDB's own order puts obscure entries first, so Core asks for more and sorts by popularity itself. // IGDB's own order puts obscure entries first, so Core asks for more and sorts by popularity itself.
private static final int SEARCH_FETCH_LIMIT = 50; private static final int SEARCH_FETCH_LIMIT = 50;
private static final int SEARCH_LIMIT = 20; private static final int SEARCH_LIMIT = 20;
@@ -382,18 +386,31 @@ public class IgdbApiService {
* Like {@link #getReleaseDates}, but for IGDB game ids. * Like {@link #getReleaseDates}, but for IGDB game ids.
*/ */
public Optional<Map<Long, LocalDate>> getReleaseDatesByGameIds(Collection<Long> igdbIds) { public Optional<Map<Long, LocalDate>> getReleaseDatesByGameIds(Collection<Long> igdbIds) {
return inBatches(igdbIds, "release dates", this::datesOf);
}
/**
* The name and release year of IGDB games by id. Games IGDB does not know, or without a name, are missing. Empty
* when the adapter is off, nothing was asked or a request failed.
*/
public Optional<Map<Long, NameAndYear>> getNamesAndYears(Collection<Long> igdbIds) {
return inBatches(igdbIds, "game names", this::namesAndYearsOf);
}
private <T> Optional<Map<Long, T>> inBatches(Collection<Long> igdbIds, String what,
Function<List<Long>, Map<Long, T>> batchQuery) {
if (!isEnabled() || igdbIds.isEmpty()) { if (!isEnabled() || igdbIds.isEmpty()) {
return Optional.empty(); return Optional.empty();
} }
try { try {
List<Long> ids = List.copyOf(igdbIds); List<Long> ids = List.copyOf(igdbIds);
Map<Long, LocalDate> result = new HashMap<>(); Map<Long, T> result = new HashMap<>();
for (int from = 0; from < ids.size(); from += BATCH) { for (int from = 0; from < ids.size(); from += BATCH) {
result.putAll(datesOf(ids.subList(from, Math.min(from + BATCH, ids.size())))); result.putAll(batchQuery.apply(ids.subList(from, Math.min(from + BATCH, ids.size()))));
} }
return Optional.of(result); return Optional.of(result);
} catch (Exception e) { } catch (Exception e) {
log.error("Error communicating with IGDB for release dates", e); log.error("Error communicating with IGDB for {}", what, e);
return Optional.empty(); return Optional.empty();
} }
} }
@@ -601,10 +618,7 @@ public class IgdbApiService {
if (igdbIds.isEmpty()) { if (igdbIds.isEmpty()) {
return Map.of(); return Map.of();
} }
String gameIds = igdbIds.stream().map(String::valueOf).collect(Collectors.joining(",")); IgdbResponses.Game[] games = gamesById(igdbIds, "first_release_date");
IgdbResponses.Game[] games = query("/games",
"fields first_release_date; where id = (" + gameIds + "); limit " + LIMIT + ";",
IgdbResponses.Game[].class);
Map<Long, LocalDate> result = new HashMap<>(); Map<Long, LocalDate> result = new HashMap<>();
for (IgdbResponses.Game game : games) { for (IgdbResponses.Game game : games) {
if (game.id() != null && game.firstReleaseDate() != null && igdbIds.contains(game.id())) { if (game.id() != null && game.firstReleaseDate() != null && igdbIds.contains(game.id())) {
@@ -614,6 +628,23 @@ public class IgdbApiService {
return result; return result;
} }
private Map<Long, NameAndYear> namesAndYearsOf(Collection<Long> igdbIds) {
Map<Long, NameAndYear> result = new HashMap<>();
for (IgdbResponses.Game game : gamesById(igdbIds, "name,first_release_date")) {
String name = trimmedName(game);
if (game.id() != null && igdbIds.contains(game.id()) && !name.isEmpty()) {
result.put(game.id(), new NameAndYear(name, game.firstReleaseDate() == null ? null : year(game.firstReleaseDate())));
}
}
return result;
}
private IgdbResponses.Game[] gamesById(Collection<Long> igdbIds, String fields) {
String gameIds = igdbIds.stream().map(String::valueOf).collect(Collectors.joining(","));
return query("/games", "fields " + fields + "; where id = (" + gameIds + "); limit " + LIMIT + ";",
IgdbResponses.Game[].class);
}
private static int year(long unixSeconds) { private static int year(long unixSeconds) {
return date(unixSeconds).getYear(); return date(unixSeconds).getYear();
} }
@@ -1,13 +1,17 @@
package core_service.modules.inbox.api; package core_service.modules.inbox.api;
import core_service.modules.inbox.api.dto.InboxDto; import core_service.modules.inbox.api.dto.InboxDto;
import core_service.modules.inbox.api.dto.NoteDetailsDto;
import core_service.modules.inbox.domain.Inbox; import core_service.modules.inbox.domain.Inbox;
import core_service.modules.inbox.service.InboxService; import core_service.modules.inbox.service.InboxService;
import core_service.shared.api.ErrorResponse;
import core_service.shared.security.UserPrincipal; import core_service.shared.security.UserPrincipal;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity; import org.springframework.http.ResponseEntity;
import org.springframework.security.core.annotation.AuthenticationPrincipal; import org.springframework.security.core.annotation.AuthenticationPrincipal;
import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PutMapping; import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController; import org.springframework.web.bind.annotation.RestController;
@@ -23,11 +27,22 @@ public class InboxController {
public ResponseEntity<InboxDto> inbox(@AuthenticationPrincipal UserPrincipal principal) { public ResponseEntity<InboxDto> inbox(@AuthenticationPrincipal UserPrincipal principal) {
Inbox inbox = inboxService.inbox(principal.getIdAsLong()); Inbox inbox = inboxService.inbox(principal.getIdAsLong());
return ResponseEntity.ok(new InboxDto(inbox.notes().stream() return ResponseEntity.ok(new InboxDto(inbox.notes().stream()
.map(note -> new InboxDto.Note(note.id(), note.kind().name(), note.subjectName(), note.newGames(), .map(note -> new InboxDto.Note(note.id(), note.kind().name(), note.subjectName(), note.profileName(), note.newGames(),
note.achievements(), note.statusChanges(), note.wishlistAdds(), note.createdAt(), note.read())) note.achievements(), note.statusChanges(), note.wishlistAdds(), note.createdAt(), note.read(), note.details()))
.toList(), inbox.unread())); .toList(), inbox.unread()));
} }
@GetMapping("/{id}/details")
public ResponseEntity<?> details(@AuthenticationPrincipal UserPrincipal principal, @PathVariable("id") long id) {
return inboxService.details(principal.getIdAsLong(), id)
.<ResponseEntity<?>>map(details -> ResponseEntity.ok(new NoteDetailsDto(details.stream()
.map(detail -> new NoteDetailsDto.Detail(detail.appid(), detail.kind().name(), detail.gameName(),
detail.itemName(), detail.fromStatus(), detail.toStatus()))
.toList())))
.orElseGet(() -> ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(new ErrorResponse("NOTE_NOT_FOUND", "No such note")));
}
@PutMapping("/read") @PutMapping("/read")
public ResponseEntity<Void> markAllRead(@AuthenticationPrincipal UserPrincipal principal) { public ResponseEntity<Void> markAllRead(@AuthenticationPrincipal UserPrincipal principal) {
inboxService.markAllRead(principal.getIdAsLong()); inboxService.markAllRead(principal.getIdAsLong());
@@ -11,8 +11,10 @@ public record InboxDto(List<Note> results, int unread) {
/** /**
* @param kind DIGEST, VISIBILITY, PSN_EXPIRING or PSN_EXPIRED (the PSN kinds have no displayName) * @param kind DIGEST, VISIBILITY, PSN_EXPIRING or PSN_EXPIRED (the PSN kinds have no displayName)
* @param displayName the name of the followed user when the note was written; never a user id * @param displayName the name of the followed user when the note was written; never a user id
* @param profileName the subject's current name while the profile is public, else null; never an id
* @param details the note can be opened: it has detail rows and the subject is still followed and public
*/ */
public record Note(long id, String kind, String displayName, int newGames, int achievements, int statusChanges, public record Note(long id, String kind, String displayName, String profileName, int newGames, int achievements, int statusChanges,
int wishlistAdds, Instant createdAt, boolean read) { int wishlistAdds, Instant createdAt, boolean read, boolean details) {
} }
} }
@@ -0,0 +1,16 @@
package core_service.modules.inbox.api.dto;
import java.util.List;
public record NoteDetailsDto(List<Detail> results) {
/**
* @param appid the game key as stored in the note; may be a merged key
* @param kind NEW_GAME, ACHIEVEMENT, STATUS or WISHLIST
* @param itemName ACHIEVEMENT only: the achievement or trophy name
* @param fromStatus STATUS only
* @param toStatus STATUS only
*/
public record Detail(long appid, String kind, String gameName, String itemName, String fromStatus, String toStatus) {
}
}
@@ -4,7 +4,9 @@ import java.time.Instant;
/** /**
* @param subjectName the name of the followed user when the note was written * @param subjectName the name of the followed user when the note was written
* @param profileName the current name of the subject while the profile is public, else null
* @param details the note can be opened: it has detail rows and the subject is still followed and public
*/ */
public record InboxNote(long id, NoteKind kind, String subjectName, int newGames, int achievements, public record InboxNote(long id, NoteKind kind, String subjectName, String profileName, int newGames, int achievements,
int statusChanges, int wishlistAdds, Instant createdAt, boolean read) { int statusChanges, int wishlistAdds, Instant createdAt, boolean read, boolean details) {
} }
@@ -0,0 +1,7 @@
package core_service.modules.inbox.domain;
/**
* One line behind a digest note. Names are snapshots from when the note was written.
*/
public record NoteDetail(long appid, NoteDetailKind kind, String gameName, String itemName, String fromStatus, String toStatus) {
}
@@ -0,0 +1,29 @@
package core_service.modules.inbox.domain;
import java.util.Collection;
import java.util.List;
public enum NoteDetailKind {
NEW_GAME,
ACHIEVEMENT,
STATUS,
WISHLIST;
private static final List<String> STATUS_SECTIONS = List.of("STATUS", "PLAYING", "FINISHED");
/**
* Mirror of follow.domain.Digest.visibleOn, modules may not import each other.
*
* @param sections the profile parts the user shows, null = all
*/
public boolean visibleOn(Collection<String> sections) {
if (sections == null) {
return true;
}
return switch (this) {
case NEW_GAME, ACHIEVEMENT -> sections.contains("SUMMARY");
case STATUS -> STATUS_SECTIONS.stream().anyMatch(sections::contains);
case WISHLIST -> true;
};
}
}
@@ -1,15 +1,20 @@
package core_service.modules.inbox.repo; package core_service.modules.inbox.repo;
import core_service.modules.inbox.domain.InboxNote; import core_service.modules.inbox.domain.InboxNote;
import core_service.modules.inbox.domain.NoteDetail;
import core_service.modules.inbox.domain.NoteDetailKind;
import core_service.modules.inbox.domain.NoteKind; import core_service.modules.inbox.domain.NoteKind;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; import org.springframework.jdbc.core.namedparam.MapSqlParameterSource;
import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;
import org.springframework.stereotype.Repository; import org.springframework.stereotype.Repository;
import java.sql.Array;
import java.sql.Timestamp; import java.sql.Timestamp;
import java.time.Instant; import java.time.Instant;
import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.Optional;
@Repository @Repository
@RequiredArgsConstructor @RequiredArgsConstructor
@@ -19,14 +24,67 @@ public class InboxRepository {
public List<InboxNote> findNewest(long userId, int limit) { public List<InboxNote> findNewest(long userId, int limit) {
return jdbc.query(""" return jdbc.query("""
select id, kind, subject_name, new_games, achievements, status_changes, wishlist_adds, created_at, read_at select n.id, n.kind, n.subject_name, n.new_games, n.achievements, n.status_changes, n.wishlist_adds,
from inbox_note where user_id = :userId n.created_at, n.read_at,
order by created_at desc, id desc (select a.display_name from account_name a join public_user p on p.user_id = a.user_id
where a.user_id = n.subject_user_id
and not exists (select 1 from blocked_user b where b.user_id = a.user_id)
and not exists (select 1 from deleted_user d where d.user_id = a.user_id)) as profile_name,
exists (select 1 from inbox_note_detail d where d.note_id = n.id)
and exists (select 1 from follow f where f.user_id = n.user_id and f.followed_user_id = n.subject_user_id)
and exists (select 1 from public_user p where p.user_id = n.subject_user_id) as details
from inbox_note n where n.user_id = :userId
order by n.created_at desc, n.id desc
limit :limit""", new MapSqlParameterSource("userId", userId).addValue("limit", limit), limit :limit""", new MapSqlParameterSource("userId", userId).addValue("limit", limit),
(rs, i) -> new InboxNote(rs.getLong("id"), NoteKind.valueOf(rs.getString("kind")), (rs, i) -> new InboxNote(rs.getLong("id"), NoteKind.valueOf(rs.getString("kind")),
rs.getString("subject_name"), rs.getInt("new_games"), rs.getInt("achievements"), rs.getString("subject_name"), rs.getString("profile_name"), rs.getInt("new_games"), rs.getInt("achievements"),
rs.getInt("status_changes"), rs.getInt("wishlist_adds"), rs.getInt("status_changes"), rs.getInt("wishlist_adds"),
rs.getTimestamp("created_at").toInstant(), rs.getTimestamp("read_at") != null)); rs.getTimestamp("created_at").toInstant(), rs.getTimestamp("read_at") != null,
rs.getBoolean("details")));
}
/**
* Ownership and the follow re-check in one statement.
*
* @return the followed user of a DIGEST note of this user, empty when the note is not theirs, they no longer
* follow the subject or the subject is no longer public
*/
public Optional<Long> findFollowedSubject(long userId, long noteId) {
return jdbc.query("""
select n.subject_user_id from inbox_note n
where n.id = :noteId and n.user_id = :userId and n.kind = 'DIGEST'
and exists (select 1 from follow f where f.user_id = n.user_id and f.followed_user_id = n.subject_user_id)
and exists (select 1 from public_user p where p.user_id = n.subject_user_id)""",
new MapSqlParameterSource("userId", userId).addValue("noteId", noteId),
(rs, i) -> rs.getLong("subject_user_id")).stream().findFirst();
}
/**
* @return null when there is no row or the user did not send sections: nothing is filtered
*/
public List<String> findPublicSections(long subjectUserId) {
return jdbc.query("select sections from public_user where user_id = :userId",
new MapSqlParameterSource("userId", subjectUserId), rs -> {
if (!rs.next()) {
return null;
}
Array sections = rs.getArray("sections");
return sections == null ? null : Arrays.asList((String[]) sections.getArray());
});
}
/**
* Without the games the subject hides now.
*/
public List<NoteDetail> findDetails(long noteId, long subjectUserId) {
return jdbc.query("""
select d.appid, d.kind, d.game_name, d.item_name, d.from_status, d.to_status from inbox_note_detail d
where d.note_id = :noteId
and not exists (select 1 from user_game ug
where ug.user_id = :subjectUserId and ug.appid = d.appid and ug.hidden_at is not null)
order by d.id""", new MapSqlParameterSource("noteId", noteId).addValue("subjectUserId", subjectUserId),
(rs, i) -> new NoteDetail(rs.getLong("appid"), NoteDetailKind.valueOf(rs.getString("kind")), rs.getString("game_name"),
rs.getString("item_name"), rs.getString("from_status"), rs.getString("to_status")));
} }
public int countUnread(long userId) { public int countUnread(long userId) {
@@ -1,6 +1,7 @@
package core_service.modules.inbox.service; package core_service.modules.inbox.service;
import core_service.modules.inbox.domain.Inbox; import core_service.modules.inbox.domain.Inbox;
import core_service.modules.inbox.domain.NoteDetail;
import core_service.modules.inbox.repo.InboxRepository; import core_service.modules.inbox.repo.InboxRepository;
import core_service.shared.security.RequiresRole; import core_service.shared.security.RequiresRole;
import core_service.shared.security.Role; import core_service.shared.security.Role;
@@ -13,6 +14,8 @@ import org.springframework.transaction.annotation.Transactional;
import java.time.Duration; import java.time.Duration;
import java.time.Instant; import java.time.Instant;
import java.util.List;
import java.util.Optional;
@Service @Service
@Slf4j @Slf4j
@@ -32,6 +35,22 @@ public class InboxService {
return new Inbox(inboxRepository.findNewest(userId, LIST_SIZE), inboxRepository.countUnread(userId)); return new Inbox(inboxRepository.findNewest(userId, LIST_SIZE), inboxRepository.countUnread(userId));
} }
/**
* Checked again on every read: the caller must own the note, still follow the subject, the subject must be
* public and show the part.
*
* @return empty when the note cannot be opened
*/
@RequiresRole(Role.USER)
@Transactional(readOnly = true)
public Optional<List<NoteDetail>> details(long userId, long noteId) {
return inboxRepository.findFollowedSubject(userId, noteId).map(subject -> {
List<String> sections = inboxRepository.findPublicSections(subject);
return inboxRepository.findDetails(noteId, subject).stream()
.filter(detail -> detail.kind().visibleOn(sections)).toList();
});
}
@RequiresRole(Role.USER) @RequiresRole(Role.USER)
@Transactional @Transactional
public void markAllRead(long userId) { public void markAllRead(long userId) {
@@ -10,10 +10,12 @@ import core_service.modules.library.api.dto.AuditEntryDto;
import core_service.modules.library.api.dto.BrokenImageReport; import core_service.modules.library.api.dto.BrokenImageReport;
import core_service.modules.library.api.dto.CatalogGameDto; import core_service.modules.library.api.dto.CatalogGameDto;
import core_service.modules.library.api.dto.CatalogGameDetailDto; import core_service.modules.library.api.dto.CatalogGameDetailDto;
import core_service.modules.library.api.dto.CatalogPreviewDto;
import core_service.modules.library.api.dto.CatalogHitDto; import core_service.modules.library.api.dto.CatalogHitDto;
import core_service.modules.library.api.dto.CorrectionMarkDto; import core_service.modules.library.api.dto.CorrectionMarkDto;
import core_service.modules.library.api.dto.GameDetailDto; import core_service.modules.library.api.dto.GameDetailDto;
import core_service.modules.library.api.dto.CorrectionDto; import core_service.modules.library.api.dto.CorrectionDto;
import core_service.modules.library.api.dto.DashboardGamesDto;
import core_service.modules.library.api.dto.FinishedHintDismissal; import core_service.modules.library.api.dto.FinishedHintDismissal;
import core_service.modules.library.api.dto.GameDto; import core_service.modules.library.api.dto.GameDto;
import core_service.modules.library.api.dto.HiddenUpdateDto; import core_service.modules.library.api.dto.HiddenUpdateDto;
@@ -38,6 +40,7 @@ import core_service.modules.library.domain.CatalogGameDetail;
import core_service.modules.library.domain.CatalogHit; import core_service.modules.library.domain.CatalogHit;
import core_service.modules.library.domain.GameSource; import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.GameTimeToBeat;
import core_service.modules.library.domain.LinkPair; import core_service.modules.library.domain.LinkPair;
import core_service.modules.library.domain.Mood; import core_service.modules.library.domain.Mood;
import core_service.modules.library.domain.PickSource; import core_service.modules.library.domain.PickSource;
@@ -99,7 +102,9 @@ public class LibraryController {
private static final int MAX_ACTIVITY_DAYS = 90; private static final int MAX_ACTIVITY_DAYS = 90;
private static final int MAX_PICKER_EXCLUDES = 100; private static final int MAX_PICKER_EXCLUDES = 100;
private static final int MAX_BACKLOG_GAMES = 20; private static final int MAX_BACKLOG_GAMES = 20;
private static final Set<String> SORTABLE_FIELDS = Set.of("playtimeMinutes", "firstSeenAt", "lastSeenAt", "appid", "rating", "priority"); private static final int MAX_DASHBOARD_GAMES = 20;
private static final Set<String> SORTABLE_FIELDS = Set.of("playtimeMinutes", "firstSeenAt", "lastSeenAt", "appid", "rating", "priority",
"name", "lastPlayedAt", "timeToBeat");
private final LibrarySyncService syncService; private final LibrarySyncService syncService;
private final LibraryQueryService queryService; private final LibraryQueryService queryService;
@@ -148,6 +153,16 @@ public class LibraryController {
return ResponseEntity.ok(PageDto.of(dtoPage)); return ResponseEntity.ok(PageDto.of(dtoPage));
} }
@GetMapping("/dashboard-games")
public ResponseEntity<DashboardGamesDto> getDashboardGames(@AuthenticationPrincipal UserPrincipal principal,
@RequestParam(defaultValue = "10") int top,
@RequestParam(defaultValue = "8") int playing) {
Long userId = principal.getIdAsLong();
syncService.syncIfStale(userId, principal.steamIdText().orElse(null));
return ResponseEntity.ok(queryService.listDashboardGames(userId, Math.clamp(top, 1, MAX_DASHBOARD_GAMES),
Math.clamp(playing, 1, MAX_DASHBOARD_GAMES)));
}
private Sort parseSort(String sort) { private Sort parseSort(String sort) {
Sort fallback = Sort.by(Sort.Direction.DESC, "playtimeMinutes"); Sort fallback = Sort.by(Sort.Direction.DESC, "playtimeMinutes");
if (sort == null || sort.isBlank()) return fallback; if (sort == null || sort.isBlank()) return fallback;
@@ -177,11 +192,20 @@ public class LibraryController {
} }
} }
@GetMapping("/games/{appid}/preview")
public ResponseEntity<?> getGamePreview(@PathVariable("appid") Long appid) {
return gameDetailService.preview(appid)
.<ResponseEntity<?>>map(p -> ResponseEntity.ok(new CatalogPreviewDto(p.appid(), p.name(), p.imageUrl(),
p.genres(), toDto(p.timeToBeat()))))
.orElseGet(LibraryController::unknownGame);
}
@GetMapping("/games/{appid}") @GetMapping("/games/{appid}")
public ResponseEntity<?> getGameDetail(@AuthenticationPrincipal UserPrincipal principal, public ResponseEntity<?> getGameDetail(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") Long appid) { @PathVariable("appid") Long appid,
@RequestParam(defaultValue = "true") boolean refresh) {
Long userId = principal.getIdAsLong(); Long userId = principal.getIdAsLong();
return gameDetailService.getGameDetail(userId, principal.steamIdText().orElse(null), appid) return gameDetailService.getGameDetail(userId, principal.steamIdText().orElse(null), appid, refresh)
.<ResponseEntity<?>>map(ResponseEntity::ok) .<ResponseEntity<?>>map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.status(HttpStatus.NOT_FOUND) .orElseGet(() -> ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(GAME_NOT_FOUND)); .body(GAME_NOT_FOUND));
@@ -354,13 +378,16 @@ public class LibraryController {
} }
private static CatalogGameDetailDto toDto(CatalogGameDetail d) { private static CatalogGameDetailDto toDto(CatalogGameDetail d) {
var t = d.timeToBeat();
return new CatalogGameDetailDto(d.appid(), d.name(), d.imageUrl(), d.speedrunUrl(), d.moods(), d.endless(), return new CatalogGameDetailDto(d.appid(), d.name(), d.imageUrl(), d.speedrunUrl(), d.moods(), d.endless(),
d.genres(), t == null ? null : new GameDetailDto.TimeToBeatDto(t.getHastilyMinutes(), d.genres(), toDto(d.timeToBeat()),
t.getNormallyMinutes(), t.getCompletelyMinutes()),
d.corrections().stream().map(c -> new CorrectionMarkDto(c.getField(), c.getCorrectedAt())).toList()); d.corrections().stream().map(c -> new CorrectionMarkDto(c.getField(), c.getCorrectedAt())).toList());
} }
private static GameDetailDto.TimeToBeatDto toDto(GameTimeToBeat t) {
return t == null ? null : new GameDetailDto.TimeToBeatDto(t.getHastilyMinutes(), t.getNormallyMinutes(),
t.getCompletelyMinutes());
}
private static CatalogHitDto toDto(CatalogHit hit) { private static CatalogHitDto toDto(CatalogHit hit) {
return new CatalogHitDto(hit.appid(), hit.name(), hit.imageUrl(), hit.platforms()); return new CatalogHitDto(hit.appid(), hit.name(), hit.imageUrl(), hit.platforms());
} }
@@ -5,6 +5,7 @@ import core_service.modules.library.api.dto.AddGameRequest;
import core_service.modules.library.api.dto.AddedGameDto; import core_service.modules.library.api.dto.AddedGameDto;
import core_service.modules.library.api.dto.GameSearchDto; import core_service.modules.library.api.dto.GameSearchDto;
import core_service.modules.library.api.dto.ManualGameUpdate; import core_service.modules.library.api.dto.ManualGameUpdate;
import core_service.modules.library.api.dto.RemovedGamesDto;
import core_service.modules.library.service.ManualGameService; import core_service.modules.library.service.ManualGameService;
import core_service.shared.security.UserPrincipal; import core_service.shared.security.UserPrincipal;
import jakarta.validation.Valid; import jakarta.validation.Valid;
@@ -79,6 +80,20 @@ public class ManualGameController {
return toResponse(manualGameService.remove(principal.getIdAsLong(), appid)); return toResponse(manualGameService.remove(principal.getIdAsLong(), appid));
} }
@PostMapping("/games/{appid}/restore")
public ResponseEntity<?> restore(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") long appid) {
return toResponse(manualGameService.restore(principal.getIdAsLong(), appid));
}
@GetMapping("/removed-games")
public RemovedGamesDto removedGames(@AuthenticationPrincipal UserPrincipal principal) {
return new RemovedGamesDto(manualGameService.removedGames(principal.getIdAsLong()).stream()
.map(game -> new RemovedGamesDto.Game(game.appid(), game.name(), game.imageUrl(), game.source().name(),
game.platform(), game.removedAt()))
.toList());
}
private static ResponseEntity<?> toResponse(ManualGameService.ChangeResult result) { private static ResponseEntity<?> toResponse(ManualGameService.ChangeResult result) {
return switch (result) { return switch (result) {
case DONE -> ResponseEntity.noContent().build(); case DONE -> ResponseEntity.noContent().build();
@@ -11,6 +11,7 @@ import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpStatus; import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity; import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestHeader; import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RequestParam;
@@ -38,6 +39,17 @@ public class PublicProfileController {
.orElseGet(PublicProfileController::invalidTicket); .orElseGet(PublicProfileController::invalidTicket);
} }
@GetMapping("/library/games/{appid}")
public ResponseEntity<?> getPublicGame(@RequestHeader(value = "X-Profile-Ticket", required = false) String ticket,
@PathVariable("appid") long appid) {
return verify(ticket)
.<ResponseEntity<?>>map(t -> queryService.getPublicGame(t.userId(), t.sections(), appid)
.<ResponseEntity<?>>map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(new ErrorResponse("GAME_NOT_FOUND", "No such game on this profile"))))
.orElseGet(PublicProfileController::invalidTicket);
}
@GetMapping("/library/stats/games") @GetMapping("/library/stats/games")
public ResponseEntity<?> getPublicStatGames(@RequestHeader(value = "X-Profile-Ticket", required = false) String ticket, public ResponseEntity<?> getPublicStatGames(@RequestHeader(value = "X-Profile-Ticket", required = false) String ticket,
@RequestParam StatKey key, @RequestParam StatKey key,
@@ -0,0 +1,13 @@
package core_service.modules.library.api.dto;
import java.util.List;
/**
* A game the caller may not own: catalog data only, nothing per user.
*
* @param appid the kept key when the asked key was merged
* @param timeToBeat null when unknown
*/
public record CatalogPreviewDto(long appid, String name, String imageUrl, List<String> genres,
GameDetailDto.TimeToBeatDto timeToBeat) {
}
@@ -0,0 +1,8 @@
package core_service.modules.library.api.dto;
import java.util.List;
/**
* The MOST_PLAYED and PLAYING tiles: listed entries by set playtime, most first; a game can be in both.
*/
public record DashboardGamesDto(List<GameDto> top, List<GameDto> playing) {}
@@ -61,7 +61,9 @@ public record GameDetailDto(
// one line per platform of the entry, Steam first; empty for a hidden game // one line per platform of the entry, Steam first; empty for a hidden game
List<PlatformPlayDto> platforms, List<PlatformPlayDto> platforms,
// the sum of the platform lines; the counts above and playtimeMinutes stay this edition's own // the sum of the platform lines; the counts above and playtimeMinutes stay this edition's own
int totalPlaytimeMinutes int totalPlaytimeMinutes,
// true when this answer skipped a due Steam refresh (refresh=false); the same call without it refreshes
boolean achievementsStale
) { ) {
/** /**
@@ -0,0 +1,14 @@
package core_service.modules.library.api.dto;
import core_service.modules.library.domain.GameStatus;
/**
* One game of a public profile. A field is null when a hidden section covers it or the value is unknown.
*
* @param status hidden with the STATUS, PLAYING and FINISHED sections
* @param playtimeMinutes hidden with the SUMMARY section, like the achievements
* @param achievementsUnlocked null when the game has no achievements, Steam hides them or none were fetched
*/
public record PublicGameDto(long appid, String name, String imageUrl, GameStatus status, Integer playtimeMinutes,
Integer achievementsUnlocked, Integer achievementsTotal, Double completionPercent) {
}
@@ -0,0 +1,9 @@
package core_service.modules.library.api.dto;
import java.time.Instant;
import java.util.List;
public record RemovedGamesDto(List<Game> results) {
public record Game(long appid, String name, String imageUrl, String source, String platform, Instant removedAt) {}
}
@@ -24,6 +24,8 @@ public record UserStatsDto(
) { ) {
/** /**
* @param playedGames games with a set total above 0
* @param neverStarted games with a set total of 0
* @param byStatus has every status, also those with 0 games * @param byStatus has every status, also those with 0 games
*/ */
public record Overview(long totalGames, long totalPlaytimeMinutes, long playedGames, long neverStarted, public record Overview(long totalGames, long totalPlaytimeMinutes, long playedGames, long neverStarted,
@@ -56,6 +58,7 @@ public record UserStatsDto(
/** /**
* @param moonTrips the total playtime as flights to the moon and back (Apollo 11, 195.3 hours) * @param moonTrips the total playtime as flights to the moon and back (Apollo 11, 195.3 hours)
* @param longestDay the day with the most playtime; null without history * @param longestDay the day with the most playtime; null without history
* @param underOneHourGames games whose set total is 1 to 59 minutes
* @param underOneHourShare share of the started games with less than one hour; null when none is started * @param underOneHourShare share of the started games with less than one hour; null when none is started
*/ */
public record Habits(double moonTrips, DayMinutes longestDay, long underOneHourGames, Double underOneHourShare) {} public record Habits(double moonTrips, DayMinutes longestDay, long underOneHourGames, Double underOneHourShare) {}
@@ -0,0 +1,11 @@
package core_service.modules.library.domain;
import java.util.List;
/**
* Catalog data of one game key, the same for every user.
*
* @param timeToBeat null when unknown
*/
public record CatalogPreview(long appid, String name, String imageUrl, List<String> genres, GameTimeToBeat timeToBeat) {
}
@@ -1,10 +1,17 @@
package core_service.modules.library.domain; package core_service.modules.library.domain;
import java.time.Instant;
/** /**
* A PlayStation title the sync has seen for a user (V39). A LINKED title has the library key of the row PlayStation * A PlayStation title the sync has seen for a user (V39). A LINKED title has the library key of the row PlayStation
* made or feeds; STEAM_OWNED has the key of a Steam row it only matched; DISMISSED titles were left unchecked in a review. * made or feeds; STEAM_OWNED has the key of a Steam row it only matched; DISMISSED titles were left unchecked in a review.
* The stored play (playtimeMinutes, lastPlayedAt) is what the last sync saw; both are null until the first savePlays.
*/ */
public record PsnTitle(String psnKey, State state, Long appid) { public record PsnTitle(String psnKey, State state, Long appid, Integer playtimeMinutes, Instant lastPlayedAt) {
public PsnTitle(String psnKey, State state, Long appid) {
this(psnKey, state, appid, null, null);
}
public enum State { LINKED, STEAM_OWNED, DISMISSED } public enum State { LINKED, STEAM_OWNED, DISMISSED }
} }
@@ -0,0 +1,8 @@
package core_service.modules.library.domain;
import java.time.Instant;
/**
* A game the user removed and can restore.
*/
public record RemovedGame(long appid, String name, String imageUrl, GameSource source, String platform, Instant removedAt) {}
@@ -27,6 +27,10 @@ public record SyncChange(SyncChangeKind kind, long appid, Integer amount, GameSt
return new SyncChange(SyncChangeKind.STATUS, appid, null, from, to); return new SyncChange(SyncChangeKind.STATUS, appid, null, from, to);
} }
public static SyncChange allUnlocked(long appid) {
return new SyncChange(SyncChangeKind.ALL_UNLOCKED, appid, null, null, null);
}
public static SyncChange wishlist(long appid) { public static SyncChange wishlist(long appid) {
return new SyncChange(SyncChangeKind.WISHLIST, appid, null, null, null); return new SyncChange(SyncChangeKind.WISHLIST, appid, null, null, null);
} }
@@ -6,6 +6,8 @@ public enum SyncChangeKind {
PLAYTIME, PLAYTIME,
ACHIEVEMENTS, ACHIEVEMENTS,
STATUS, STATUS,
// The sync unlocked the last achievement of a game that is not COMPLETED.
ALL_UNLOCKED,
// Only for followers' digests: never shown in the sync banner. // Only for followers' digests: never shown in the sync banner.
WISHLIST WISHLIST
} }
@@ -12,6 +12,7 @@ import java.util.List;
/** /**
* The daily pile size per user, table backlog_snapshot. Counts listed games only, like {@link UserGameRepository#LISTED}, and a linked set once (V28). * The daily pile size per user, table backlog_snapshot. Counts listed games only, like {@link UserGameRepository#LISTED}, and a linked set once (V28).
* Never played is judged by the set total (V45), like {@link UserGameRepository#countListedNeverPlayed}.
*/ */
@Repository @Repository
@RequiredArgsConstructor @RequiredArgsConstructor
@@ -19,11 +20,12 @@ public class BacklogSnapshotRepository {
private static final String PILE_COLUMNS = """ private static final String PILE_COLUMNS = """
count(*) filter (where ug.status = 'BACKLOG' and cg.appid is null) as backlog_games, count(*) filter (where ug.status = 'BACKLOG' and cg.appid is null) as backlog_games,
count(*) filter (where ug.playtime_minutes = 0) as never_played, count(*) filter (where ep.playtime_minutes = 0) as never_played,
coalesce(sum(t.normally_minutes) filter (where ug.status = 'BACKLOG' and cg.appid is null), 0) as backlog_minutes, coalesce(sum(t.normally_minutes) filter (where ug.status = 'BACKLOG' and cg.appid is null), 0) as backlog_minutes,
count(*) as listed_games count(*) as listed_games
from counted_user_game ug left join game_time_to_beat t on t.appid = ug.appid from counted_user_game ug left join game_time_to_beat t on t.appid = ug.appid
left join coming_game cg on cg.appid = ug.appid left join coming_game cg on cg.appid = ug.appid
left join user_game_entry_playtime ep on ep.user_id = ug.user_id and ep.appid = ug.appid
"""; """;
private final NamedParameterJdbcTemplate jdbc; private final NamedParameterJdbcTemplate jdbc;
@@ -309,7 +309,8 @@ public class GameLinkRepository {
/** /**
* @return per given entry key, the platforms of its editions (see V43): Steam, PlayStation, then manual, by * @return per given entry key, the platforms of its editions (see V43): Steam, PlayStation, then manual, by
* platform name; a PlayStation platform without a name is left out when the entry has a named PlayStation one, or its own member row has a manual platform * platform name; a PlayStation platform without a name is left out when the entry has a named PlayStation one, or its own member row has a manual platform.
* The view is read once: each reference costs ~16 ms of planning (nested views), more than the query runs.
*/ */
public Map<Long, List<EntryPlatform>> entryPlatforms(long userId, Collection<Long> appids) { public Map<Long, List<EntryPlatform>> entryPlatforms(long userId, Collection<Long> appids) {
Map<Long, List<EntryPlatform>> platforms = new HashMap<>(); Map<Long, List<EntryPlatform>> platforms = new HashMap<>();
@@ -317,13 +318,15 @@ public class GameLinkRepository {
return platforms; return platforms;
} }
jdbc.query(""" jdbc.query("""
with p as materialized (
select appid, member_appid, source, platform from user_game_entry_platform
where user_id = :user and appid in (:appids))
select appid, source, platform select appid, source, platform
from user_game_entry_platform p from p
where user_id = :user and appid in (:appids) where not (source = 'PSN' and platform = '' and exists (
and not (source = 'PSN' and platform = '' and exists ( select 1 from p n
select 1 from user_game_entry_platform n where n.appid = p.appid and n.platform <> ''
where n.user_id = p.user_id and n.appid = p.appid and n.platform <> '' and (n.source = 'PSN' or (n.source = 'MANUAL' and n.member_appid = p.member_appid))))
and (n.source = 'PSN' or (n.source = 'MANUAL' and n.member_appid = p.member_appid))))
group by appid, source, platform group by appid, source, platform
order by appid, case source when 'STEAM' then 0 when 'PSN' then 1 else 2 end, platform order by appid, case source when 'STEAM' then 0 when 'PSN' then 1 else 2 end, platform
""", new MapSqlParameterSource("user", userId).addValue("appids", appids), """, new MapSqlParameterSource("user", userId).addValue("appids", appids),
@@ -20,7 +20,8 @@ import java.util.Map;
import java.util.Optional; import java.util.Optional;
/** /**
* Merges one library key into another (V21). Plain SQL: a merge rewrites the key of rows in many tables at once. * Merges one library key into another (V21), and moves one user's row to another key (Steam takeover).
* Plain SQL: a merge rewrites the key of rows in many tables at once.
*/ */
@Repository @Repository
@RequiredArgsConstructor @RequiredArgsConstructor
@@ -39,6 +40,10 @@ public class GameMergeRepository {
+ " and not exists (select 1 from reward_ledger o where o.user_id = l.user_id and o.kind = l.kind" + " and not exists (select 1 from reward_ledger o where o.user_id = l.user_id and o.kind = l.kind"
+ " and o.ref = cast(:to as text))"; + " and o.ref = cast(:to as text))";
// A row added by hand for PC, Steam, Mac or Linux (or with no platform) is the Steam copy.
private static final String TAKEOVER_PLATFORM =
"lower(btrim(coalesce(ug.platform, ''))) in ('', 'pc', 'steam', 'mac', 'linux')";
// What the snapshot saves besides the catalog tables: the link sets of both keys and the pairs and requests of from. // What the snapshot saves besides the catalog tables: the link sets of both keys and the pairs and requests of from.
private static final String LINK_SET_ROWS = "select * from game_link where set_id in " private static final String LINK_SET_ROWS = "select * from game_link where set_id in "
+ "(select set_id from game_link where appid in (:from, :to))"; + "(select set_id from game_link where appid in (:from, :to))";
@@ -259,6 +264,13 @@ public class GameMergeRepository {
return targets; return targets;
} }
/**
* @return the key a merged key now lives under, else the key itself
*/
public long findTarget(long appid) {
return findTargets(List.of(appid)).getOrDefault(appid, appid);
}
/** /**
* @return the saved copy of a merge, locked until the transaction ends; empty when there is none * @return the saved copy of a merge, locked until the transaction ends; empty when there is none
*/ */
@@ -366,4 +378,75 @@ public class GameMergeRepository {
return jdbc.update("delete from game_merge_undo where created_at < :cutoff", return jdbc.update("delete from game_merge_undo where created_at < :cutoff",
new MapSqlParameterSource("cutoff", Timestamp.from(cutoff))); new MapSqlParameterSource("cutoff", Timestamp.from(cutoff)));
} }
public boolean hasSteamTakeoverCandidates(long userId) {
return Boolean.TRUE.equals(jdbc.queryForObject("""
select exists (select 1 from user_game ug
where ug.user_id = :user and ug.appid < 0 and ug.source = 'MANUAL' and %s)
""".formatted(TAKEOVER_PLATFORM), new MapSqlParameterSource("user", userId), Boolean.class));
}
/**
* @return the keys of {@code appids} the user has no row for and whose stored edition has no IGDB game
*/
public List<Long> findUnmappedNewlyOwned(long userId, Collection<Long> appids) {
if (appids.isEmpty()) {
return List.of();
}
return jdbc.queryForList("""
select e.appid from game_edition e
where e.appid in (:appids) and e.igdb_id is null
and not exists (select 1 from user_game t where t.user_id = :user and t.appid = e.appid)
""", new MapSqlParameterSource("user", userId).addValue("appids", appids), Long.class);
}
/**
* @return a manual key of the user, by the Steam appid that takes its row; only a Steam appid the user has no row for
*/
public Map<Long, Long> findSteamTakeovers(long userId, Collection<Long> steamAppids) {
if (steamAppids.isEmpty()) {
return Map.of();
}
Map<Long, Long> takeovers = new LinkedHashMap<>();
jdbc.query("""
select distinct on (ug.appid) ug.appid as from_appid, e.appid as to_appid
from user_game ug
join game_edition e on e.igdb_id = -ug.appid and e.appid in (:appids)
where ug.user_id = :user and ug.appid < 0 and ug.source = 'MANUAL' and %s
and not exists (select 1 from user_game t where t.user_id = ug.user_id and t.appid = e.appid)
order by ug.appid, e.appid
""".formatted(TAKEOVER_PLATFORM),
new MapSqlParameterSource("user", userId).addValue("appids", steamAppids),
rs -> { takeovers.put(rs.getLong("from_appid"), rs.getLong("to_appid")); });
return takeovers;
}
/**
* Moves one user's row to another key. The cascading FKs move status log, backlog order, questions, splits and
* set backups. Needs the steam_game row of {@code to} flushed.
*
* @return false when the user has no row on {@code from}, or already has one on {@code to}
*/
public boolean rekeyUserGame(long userId, long from, long to) {
MapSqlParameterSource params = new MapSqlParameterSource("user", userId).addValue("from", from).addValue("to", to);
if (jdbc.update("""
update user_game set appid = :to where user_id = :user and appid = :from
and not exists (select 1 from user_game t where t.user_id = :user and t.appid = :to)
""", params) == 0) {
return false;
}
// A day both keys have adds up on the target.
jdbc.update("""
insert into user_game_playtime_day (user_id, day, appid, minutes)
select user_id, day, :to, minutes from user_game_playtime_day where user_id = :user and appid = :from
on conflict (user_id, day, appid) do update
set minutes = least(user_game_playtime_day.minutes + excluded.minutes, 1440)
""", params);
jdbc.update("delete from user_game_playtime_day where user_id = :user and appid = :from", params);
jdbc.update("update user_psn_title set appid = :to where user_id = :user and appid = :from", params);
jdbc.update("update user_psn_trophy_set set appid = :to where user_id = :user and appid = :from", params);
jdbc.update("update reward_ledger l set ref = cast(:to as text) where l.user_id = :user and " + LEDGER_MOVED,
params);
return true;
}
} }
@@ -9,6 +9,7 @@ import org.springframework.transaction.annotation.Transactional;
import java.time.Instant; import java.time.Instant;
import java.util.Collection; import java.util.Collection;
import java.util.List;
import java.util.Set; import java.util.Set;
public interface GameSpeedrunRepository extends JpaRepository<GameSpeedrun, Long> { public interface GameSpeedrunRepository extends JpaRepository<GameSpeedrun, Long> {
@@ -48,4 +49,12 @@ public interface GameSpeedrunRepository extends JpaRepository<GameSpeedrun, Long
@Modifying @Modifying
@Query("update GameSpeedrun s set s.checkedAt = :longAgo where s.appid = :appid") @Query("update GameSpeedrun s set s.checkedAt = :longAgo where s.appid = :appid")
void markStale(@Param("appid") Long appid, @Param("longAgo") Instant longAgo); void markStale(@Param("appid") Long appid, @Param("longAgo") Instant longAgo);
/**
* Rows marked for a new check (checked_at <= before) of games some user lists, without an auditor's link.
*/
@Query(value = "select appid from game_speedrun where checked_at <= :before and not " + CORRECTED
+ " and exists (select 1 from listed_user_game ug where ug.appid = game_speedrun.appid) order by appid",
nativeQuery = true)
List<Long> findAppidsToRecheck(@Param("before") Instant before);
} }
@@ -25,10 +25,11 @@ public class PsnTitleRepository {
public Map<String, PsnTitle> findAll(long userId) { public Map<String, PsnTitle> findAll(long userId) {
Map<String, PsnTitle> titles = new LinkedHashMap<>(); Map<String, PsnTitle> titles = new LinkedHashMap<>();
jdbc.query("select psn_key, state, appid from user_psn_title where user_id = :user", jdbc.query("select psn_key, state, appid, playtime_minutes, last_played_at from user_psn_title where user_id = :user",
new MapSqlParameterSource("user", userId), rs -> { new MapSqlParameterSource("user", userId), rs -> {
titles.put(rs.getString("psn_key"), new PsnTitle(rs.getString("psn_key"), titles.put(rs.getString("psn_key"), new PsnTitle(rs.getString("psn_key"),
PsnTitle.State.valueOf(rs.getString("state")), rs.getObject("appid", Long.class))); PsnTitle.State.valueOf(rs.getString("state")), rs.getObject("appid", Long.class),
rs.getObject("playtime_minutes", Integer.class), instant(rs.getTimestamp("last_played_at"))));
}); });
return titles; return titles;
} }
@@ -104,7 +104,8 @@ public class SyncChangeRepository {
} }
/** /**
* Hidden games are left out when read, so hiding a game after the sync hides it here too. * Hidden games are left out when read, so hiding a game after the sync hides it here too. A removed game stays only for its own REMOVED entry.
* An ALL_UNLOCKED row goes once the game is COMPLETED.
*/ */
public Optional<SyncChanges> find(long userId) { public Optional<SyncChanges> find(long userId) {
MapSqlParameterSource params = new MapSqlParameterSource("userId", userId); MapSqlParameterSource params = new MapSqlParameterSource("userId", userId);
@@ -131,7 +132,8 @@ public class SyncChangeRepository {
from sync_change c from sync_change c
join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid join user_game ug on ug.user_id = c.user_id and ug.appid = c.appid
left join steam_game sg on sg.appid = c.appid left join steam_game sg on sg.appid = c.appid
where c.user_id = :userId and ug.hidden_at is null and c.kind <> 'WISHLIST' where c.user_id = :userId and ug.hidden_at is null and (ug.removed_at is null or c.kind = 'REMOVED') and c.kind <> 'WISHLIST'
and (c.kind <> 'ALL_UNLOCKED' or ug.status <> 'COMPLETED')
group by c.kind, c.appid, c.from_status, c.to_status, sg.name, sg.image_url group by c.kind, c.appid, c.from_status, c.to_status, sg.name, sg.image_url
order by c.kind, amount desc nulls last, sg.name order by c.kind, amount desc nulls last, sg.name
""", params, (rs, i) -> new SyncChanges.Entry( """, params, (rs, i) -> new SyncChanges.Entry(
@@ -6,9 +6,11 @@ import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.Mood; import core_service.modules.library.domain.Mood;
import core_service.modules.library.domain.RecentEntry; import core_service.modules.library.domain.RecentEntry;
import core_service.modules.library.domain.RemovedGame;
import core_service.modules.library.domain.StatusCount; import core_service.modules.library.domain.StatusCount;
import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId; import core_service.modules.library.domain.UserGameId;
import core_service.shared.sql.LikePattern;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort; import org.springframework.data.domain.Sort;
@@ -24,9 +26,9 @@ import java.time.LocalDate;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collection; import java.util.Collection;
import java.util.EnumMap; import java.util.EnumMap;
import java.util.EnumSet;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Locale;
import java.util.Optional; import java.util.Optional;
public interface UserGameRepository extends JpaRepository<UserGame, UserGameId> { public interface UserGameRepository extends JpaRepository<UserGame, UserGameId> {
@@ -43,21 +45,22 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
String LISTED = "ug.userId = :userId and ug.removedAt is null and ug.hiddenAt is null "; String LISTED = "ug.userId = :userId and ug.removedAt is null and ug.hiddenAt is null ";
/** /**
* The filters of {@link #findOwnedGames}, shared by its value and count query. Needs the aliases {@code ug} and {@code sg}. * The filters of {@link #findOwnedEntries}, shared by its value and count query. Needs the aliases {@code ug} and {@code sg}.
* Every subquery filters by {@code :userId} itself: Postgres does not pull a subquery inside an {@code or} up into a
* join, so without it the subquery is hashed over the rows of all users.
*/ */
String OWNED_WHERE = "where ug.userId = :userId and ug.removedAt is null " + String OWNED_WHERE = "where ug.userId = :userId and ug.removedAt is null " +
"and ((:hidden = true and ug.hiddenAt is not null) or (:hidden = false and ug.hiddenAt is null)) " + "and ((:hidden = true and ug.hiddenAt is not null) or (:hidden = false and ug.hiddenAt is null)) " +
"and (:status is null or ug.status = :status) " + "and (:status is null or ug.status = :status) " +
"and (:namePattern is null or (:hidden = true and lower(sg.name) like :namePattern escape '\\') " + "and (:namePattern is null or (:hidden = true and lower(sg.name) like :namePattern escape '\\') " +
" or (:hidden = false and exists (select 1 from UserGameEntryPlatform p " + " or (:hidden = false and ug.appid in (select p.appid from UserGameEntryPlatform p " +
" join SteamGame mg on mg.appid = p.memberAppid " + " join SteamGame mg on mg.appid = p.memberAppid " +
" where p.userId = ug.userId and p.appid = ug.appid and lower(mg.name) like :namePattern escape '\\'))) " + " where p.userId = :userId and lower(mg.name) like :namePattern escape '\\'))) " +
"and (:maxMinutes is null or exists (select 1 from GameTimeToBeat t " + "and (:maxMinutes is null or exists (select 1 from GameTimeToBeat t " +
" where t.appid = ug.appid and t.normallyMinutes <= :maxMinutes)) " + " where t.appid = ug.appid and t.normallyMinutes <= :maxMinutes)) " +
"and (:hidden = true or exists (select 1 from CountedUserGame c " + "and (:hidden = true or ug.appid in (select c.appid from CountedUserGame c where c.userId = :userId)) " +
" where c.userId = ug.userId and c.appid = ug.appid)) " + "and (:hidden = true or :anySource = true or ug.appid in (select p.appid from UserGameEntryPlatform p " +
"and (:hidden = true or exists (select 1 from UserGameEntryPlatform p " + " where p.userId = :userId and p.source in :sources))";
" where p.userId = ug.userId and p.appid = ug.appid and p.source in :sources))";
/** /**
* The sort text of the set total: the own minutes for a hidden row. The query needs * The sort text of the set total: the own minutes for a hidden row. The query needs
@@ -120,7 +123,8 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
nativeQuery = true) nativeQuery = true)
List<RecentEntry> findEntriesPlayedInTwoWeeks(@Param("userId") Long userId, @Param("limit") int limit); List<RecentEntry> findEntriesPlayedInTwoWeeks(@Param("userId") Long userId, @Param("limit") int limit);
@Query("select count(c) from CountedUserGame c where c.userId = :userId and c.playtimeMinutes = 0") /** The entries whose set total (V45) is 0. */
@Query("select count(ep) from UserGameEntryPlaytime ep where ep.userId = :userId and ep.playtimeMinutes = 0")
long countListedNeverPlayed(@Param("userId") Long userId); long countListedNeverPlayed(@Param("userId") Long userId);
@Query("select new core_service.modules.library.domain.StatusCount(c.status, count(c)) from CountedUserGame c " + @Query("select new core_service.modules.library.domain.StatusCount(c.status, count(c)) from CountedUserGame c " +
@@ -359,9 +363,26 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
return mood == null || mood == Mood.SPEEDRUN ? null : mood.name(); return mood == null || mood == Mood.SPEEDRUN ? null : mood.name();
} }
// ponytail: the view is joined on every page, not only for the playtime sort: about +4 ms per page. Upgrade path:
// join it only for the playtime sort or the PLAYING status, with a second query.
@Query(value = "select ug from UserGame ug join SteamGame sg on sg.appid = ug.appid " +
"left join UserGameEntryPlaytime ep on ep.userId = ug.userId and ep.appid = ug.appid " +
"left join GameTimeToBeat tb on tb.appid = ug.appid " + OWNED_WHERE,
countQuery = "select count(ug) from UserGame ug join SteamGame sg on sg.appid = ug.appid " + OWNED_WHERE)
Page<UserGame> findOwnedEntries(@Param("userId") Long userId,
@Param("hidden") boolean hidden,
@Param("anySource") boolean anySource,
@Param("status") GameStatus status,
@Param("namePattern") String namePattern,
@Param("maxMinutes") Integer maxMinutes,
@Param("sources") Collection<GameSource> sources,
Pageable pageable);
/** /**
* One row per entry (see V43): a listed game is the counted edition of its set; a hidden game is a row of its own. * One row per entry (see V43): a listed game is the counted edition of its set; a hidden game is a row of its own.
* *
* All sources skip the source filter: every counted entry has a platform row of its own source.
*
* @param hidden true for only the hidden games, false for only the listed ones * @param hidden true for only the hidden games, false for only the listed ones
* @param status null for every status * @param status null for every status
* @param namePattern a {@link #containsPattern} result, null for every name; a listed entry matches by the name of * @param namePattern a {@link #containsPattern} result, null for every name; a listed entry matches by the name of
@@ -370,18 +391,11 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
* @param sources the sources one platform of the entry must have (a PlayStation title on a Steam row counts as * @param sources the sources one platform of the entry must have (a PlayStation title on a Steam row counts as
* PSN); never empty, and ignored for the hidden games * PSN); never empty, and ignored for the hidden games
*/ */
// ponytail: the view is joined on every page, not only for the playtime sort: about +4 ms per page. Upgrade path: default Page<UserGame> findOwnedGames(Long userId, boolean hidden, GameStatus status, String namePattern,
// join it only for the playtime sort or the PLAYING status, with a second query. Integer maxMinutes, Collection<GameSource> sources, Pageable pageable) {
@Query(value = "select ug from UserGame ug join SteamGame sg on sg.appid = ug.appid " + return findOwnedEntries(userId, hidden, sources.containsAll(EnumSet.allOf(GameSource.class)), status,
"left join UserGameEntryPlaytime ep on ep.userId = ug.userId and ep.appid = ug.appid " + OWNED_WHERE, namePattern, maxMinutes, sources, pageable);
countQuery = "select count(ug) from UserGame ug join SteamGame sg on sg.appid = ug.appid " + OWNED_WHERE) }
Page<UserGame> findOwnedGames(@Param("userId") Long userId,
@Param("hidden") boolean hidden,
@Param("status") GameStatus status,
@Param("namePattern") String namePattern,
@Param("maxMinutes") Integer maxMinutes,
@Param("sources") Collection<GameSource> sources,
Pageable pageable);
@Query("select distinct ug.source from UserGame ug where ug.userId = :userId and ug.removedAt is null") @Query("select distinct ug.source from UserGame ug where ug.userId = :userId and ug.removedAt is null")
List<GameSource> findSources(@Param("userId") Long userId); List<GameSource> findSources(@Param("userId") Long userId);
@@ -405,7 +419,9 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
/** /**
* Makes a {@link #findOwnedGames} sort stable and meaningful: games without a rating or priority come last * Makes a {@link #findOwnedGames} sort stable and meaningful: games without a rating or priority come last
* in both directions, priority sorts by HIGH, MEDIUM, LOW (the enum names would sort alphabetically), and * in both directions, priority sorts by HIGH, MEDIUM, LOW (the enum names would sort alphabetically), and
* appid breaks every tie so pages do not repeat or skip games. Playtime sorts by the set total. * appid breaks every tie so pages do not repeat or skip games. Playtime sorts by the set total. A game never
* played or without a known time to beat comes last in both directions; the name sorts case-insensitive.
* The time to beat needs {@code left join GameTimeToBeat tb on tb.appid = ug.appid}.
*/ */
static Sort querySort(Sort requested) { static Sort querySort(Sort requested) {
Sort result = Sort.unsorted(); Sort result = Sort.unsorted();
@@ -417,6 +433,10 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
case "priority" -> Sort.by(JpaSort.unsafe(order.getDirection(), case "priority" -> Sort.by(JpaSort.unsafe(order.getDirection(),
"(case ug.priority when 'HIGH' then 3 when 'MEDIUM' then 2 when 'LOW' then 1 end)") "(case ug.priority when 'HIGH' then 3 when 'MEDIUM' then 2 when 'LOW' then 1 end)")
.iterator().next().nullsLast()); .iterator().next().nullsLast());
case "name" -> Sort.by(JpaSort.unsafe(order.getDirection(), "lower(sg.name)").iterator().next());
case "lastPlayedAt" -> Sort.by(order.nullsLast());
case "timeToBeat" -> Sort.by(JpaSort.unsafe(order.getDirection(), "tb.normallyMinutes")
.iterator().next().nullsLast());
default -> Sort.by(order); default -> Sort.by(order);
}); });
} }
@@ -428,9 +448,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*/ */
static String containsPattern(String query) { static String containsPattern(String query) {
if (query == null || query.isBlank()) return null; if (query == null || query.isBlank()) return null;
String escaped = query.trim().toLowerCase(Locale.ROOT) return "%" + LikePattern.escapeLower(query.trim()) + "%";
.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_");
return "%" + escaped + "%";
} }
/** /**
@@ -450,6 +468,16 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
"core_service.modules.library.domain.GameStatus.PLAYING)") "core_service.modules.library.domain.GameStatus.PLAYING)")
int finishIfUntouched(@Param("userId") Long userId, @Param("appid") Long appid); int finishIfUntouched(@Param("userId") Long userId, @Param("appid") Long appid);
/**
* Games the user removed (MANUAL and PSN rows); a Steam row the sync removed is not listed, the next sync would
* remove it again.
*/
@Query("select new core_service.modules.library.domain.RemovedGame(ug.appid, sg.name, sg.imageUrl, ug.source, ug.platform, ug.removedAt) " +
"from UserGame ug join SteamGame sg on sg.appid = ug.appid " +
"where ug.userId = :userId and ug.removedAt is not null " +
"and ug.source <> core_service.modules.library.domain.GameSource.STEAM order by ug.removedAt desc, ug.appid")
List<RemovedGame> findUserRemoved(@Param("userId") Long userId);
/** /**
* Only games with an unlocked achievement: the others cannot get an estimate. * Only games with an unlocked achievement: the others cannot get an estimate.
*/ */
@@ -103,7 +103,7 @@ public class UserStatsRepository {
private static final String PILE_TTB_SET = "select ug.appid, t.normally_minutes as minutes, " + NO_EVENT + ", " private static final String PILE_TTB_SET = "select ug.appid, t.normally_minutes as minutes, " + NO_EVENT + ", "
+ NO_MOVE + " " + PILE_TTB_FROM + "and t.normally_minutes is not null"; + NO_MOVE + " " + PILE_TTB_FROM + "and t.normally_minutes is not null";
private static final String LONGEST_DAY_SET = "select coalesce(e.appid, p.appid) as appid, sum(p.minutes) as minutes, " private static final String LONGEST_DAY_SET = "select coalesce(e.appid, p.appid) as appid, least(sum(p.minutes), 1440) as minutes, "
+ NO_EVENT + ", " + NO_MOVE + " " + ENTRY_DAYS + "and p.day = :arg group by coalesce(e.appid, p.appid)"; + NO_EVENT + ", " + NO_MOVE + " " + ENTRY_DAYS + "and p.day = :arg group by coalesce(e.appid, p.appid)";
private static final String FINISHED_YEAR_SET = "select ug.appid, " + NO_MINUTES + ", " + FINISHED_AT private static final String FINISHED_YEAR_SET = "select ug.appid, " + NO_MINUTES + ", " + FINISHED_AT
+ " as event_at, " + NO_MOVE + " " + FINISHED_FROM + "and " + FINISHED_YEAR + " = :arg"; + " as event_at, " + NO_MOVE + " " + FINISHED_FROM + "and " + FINISHED_YEAR + " = :arg";
@@ -114,11 +114,11 @@ public class UserStatsRepository {
private static final Map<StatKey, String> SETS = Map.ofEntries( private static final Map<StatKey, String> SETS = Map.ofEntries(
Map.entry(StatKey.TOTAL, counted("true")), Map.entry(StatKey.TOTAL, counted("true")),
Map.entry(StatKey.STATUS, counted("ug.status = :arg")), Map.entry(StatKey.STATUS, counted("ug.status = :arg")),
Map.entry(StatKey.PLAYED, counted("ug.playtime_minutes > 0")), Map.entry(StatKey.PLAYED, entries("ug.playtime_minutes > 0")),
Map.entry(StatKey.NEVER_STARTED, counted("ug.playtime_minutes = 0")), Map.entry(StatKey.NEVER_STARTED, entries("ug.playtime_minutes = 0")),
Map.entry(StatKey.PILE, counted("ug.status = 'BACKLOG' and " + UserGameRepository.NOT_COMING)), Map.entry(StatKey.PILE, counted("ug.status = 'BACKLOG' and " + UserGameRepository.NOT_COMING)),
Map.entry(StatKey.FINISHED_ALL, counted("ug.status in " + FINISHED)), Map.entry(StatKey.FINISHED_ALL, counted("ug.status in " + FINISHED)),
Map.entry(StatKey.UNDER_ONE_HOUR, counted(UNDER_ONE_HOUR)), Map.entry(StatKey.UNDER_ONE_HOUR, entries(UNDER_ONE_HOUR)),
Map.entry(StatKey.FINISHED_UNTRACKED, counted(UNTRACKED)), Map.entry(StatKey.FINISHED_UNTRACKED, counted(UNTRACKED)),
Map.entry(StatKey.PILE_TIME_TO_BEAT, PILE_TTB_SET), Map.entry(StatKey.PILE_TIME_TO_BEAT, PILE_TTB_SET),
Map.entry(StatKey.MOST_PLAYED, "select ug.appid, ug.playtime_minutes as minutes, " + NO_EVENT + ", " + NO_MOVE Map.entry(StatKey.MOST_PLAYED, "select ug.appid, ug.playtime_minutes as minutes, " + NO_EVENT + ", " + NO_MOVE
@@ -168,9 +168,10 @@ public class UserStatsRepository {
return jdbc.query(""" return jdbc.query("""
select ug.appid, sg.name, sg.image_url, ug.first_seen_at select ug.appid, sg.name, sg.image_url, ug.first_seen_at
from counted_user_game ug join steam_game sg on sg.appid = ug.appid from counted_user_game ug join steam_game sg on sg.appid = ug.appid
where %s and ug.status = 'BACKLOG' and ug.playtime_minutes = 0 and %s join %s ep on ep.appid = ug.appid
where %s and ug.status = 'BACKLOG' and ep.playtime_minutes = 0 and %s
order by ug.first_seen_at, ug.appid limit 1 order by ug.first_seen_at, ug.appid limit 1
""".formatted(OF_USER, UserGameRepository.NOT_COMING), user(userId), """.formatted(ENTRY_PLAYTIME, OF_USER, UserGameRepository.NOT_COMING), user(userId),
(rs, i) -> new UserStats.OldestGame(rs.getLong("appid"), rs.getString("name"), (rs, i) -> new UserStats.OldestGame(rs.getLong("appid"), rs.getString("name"),
rs.getString("image_url"), rs.getTimestamp("first_seen_at").toInstant())) rs.getString("image_url"), rs.getTimestamp("first_seen_at").toInstant()))
.stream().findFirst(); .stream().findFirst();
@@ -241,7 +242,7 @@ public class UserStatsRepository {
} }
public Optional<DayMinutes> longestDay(long userId) { public Optional<DayMinutes> longestDay(long userId) {
return jdbc.query("select p.day, sum(p.minutes) as minutes " + PLAYTIME_DAYS_NOT_HIDDEN + return jdbc.query("select p.day, least(sum(p.minutes), 1440) as minutes " + PLAYTIME_DAYS_NOT_HIDDEN +
"group by p.day order by minutes desc, p.day desc limit 1", user(userId), "group by p.day order by minutes desc, p.day desc limit 1", user(userId),
(rs, i) -> new DayMinutes(rs.getDate("day").toLocalDate(), rs.getLong("minutes"))) (rs, i) -> new DayMinutes(rs.getDate("day").toLocalDate(), rs.getLong("minutes")))
.stream().findFirst(); .stream().findFirst();
@@ -251,8 +252,8 @@ public class UserStatsRepository {
return jdbc.queryForObject(""" return jdbc.queryForObject("""
select count(*) filter (where ug.playtime_minutes > 0) as started, select count(*) filter (where ug.playtime_minutes > 0) as started,
count(*) filter (where %s) as under_one_hour count(*) filter (where %s) as under_one_hour
from counted_user_game ug where %s from %s ug
""".formatted(UNDER_ONE_HOUR, OF_USER), user(userId), """.formatted(UNDER_ONE_HOUR, ENTRY_PLAYTIME), user(userId),
(rs, i) -> new UserStats.PlayedCounts(rs.getLong("started"), rs.getLong("under_one_hour"))); (rs, i) -> new UserStats.PlayedCounts(rs.getLong("started"), rs.getLong("under_one_hour")));
} }
@@ -321,6 +322,11 @@ public class UserStatsRepository {
return games != null ? games : 0L; return games != null ? games : 0L;
} }
/** The entries whose set total (V45) matches {@code where}; alias ug. */
private static String entries(String where) {
return "select ug.appid, %s from %s ug where %s".formatted(NO_STAT, ENTRY_PLAYTIME, where);
}
private static String counted(String where) { private static String counted(String where) {
return "select ug.appid, %s from counted_user_game ug where %s and %s".formatted(NO_STAT, OF_USER, where); return "select ug.appid, %s from counted_user_game ug where %s and %s".formatted(NO_STAT, OF_USER, where);
} }
@@ -36,7 +36,6 @@ import java.util.HashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.OptionalInt;
import java.util.Optional; import java.util.Optional;
import java.util.Set; import java.util.Set;
import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentHashMap;
@@ -135,12 +134,26 @@ public class AchievementSyncService {
} }
public void refreshSchemaIfStale(Long appid) { public void refreshSchemaIfStale(Long appid) {
Optional<GameAchievementSchemaMeta> existing = schemaMetaRepository.findById(appid); if (schemaStale(appid)) {
if (existing.isEmpty() || isStale(existing.get())) {
refreshSchema(appid, false, false); refreshSchema(appid, false, false);
} }
} }
/**
* @return true when the schema meta of the game is missing or older than the schema TTL
*/
public boolean schemaStale(Long appid) {
return schemaMetaRepository.findById(appid).map(this::isStale).orElse(true);
}
/**
* @return true when the user's achievement meta of the game is missing or older than ttl
*/
public boolean userStale(Long userId, Long appid, Duration ttl) {
return userAchievementMetaRepository.findByUserIdAndAppid(userId, appid)
.map(meta -> meta.getRefreshedAt().isBefore(Instant.now().minus(ttl))).orElse(true);
}
/** /**
* Sets FINISHED when the user unlocked the ending achievement (see {@link EndingAchievement}) of a Steam game whose * Sets FINISHED when the user unlocked the ending achievement (see {@link EndingAchievement}) of a Steam game whose
* status they never chose. Uses stored rows only. * status they never chose. Uses stored rows only.
@@ -169,8 +182,7 @@ public class AchievementSyncService {
} }
public void refreshUserIfStale(Long userId, String steamId, Long appid, int playtimeMinutes, Duration ttl) { public void refreshUserIfStale(Long userId, String steamId, Long appid, int playtimeMinutes, Duration ttl) {
Optional<UserGameAchievementMeta> existing = userAchievementMetaRepository.findByUserIdAndAppid(userId, appid); if (userStale(userId, appid, ttl)) {
if (existing.isEmpty() || existing.get().getRefreshedAt().isBefore(Instant.now().minus(ttl))) {
refreshUser(userId, steamId, appid, playtimeMinutes); refreshUser(userId, steamId, appid, playtimeMinutes);
} }
} }
@@ -237,12 +249,17 @@ public class AchievementSyncService {
return Outcome.UNCHANGED; return Outcome.UNCHANGED;
} }
pause(); pause();
OptionalInt unlocked = refreshUser(userId, steamId, appid, playtime); Optional<Unlocks> refreshed = refreshUser(userId, steamId, appid, playtime);
if (unlocked.isEmpty()) return Outcome.HIDDEN; if (refreshed.isEmpty()) return Outcome.HIDDEN;
Unlocks unlocks = refreshed.get();
// Without an earlier visible refresh there is no baseline: the first refresh would show every unlock. // Without an earlier visible refresh there is no baseline: the first refresh would show every unlock.
if (meta != null && meta.isStatsVisible() && unlocked.getAsInt() > meta.getAchievementsUnlocked()) { if (meta != null && meta.isStatsVisible() && unlocks.unlocked() > meta.getAchievementsUnlocked()) {
syncChangeRepository.add(userId, syncChangeRepository.add(userId,
SyncChange.achievements(appid, unlocked.getAsInt() - meta.getAchievementsUnlocked())); SyncChange.achievements(appid, unlocks.unlocked() - meta.getAchievementsUnlocked()));
// Only the step to all: a game at all achievements before this sync gets no line.
if (unlocks.unlocked() == unlocks.total() && game.getStatus() != GameStatus.COMPLETED) {
syncChangeRepository.add(userId, SyncChange.allUnlocked(appid));
}
} }
return Outcome.VISIBLE; return Outcome.VISIBLE;
} }
@@ -300,9 +317,10 @@ public class AchievementSyncService {
} }
/** /**
* @return the unlocked count, empty when Steam hides the achievements of this user * @return the unlocked count and the number of achievements Steam listed, empty when Steam hides the achievements
* of this user
*/ */
private OptionalInt refreshUser(Long userId, String steamId, Long appid, int playtimeMinutes) { private Optional<Unlocks> refreshUser(Long userId, String steamId, Long appid, int playtimeMinutes) {
Optional<SteamPlayerAchievementsResponse.PlayerStats> response = steamApiService.getPlayerAchievements(steamId, appid); Optional<SteamPlayerAchievementsResponse.PlayerStats> response = steamApiService.getPlayerAchievements(steamId, appid);
boolean visible = response.isPresent() && (response.get().success() == null || response.get().success()); boolean visible = response.isPresent() && (response.get().success() == null || response.get().success());
List<UserGameAchievement> rows = visible && response.get().achievements() != null List<UserGameAchievement> rows = visible && response.get().achievements() != null
@@ -326,7 +344,7 @@ public class AchievementSyncService {
userAchievementMetaRepository.save( userAchievementMetaRepository.save(
new UserGameAchievementMeta(userId, appid, Instant.now(), visible, unlockedCount, playtimeAtRefresh)); new UserGameAchievementMeta(userId, appid, Instant.now(), visible, unlockedCount, playtimeAtRefresh));
}); });
return visible ? OptionalInt.of(unlockedCount) : OptionalInt.empty(); return visible ? Optional.of(new Unlocks(unlockedCount, rows.size())) : Optional.empty();
} }
/** /**
@@ -364,5 +382,7 @@ public class AchievementSyncService {
} }
} }
private record Unlocks(int unlocked, int total) {}
private enum Outcome { UNCHANGED, VISIBLE, HIDDEN } private enum Outcome { UNCHANGED, VISIBLE, HIDDEN }
} }
@@ -9,6 +9,7 @@ import core_service.modules.library.api.dto.GameDetailDto;
import core_service.modules.library.api.dto.PlatformPlayDto; import core_service.modules.library.api.dto.PlatformPlayDto;
import core_service.modules.library.domain.AchievementKind; import core_service.modules.library.domain.AchievementKind;
import core_service.modules.library.domain.AchievementProgress; import core_service.modules.library.domain.AchievementProgress;
import core_service.modules.library.domain.CatalogPreview;
import core_service.modules.library.domain.ComingGame; import core_service.modules.library.domain.ComingGame;
import core_service.modules.library.domain.Edition; import core_service.modules.library.domain.Edition;
import core_service.modules.library.domain.GameAchievementSchema; import core_service.modules.library.domain.GameAchievementSchema;
@@ -98,6 +99,15 @@ public class GameDetailService {
* @param steamId null when the user has none; then the stored achievements are shown without a refresh * @param steamId null when the user has none; then the stored achievements are shown without a refresh
*/ */
public Optional<GameDetailDto> getGameDetail(Long userId, String steamId, Long askedAppid) { public Optional<GameDetailDto> getGameDetail(Long userId, String steamId, Long askedAppid) {
return getGameDetail(userId, steamId, askedAppid, true);
}
/**
* @param refreshAchievements false answers from stored data and only says in {@code achievementsStale} that the
* Steam refresh is due
*/
public Optional<GameDetailDto> getGameDetail(Long userId, String steamId, Long askedAppid,
boolean refreshAchievements) {
Long appid = askedAppid; Long appid = askedAppid;
Optional<UserGame> userGameOpt = userGameRepository.findById(new UserGameId(userId, appid)); Optional<UserGame> userGameOpt = userGameRepository.findById(new UserGameId(userId, appid));
if (userGameOpt.isEmpty()) { if (userGameOpt.isEmpty()) {
@@ -112,17 +122,31 @@ public class GameDetailService {
return Optional.empty(); return Optional.empty();
} }
SteamGame meta = steamGameRepository.findById(appid).orElse(null); SteamGame meta = steamGameRepository.findById(appid).orElse(null);
Refreshed refreshed = refresh(userGameOpt.get(), steamId); Refreshed refreshed = refresh(userGameOpt.get(), steamId, refreshAchievements);
return Optional.of(assemble(refreshed.userGame(), meta, refreshed.hasSchema())); return Optional.of(assemble(refreshed.userGame(), meta, refreshed.hasSchema(), refreshed.achievementsStale()));
}
/**
* Catalog data of a key the caller may not own. No call to Steam or IGDB.
*
* @return empty when the catalog has no such game
*/
public Optional<CatalogPreview> preview(long askedAppid) {
long appid = mergeRepository.findTarget(askedAppid);
return steamGameRepository.findById(appid).map(game -> new CatalogPreview(appid, game.getName(),
game.getImageUrl(),
gameMoodsRepository.findById(appid).map(GameMoods::getGenres).orElse(List.of()),
gameTimeToBeatRepository.findById(appid).orElse(null)));
} }
/** /**
* @param userGame the game after the refresh, re-read when the finish rule changed its status * @param userGame the game after the refresh, re-read when the finish rule changed its status
* @param achievementsStale a due Steam refresh was skipped
*/ */
private record Refreshed(UserGame userGame, boolean hasSchema) {} private record Refreshed(UserGame userGame, boolean hasSchema, boolean achievementsStale) {}
// May call Steam, IGDB and speedrun.com and write. // May call Steam, IGDB and speedrun.com and write.
private Refreshed refresh(UserGame userGame, String steamId) { private Refreshed refresh(UserGame userGame, String steamId, boolean refreshAchievements) {
Long userId = userGame.getUserId(); Long userId = userGame.getUserId();
Long appid = userGame.getAppid(); Long appid = userGame.getAppid();
// A hidden game gets no calls to Steam, IGDB or speedrun.com; it shows what is stored. // A hidden game gets no calls to Steam, IGDB or speedrun.com; it shows what is stored.
@@ -138,42 +162,64 @@ public class GameDetailService {
timeToBeatService.refreshInBackground(List.of(appid)); timeToBeatService.refreshInBackground(List.of(appid));
moodService.refreshInBackground(List.of(appid)); moodService.refreshInBackground(List.of(appid));
releaseDateService.refreshInBackground(List.of(appid)); releaseDateService.refreshInBackground(List.of(appid));
return new Refreshed(userGame, false); return new Refreshed(userGame, false, false);
} }
if (!hidden) { boolean stale = !hidden && refreshSchemaIfAsked(appid, refreshAchievements);
try {
achievementSyncService.refreshSchemaIfStale(appid);
} catch (Exception e) {
log.warn("Achievement schema of appid={} not refreshed, showing stored data: {}", appid, e.getMessage());
}
}
boolean hasSchema = schemaMetaRepository.findById(appid).map(GameAchievementSchemaMeta::isHasSchema).orElse(false); boolean hasSchema = schemaMetaRepository.findById(appid).map(GameAchievementSchemaMeta::isHasSchema).orElse(false);
if (!hasSchema) { if (!hasSchema) {
return new Refreshed(userGame, false); return new Refreshed(userGame, false, stale);
} }
if (steamId != null && !hidden) { if (steamId != null && !hidden) {
try { stale |= refreshUserIfAsked(userGame, steamId, refreshAchievements);
achievementSyncService.refreshUserIfStale(userId, steamId, appid, userGame.getPlaytimeMinutes(),
Duration.ofHours(achievementTtlHours));
} catch (Exception e) {
log.warn("Achievements of user={} appid={} not refreshed, showing stored data: {}", userId, appid, e.getMessage());
}
} }
if (!hidden && !userGame.isStatusSetByUser() && achievementSyncService.finishIfEndingUnlocked(userId, appid)) { if (!hidden && !userGame.isStatusSetByUser() && achievementSyncService.finishIfEndingUnlocked(userId, appid)) {
userGame = userGameRepository.findById(new UserGameId(userId, appid)).orElse(userGame); userGame = userGameRepository.findById(new UserGameId(userId, appid)).orElse(userGame);
} }
return new Refreshed(userGame, true); return new Refreshed(userGame, true, stale);
}
/**
* @return true when the refresh was not asked and is due
*/
private boolean refreshSchemaIfAsked(Long appid, boolean refreshAchievements) {
if (!refreshAchievements) {
return achievementSyncService.schemaStale(appid);
}
try {
achievementSyncService.refreshSchemaIfStale(appid);
} catch (Exception e) {
log.warn("Achievement schema of appid={} not refreshed, showing stored data: {}", appid, e.getMessage());
}
return false;
}
/**
* @return true when the refresh was not asked and is due
*/
private boolean refreshUserIfAsked(UserGame userGame, String steamId, boolean refreshAchievements) {
Long userId = userGame.getUserId();
Long appid = userGame.getAppid();
Duration ttl = Duration.ofHours(achievementTtlHours);
if (!refreshAchievements) {
return achievementSyncService.userStale(userId, appid, ttl);
}
try {
achievementSyncService.refreshUserIfStale(userId, steamId, appid, userGame.getPlaytimeMinutes(), ttl);
} catch (Exception e) {
log.warn("Achievements of user={} appid={} not refreshed, showing stored data: {}", userId, appid, e.getMessage());
}
return false;
} }
// Reads only. // Reads only.
private GameDetailDto assemble(UserGame userGame, SteamGame meta, boolean hasSchema) { private GameDetailDto assemble(UserGame userGame, SteamGame meta, boolean hasSchema, boolean achievementsStale) {
List<AchievementSetDto> psnSets = psnSets(userGame); List<AchievementSetDto> psnSets = psnSets(userGame);
List<AchievementDto> psnTrophies = psnSets.isEmpty() ? List.of() : psnTrophies(userGame); List<AchievementDto> psnTrophies = psnSets.isEmpty() ? List.of() : psnTrophies(userGame);
if (!hasSchema) { if (!hasSchema) {
return buildDto(userGame, meta, true, 0, 0, List.of(), List.of(), psnSets, psnTrophies); return buildDto(userGame, meta, true, 0, 0, List.of(), List.of(), psnSets, psnTrophies, achievementsStale);
} }
Long userId = userGame.getUserId(); Long userId = userGame.getUserId();
Long appid = userGame.getAppid(); Long appid = userGame.getAppid();
@@ -206,7 +252,8 @@ public class GameDetailService {
.toList(); .toList();
return buildDto(userGame, meta, statsVisible, schemaList.size(), return buildDto(userGame, meta, statsVisible, schemaList.size(),
AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList, psnSets, psnTrophies); AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList, psnSets, psnTrophies,
achievementsStale);
} }
private List<AchievementSetDto> psnSets(UserGame userGame) { private List<AchievementSetDto> psnSets(UserGame userGame) {
@@ -233,7 +280,8 @@ public class GameDetailService {
private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible, int total, int unlocked, private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible, int total, int unlocked,
List<AchievementDto> achievements, List<GameAchievementSchema> schema, List<AchievementDto> achievements, List<GameAchievementSchema> schema,
List<AchievementSetDto> psnSets, List<AchievementDto> psnTrophies) { List<AchievementSetDto> psnSets, List<AchievementDto> psnTrophies,
boolean achievementsStale) {
Optional<GameTimeToBeat> timeToBeat = gameTimeToBeatRepository.findById(userGame.getAppid()); Optional<GameTimeToBeat> timeToBeat = gameTimeToBeatRepository.findById(userGame.getAppid());
Set<String> unlockedNames = achievements.stream().filter(AchievementDto::unlocked).map(AchievementDto::apiName) Set<String> unlockedNames = achievements.stream().filter(AchievementDto::unlocked).map(AchievementDto::apiName)
.collect(Collectors.toSet()); .collect(Collectors.toSet());
@@ -299,7 +347,8 @@ public class GameDetailService {
coming.map(ComingGame::getReleaseDate).orElse(null), coming.map(ComingGame::getReleaseDate).orElse(null),
sets, sets,
plays.stream().map(line -> toDto(line, userGame, total, unlocked, statsVisible)).toList(), plays.stream().map(line -> toDto(line, userGame, total, unlocked, statsVisible)).toList(),
PlatformPlay.total(plays) PlatformPlay.total(plays),
achievementsStale
); );
} }
@@ -45,10 +45,25 @@ public class GameEditionService {
this.transaction = new TransactionTemplate(transactionManager); this.transaction = new TransactionTemplate(transactionManager);
} }
public void refresh(Collection<Long> appids) { /**
if (!igdbApiService.isEnabled() || appids.isEmpty()) return; * @return the keys stored now; linkSameGames ran for them, but a key without a steam_game row is not linked yet
Set<Long> stale = stale(appids); */
if (stale.isEmpty()) return; public Set<Long> refresh(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return Set.of();
return store(stale(appids));
}
/**
* Like {@link #refresh}, but for keys whose stored edition has no IGDB game: IGDB may know them by now, and a
* fresh null would block a takeover until the TTL ends.
*/
public Set<Long> refreshAgain(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return Set.of();
return store(Set.copyOf(appids));
}
private Set<Long> store(Set<Long> stale) {
if (stale.isEmpty()) return Set.of();
IgdbLookup.Answer<IgdbApiService.Edition> answer = IgdbLookup.byAppid(stale, igdbApiService::getEditions, IgdbLookup.Answer<IgdbApiService.Edition> answer = IgdbLookup.byAppid(stale, igdbApiService::getEditions,
igdbApiService::getEditionsByGameIds); igdbApiService::getEditionsByGameIds);
@@ -63,6 +78,7 @@ public class GameEditionService {
edition == null ? null : edition.sameGameId(), now); edition == null ? null : edition.sameGameId(), now);
} }
linkSameGames(answer.answered()); linkSameGames(answer.answered());
return answer.answered();
} }
/** /**
@@ -3,6 +3,7 @@ package core_service.modules.library.service;
import core_service.modules.library.api.dto.AchievementSummaryDto; import core_service.modules.library.api.dto.AchievementSummaryDto;
import core_service.modules.library.api.dto.ActivityDto; import core_service.modules.library.api.dto.ActivityDto;
import core_service.modules.library.api.dto.BacklogOrderDto; import core_service.modules.library.api.dto.BacklogOrderDto;
import core_service.modules.library.api.dto.DashboardGamesDto;
import core_service.modules.library.api.dto.GameDto; import core_service.modules.library.api.dto.GameDto;
import core_service.modules.library.api.dto.LibrarySummaryDto; import core_service.modules.library.api.dto.LibrarySummaryDto;
import core_service.modules.library.api.dto.PickReasonDto; import core_service.modules.library.api.dto.PickReasonDto;
@@ -10,6 +11,7 @@ import core_service.modules.library.api.dto.PickedGameDto;
import core_service.modules.library.api.dto.PickerCategoryDto; import core_service.modules.library.api.dto.PickerCategoryDto;
import core_service.modules.library.api.dto.PickerDto; import core_service.modules.library.api.dto.PickerDto;
import core_service.modules.library.api.dto.PlatformDto; import core_service.modules.library.api.dto.PlatformDto;
import core_service.modules.library.api.dto.PublicGameDto;
import core_service.modules.library.api.dto.PublicLibraryDto; import core_service.modules.library.api.dto.PublicLibraryDto;
import core_service.modules.library.api.dto.RecentGameDto; import core_service.modules.library.api.dto.RecentGameDto;
import core_service.modules.library.api.dto.WishlistDto; import core_service.modules.library.api.dto.WishlistDto;
@@ -38,6 +40,7 @@ import core_service.modules.library.repo.FriendPlayRepository;
import core_service.modules.library.repo.GameAchievementSchemaMetaRepository; import core_service.modules.library.repo.GameAchievementSchemaMetaRepository;
import core_service.modules.library.repo.GameCreditRepository; import core_service.modules.library.repo.GameCreditRepository;
import core_service.modules.library.repo.GameLinkRepository; import core_service.modules.library.repo.GameLinkRepository;
import core_service.modules.library.repo.GameMergeRepository;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.PlaytimeDayRepository; import core_service.modules.library.repo.PlaytimeDayRepository;
import core_service.modules.library.repo.PsnTrophyRepository; import core_service.modules.library.repo.PsnTrophyRepository;
@@ -107,6 +110,7 @@ public class LibraryQueryService {
private final ReleaseDateService releaseDateService; private final ReleaseDateService releaseDateService;
private final SpeedrunService speedrunService; private final SpeedrunService speedrunService;
private final GameLinkRepository gameLinkRepository; private final GameLinkRepository gameLinkRepository;
private final GameMergeRepository mergeRepository;
private final GameCreditRepository gameCreditRepository; private final GameCreditRepository gameCreditRepository;
private final FriendPlayRepository friendPlayRepository; private final FriendPlayRepository friendPlayRepository;
private final PsnTrophyRepository psnTrophyRepository; private final PsnTrophyRepository psnTrophyRepository;
@@ -134,6 +138,20 @@ public class LibraryQueryService {
return new PageImpl<>(games, page.getPageable(), page.getTotalElements()); return new PageImpl<>(games, page.getPageable(), page.getTotalElements());
} }
/**
* The same rows and order as {@link #listOwnedGames} without a filter, sorted {@code playtimeMinutes,desc} (top)
* and with {@code status=PLAYING} (playing), but without the count query; one enrichment for both lists.
*/
public DashboardGamesDto listDashboardGames(Long userId, int topLimit, int playingLimit) {
Sort byPlaytime = UserGameRepository.querySort(Sort.by(Sort.Direction.DESC, "playtimeMinutes"));
List<UserGame> top = userGameRepository.findListedByStatusIn(userId, EnumSet.allOf(GameStatus.class),
PageRequest.of(0, topLimit, byPlaytime));
List<UserGame> playing = userGameRepository.findListedByStatusIn(userId, List.of(GameStatus.PLAYING),
PageRequest.of(0, playingLimit, byPlaytime));
List<GameDto> dtos = toDtos(Stream.concat(top.stream(), playing.stream()).toList());
return new DashboardGamesDto(dtos.subList(0, top.size()), dtos.subList(top.size(), dtos.size()));
}
/** /**
* One row per {@link PickSource} (next up, next in series, because you played, same studio, friends played, * One row per {@link PickSource} (next up, next in series, because you played, same studio, friends played,
* longest waiting, mood), in the order of its constants, each with up to three backlog games. Budget and exclude * longest waiting, mood), in the order of its constants, each with up to three backlog games. Budget and exclude
@@ -375,7 +393,8 @@ public class LibraryQueryService {
Map<LocalDate, Long> played = playtimeDayRepository.minutesPerDay(userId, from).stream() Map<LocalDate, Long> played = playtimeDayRepository.minutesPerDay(userId, from).stream()
.collect(Collectors.toMap(DayMinutes::day, DayMinutes::minutes)); .collect(Collectors.toMap(DayMinutes::day, DayMinutes::minutes));
List<DayMinutes> perDay = from.datesUntil(today.plusDays(1)) List<DayMinutes> perDay = from.datesUntil(today.plusDays(1))
.map(day -> new DayMinutes(day, played.getOrDefault(day, 0L))) // A day holds at most 24 h; rows from before V18/V49 or several games can sum higher.
.map(day -> new DayMinutes(day, Math.min(played.getOrDefault(day, 0L), 1440)))
.toList(); .toList();
long total = perDay.stream().mapToLong(DayMinutes::minutes).sum(); long total = perDay.stream().mapToLong(DayMinutes::minutes).sum();
@@ -432,6 +451,64 @@ public class LibraryQueryService {
shared.isEmpty() ? null : userStatsService.getStats(userId, shared)); shared.isEmpty() ? null : userStatsService.getStats(userId, shared));
} }
/**
* One listed game of a public profile. Loads the achievement rows of the whole library of the owner and keeps
* the one game.
*
* @param sectionNames see {@link ProfileSection#shown}
* @return empty when the owner does not have the game, removed or hid it, or every field is hidden: all the same
* answer, so the caller learns nothing about a hidden game
*/
@Public
public Optional<PublicGameDto> getPublicGame(Long userId, Collection<String> sectionNames, long askedAppid) {
Set<ProfileSection> sections = ProfileSection.shown(sectionNames);
boolean showStatus = sections.contains(ProfileSection.STATUS) || sections.contains(ProfileSection.PLAYING)
|| sections.contains(ProfileSection.FINISHED);
boolean showSummary = sections.contains(ProfileSection.SUMMARY);
if (!showStatus && !showSummary) {
return Optional.empty();
}
return userGameRepository.findById(new UserGameId(userId, mergeRepository.findTarget(askedAppid)))
.filter(ug -> ug.getRemovedAt() == null && ug.getHiddenAt() == null)
.filter(ug -> showsEveryGame(sections) || listedBySection(sections, ug.getStatus()))
.map(userGame -> toPublicGame(userId, userGame, showStatus, showSummary));
}
/** STATUS and SUMMARY already expose the whole library (status lists, totals). */
private static boolean showsEveryGame(Set<ProfileSection> sections) {
return sections.contains(ProfileSection.STATUS) || sections.contains(ProfileSection.SUMMARY);
}
/** Only PLAYING or FINISHED shown: a game is revealed only when a shown section already lists its status. */
private static boolean listedBySection(Set<ProfileSection> sections, GameStatus status) {
return sections.contains(ProfileSection.PLAYING) && status == GameStatus.PLAYING
|| sections.contains(ProfileSection.FINISHED)
&& (status == GameStatus.FINISHED || status == GameStatus.COMPLETED);
}
private PublicGameDto toPublicGame(Long userId, UserGame userGame, boolean showStatus, boolean showSummary) {
long appid = userGame.getAppid();
SteamGame meta = steamGameRepository.findById(appid).orElse(null);
Integer playtime = showSummary
? gameLinkRepository.entryPlaytimes(userId, List.of(appid)).getOrDefault(appid, userGame.getPlaytimeMinutes())
: null;
AchievementProgress achievements = showSummary ? visibleAchievements(userId, appid) : null;
return new PublicGameDto(appid, meta == null ? null : meta.getName(), meta == null ? null : meta.getImageUrl(),
showStatus ? userGame.getStatus() : null, playtime,
achievements == null ? null : achievements.unlockedCapped(),
achievements == null ? null : achievements.total(),
achievements == null ? null : achievements.percent());
}
/**
* @return null when the game has no achievements or Steam hides them
*/
private AchievementProgress visibleAchievements(Long userId, long appid) {
return AchievementProgress.perGame(progress(userId).stream()
.filter(p -> p.appid() == appid && p.hasAchievements() && p.visible()).toList())
.stream().findFirst().orElse(null);
}
/** /**
* @return null for a hidden section * @return null for a hidden section
*/ */
@@ -10,6 +10,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserSteamSync; import core_service.modules.library.domain.UserSteamSync;
import core_service.modules.library.domain.WishlistGame; import core_service.modules.library.domain.WishlistGame;
import core_service.modules.library.repo.GameCorrectionRepository; import core_service.modules.library.repo.GameCorrectionRepository;
import core_service.modules.library.repo.GameMergeRepository;
import core_service.modules.library.repo.PlaytimeDayRepository; import core_service.modules.library.repo.PlaytimeDayRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.SyncChangeRepository; import core_service.modules.library.repo.SyncChangeRepository;
@@ -59,6 +60,7 @@ public class LibrarySyncWorker {
private final SteamApiService steamApiService; private final SteamApiService steamApiService;
private final SteamGameRepository steamGameRepository; private final SteamGameRepository steamGameRepository;
private final GameCorrectionRepository gameCorrectionRepository; private final GameCorrectionRepository gameCorrectionRepository;
private final GameMergeRepository gameMergeRepository;
private final UserGameRepository userGameRepository; private final UserGameRepository userGameRepository;
private final UserSteamSyncRepository userSteamSyncRepository; private final UserSteamSyncRepository userSteamSyncRepository;
private final SyncChangeRepository syncChangeRepository; private final SyncChangeRepository syncChangeRepository;
@@ -137,6 +139,16 @@ public class LibrarySyncWorker {
} }
// Before the empty check: an account without games can still have a public wishlist. // Before the empty check: an account without games can still have a public wishlist.
Optional<Wishlist> wishlist = hasSteam ? fetchWishlist(steamId, games) : Optional.empty(); Optional<Wishlist> wishlist = hasSteam ? fetchWishlist(steamId, games) : Optional.empty();
List<Long> ownedAppids = games.stream().map(g -> g.appid().longValue()).toList();
// A game added by hand can be a Steam game this sync owns for the first time. Its mapping (game_edition)
// must be known before the save; the refresh after the save then skips these keys.
Set<Long> learned = new HashSet<>();
if (!games.isEmpty() && gameMergeRepository.hasSteamTakeoverCandidates(userId)) {
// First: a fresh edition without an IGDB game (stored before IGDB knew the app) is asked again.
learned.addAll(gameEditionService.refreshAgain(
gameMergeRepository.findUnmappedNewlyOwned(userId, ownedAppids)));
learned.addAll(gameEditionService.refresh(ownedAppids));
}
PsnSyncStep.Prepared psn = psnSyncStep.run(userId); PsnSyncStep.Prepared psn = psnSyncStep.run(userId);
boolean saved = transaction.execute(tx -> { boolean saved = transaction.execute(tx -> {
List<SyncChange> wishlistAdds = wishlist.map(found -> saveWishlist(userId, found)).orElseGet(() -> { List<SyncChange> wishlistAdds = wishlist.map(found -> saveWishlist(userId, found)).orElseGet(() -> {
@@ -152,7 +164,7 @@ public class LibrarySyncWorker {
} }
List<SyncChange> steamChanges = hasSteam ? saveGames(userId, steamId, games, syncStartedAt, playedOn) List<SyncChange> steamChanges = hasSteam ? saveGames(userId, steamId, games, syncStartedAt, playedOn)
: new ArrayList<>(); : new ArrayList<>();
List<SyncChange> psnChanges = psnSyncStep.apply(userId, psn, syncStartedAt, syncDay); List<SyncChange> psnChanges = psnSyncStep.apply(userId, psn, syncStartedAt, syncDay, zone);
// The first sync of an account is a baseline: a count, not a list. // The first sync of an account is a baseline: a count, not a list.
int imported = 0; int imported = 0;
List<SyncChange> listed = new ArrayList<>(); List<SyncChange> listed = new ArrayList<>();
@@ -174,8 +186,12 @@ public class LibrarySyncWorker {
if (!saved) { if (!saved) {
return; return;
} }
// Keys stored before their steam_game row existed were not linked by that refresh.
if (!learned.isEmpty()) {
gameEditionService.linkSameGames(learned);
}
// Also the games added by hand: here a failed refresh from the add is asked again. // Also the games added by hand: here a failed refresh from the add is asked again.
List<Long> igdbAppids = new ArrayList<>(games.stream().map(g -> g.appid().longValue()).toList()); List<Long> igdbAppids = new ArrayList<>(ownedAppids);
igdbAppids.addAll(userGameRepository.findActiveManualAppids(userId)); igdbAppids.addAll(userGameRepository.findActiveManualAppids(userId));
igdbAppids.removeAll(new HashSet<>(userGameRepository.findHiddenAppids(userId))); igdbAppids.removeAll(new HashSet<>(userGameRepository.findHiddenAppids(userId)));
timeToBeatService.refresh(igdbAppids); timeToBeatService.refresh(igdbAppids);
@@ -214,6 +230,10 @@ public class LibrarySyncWorker {
Set<Long> correctedNames = gameCorrectionRepository.findCorrectedAppids(CorrectionField.NAME, appids); Set<Long> correctedNames = gameCorrectionRepository.findCorrectedAppids(CorrectionField.NAME, appids);
Map<Long, SteamGame> knownGames = steamGameRepository.findAllById(appids).stream() Map<Long, SteamGame> knownGames = steamGameRepository.findAllById(appids).stream()
.collect(Collectors.toMap(SteamGame::getAppid, game -> game)); .collect(Collectors.toMap(SteamGame::getAppid, game -> game));
for (SteamOwnedGamesResponse.Game g : games) {
upsertGame(g, knownGames.get(g.appid().longValue()), now, correctedNames);
}
takeOverManualGames(userId, appids);
Map<Long, UserGame> userGames = userGameRepository.findByUserId(userId).stream() Map<Long, UserGame> userGames = userGameRepository.findByUserId(userId).stream()
.collect(Collectors.toMap(UserGame::getAppid, game -> game)); .collect(Collectors.toMap(UserGame::getAppid, game -> game));
List<SyncChange> changes = new ArrayList<>(); List<SyncChange> changes = new ArrayList<>();
@@ -226,7 +246,6 @@ public class LibrarySyncWorker {
List<Long> played = new ArrayList<>(); List<Long> played = new ArrayList<>();
for (SteamOwnedGamesResponse.Game g : games) { for (SteamOwnedGamesResponse.Game g : games) {
Long appid = g.appid().longValue(); Long appid = g.appid().longValue();
upsertGame(g, knownGames.get(appid), now, correctedNames);
if (upsertUserGame(userId, g, userGames.get(appid), now, playedOn, changes)) { if (upsertUserGame(userId, g, userGames.get(appid), now, playedOn, changes)) {
played.add(appid); played.add(appid);
} }
@@ -243,6 +262,21 @@ public class LibrarySyncWorker {
return changes; return changes;
} }
/**
* A game the user added by hand for a PC platform becomes the Steam row of its appid: the row moves to the
* Steam key here, and upsertUserGame takes it over. Runs before any UserGame is loaded in this transaction,
* so the persistence context holds no row under the old key.
*/
private void takeOverManualGames(Long userId, List<Long> appids) {
// user_game refers to steam_game; the JDBC update below does not flush JPA.
steamGameRepository.flush();
gameMergeRepository.findSteamTakeovers(userId, appids).forEach((from, to) -> {
if (gameMergeRepository.rekeyUserGame(userId, from, to)) {
log.info("Sync user={} moved the game added by hand appid={} to its Steam appid={}", userId, from, to);
}
});
}
private record Wishlist(List<SteamApiService.WishlistItem> wanted, Map<Long, SteamApiService.StoreItem> store) {} private record Wishlist(List<SteamApiService.WishlistItem> wanted, Map<Long, SteamApiService.StoreItem> store) {}
private Optional<Wishlist> fetchWishlist(String steamId, List<SteamOwnedGamesResponse.Game> ownedGames) { private Optional<Wishlist> fetchWishlist(String steamId, List<SteamOwnedGamesResponse.Game> ownedGames) {
@@ -304,6 +338,7 @@ public class LibrarySyncWorker {
Long appid = g.appid().longValue(); Long appid = g.appid().longValue();
int playtime = g.playtimeForever() != null ? g.playtimeForever() : 0; int playtime = g.playtimeForever() != null ? g.playtimeForever() : 0;
// Hours typed by hand are no Steam playtime, so the first Steam value of a game added by hand is a baseline. // Hours typed by hand are no Steam playtime, so the first Steam value of a game added by hand is a baseline.
// Also a row moved here from minus its IGDB id (takeOverManualGames).
boolean takesOverManualGame = existing != null && existing.getSource() != GameSource.STEAM; boolean takesOverManualGame = existing != null && existing.getSource() != GameSource.STEAM;
int previousPlaytime = existing != null ? existing.getPlaytimeMinutes() : 0; int previousPlaytime = existing != null ? existing.getPlaytimeMinutes() : 0;
int playtime2Weeks = g.playtime2Weeks() != null ? g.playtime2Weeks() : 0; int playtime2Weeks = g.playtime2Weeks() != null ? g.playtime2Weeks() : 0;
@@ -7,10 +7,13 @@ import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.IgdbGame; import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameSource; import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.RemovedGame;
import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.SteamGame;
import core_service.modules.library.domain.TitlePlay;
import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId; import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.GameMergeRepository; import core_service.modules.library.repo.GameMergeRepository;
import core_service.modules.library.repo.PsnTitleRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.shared.security.RequiresRole; import core_service.shared.security.RequiresRole;
@@ -29,6 +32,7 @@ import java.util.Comparator;
import java.util.HashSet; import java.util.HashSet;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Objects;
import java.util.Optional; import java.util.Optional;
import java.util.Set; import java.util.Set;
import java.util.function.Consumer; import java.util.function.Consumer;
@@ -69,6 +73,7 @@ public class ManualGameService {
private final GameEditionService gameEditionService; private final GameEditionService gameEditionService;
private final GameCreditService gameCreditService; private final GameCreditService gameCreditService;
private final ReleaseDateService releaseDateService; private final ReleaseDateService releaseDateService;
private final PsnTitleRepository psnTitleRepository;
private final TransactionTemplate transaction; private final TransactionTemplate transaction;
public ManualGameService(IgdbApiService igdbApiService, UserGameRepository userGameRepository, public ManualGameService(IgdbApiService igdbApiService, UserGameRepository userGameRepository,
@@ -76,7 +81,7 @@ public class ManualGameService {
TimeToBeatService timeToBeatService, MoodService moodService, TimeToBeatService timeToBeatService, MoodService moodService,
GameEditionService gameEditionService, GameCreditService gameCreditService, GameEditionService gameEditionService, GameCreditService gameCreditService,
ReleaseDateService releaseDateService, ReleaseDateService releaseDateService,
PlatformTransactionManager transactionManager) { PsnTitleRepository psnTitleRepository, PlatformTransactionManager transactionManager) {
this.igdbApiService = igdbApiService; this.igdbApiService = igdbApiService;
this.userGameRepository = userGameRepository; this.userGameRepository = userGameRepository;
this.steamGameRepository = steamGameRepository; this.steamGameRepository = steamGameRepository;
@@ -86,6 +91,7 @@ public class ManualGameService {
this.gameEditionService = gameEditionService; this.gameEditionService = gameEditionService;
this.gameCreditService = gameCreditService; this.gameCreditService = gameCreditService;
this.releaseDateService = releaseDateService; this.releaseDateService = releaseDateService;
this.psnTitleRepository = psnTitleRepository;
this.transaction = new TransactionTemplate(transactionManager); this.transaction = new TransactionTemplate(transactionManager);
} }
@@ -287,11 +293,51 @@ public class ManualGameService {
} }
/** /**
* Deletes the row with its status, rating and notes. Steam games cannot be removed: the next sync would add * Sets removed_at; the row keeps status, rating, notes and history, and {@link #restore} brings it back. Steam
* them again. * games cannot be removed: the next sync would add them again.
*/ */
public ChangeResult remove(Long userId, long appid) { public ChangeResult remove(Long userId, long appid) {
return transaction.execute(tx -> changeManual(userId, appid, userGameRepository::delete)); return transaction.execute(tx -> changeManual(userId, appid, game -> game.setRemovedAt(Instant.now())));
}
/**
* Brings a removed game back with its old data. A PSN row takes the stored title play, so no outside call can
* fail; the next sync corrects it. A removed Steam row belongs to the sync and is not restorable.
*/
public ChangeResult restore(Long userId, long appid) {
return transaction.execute(tx -> {
Optional<UserGame> found = userGameRepository.findById(new UserGameId(userId, appid));
if (found.isEmpty()) {
return ChangeResult.NOT_FOUND;
}
UserGame game = found.get();
if (game.getRemovedAt() == null) {
return ChangeResult.DONE;
}
if (game.getSource() == GameSource.STEAM) {
return ChangeResult.STEAM_GAME;
}
game.setRemovedAt(null);
game.setLastSeenAt(Instant.now());
if (game.getSource() == GameSource.PSN) {
applyPsnPlay(userId, game);
}
return ChangeResult.DONE;
});
}
public List<RemovedGame> removedGames(Long userId) {
return userGameRepository.findUserRemoved(userId);
}
private void applyPsnPlay(Long userId, UserGame game) {
List<TitlePlay> plays = psnTitleRepository.findPlays(userId, List.of(game.getAppid()));
if (plays.isEmpty()) {
return;
}
game.setPlaytimeMinutes((int) Math.min(plays.stream().mapToLong(TitlePlay::playtimeMinutes).sum(), Integer.MAX_VALUE));
plays.stream().map(TitlePlay::lastPlayedAt).filter(Objects::nonNull).max(Comparator.naturalOrder())
.ifPresent(game::setLastPlayedAt);
} }
private ChangeResult changeManual(Long userId, long appid, Consumer<UserGame> change) { private ChangeResult changeManual(Long userId, long appid, Consumer<UserGame> change) {
@@ -22,6 +22,7 @@ import org.springframework.transaction.support.TransactionTemplate;
import java.time.Instant; import java.time.Instant;
import java.time.LocalDate; import java.time.LocalDate;
import java.time.ZoneId;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collection; import java.util.Collection;
import java.util.HashMap; import java.util.HashMap;
@@ -47,17 +48,22 @@ public class PsnSyncStep {
* A LINKED title's PSN values, applied in the sync transaction. * A LINKED title's PSN values, applied in the sync transaction.
* *
* @param lastPlayedAt may be null * @param lastPlayedAt may be null
* @param played minutes to book on the day of lastPlayedAt, 0 when the title is a start point or was not played
* since the last sync
*/ */
public record Playtime(long appid, int minutes, Instant lastPlayedAt) {} public record Playtime(long appid, int minutes, Instant lastPlayedAt, int played) {}
/** /**
* What {@link #run} did outside the transaction. * What {@link #run} did outside the transaction.
* *
* @param changes the ADDED rows of this run * @param changes the ADDED rows of this run
* @param plays the title plays {@link #apply} stores; stored there so a skipped or rolled-back apply keeps the
* old baseline
* @param baseline the user had no PSN title before this run * @param baseline the user had no PSN title before this run
*/ */
public record Prepared(List<SyncChange> changes, List<Playtime> playtimes, boolean baseline) { public record Prepared(List<SyncChange> changes, List<Playtime> playtimes, List<TitlePlay> plays,
public static final Prepared NONE = new Prepared(List.of(), List.of(), false); boolean baseline) {
public static final Prepared NONE = new Prepared(List.of(), List.of(), List.of(), false);
} }
private final PsnAccountService accountService; private final PsnAccountService accountService;
@@ -106,26 +112,38 @@ public class PsnSyncStep {
List<GameImport.Row> review = new ArrayList<>(); List<GameImport.Row> review = new ArrayList<>();
for (GameImport.Line line : lines) { for (GameImport.Line line : lines) {
PsnTitle seen = known.get(line.psnKey()); PsnTitle seen = known.get(line.psnKey());
Instant lastPlayedAt = lastPlayed.get(line.psnKey());
if (seen != null && seen.state() == PsnTitle.State.LINKED) { if (seen != null && seen.state() == PsnTitle.State.LINKED) {
playtimes.add(new Playtime(seen.appid(), line.playtimeMinutes(), lastPlayed.get(line.psnKey()))); playtimes.add(new Playtime(seen.appid(), line.playtimeMinutes(), lastPlayedAt,
played(seen, line.playtimeMinutes(), lastPlayedAt)));
} else if (seen == null && !open.contains(line.psnKey())) { } else if (seen == null && !open.contains(line.psnKey())) {
matchNew(userId, line, lastPlayed.get(line.psnKey()), changes, review); matchNew(userId, line, lastPlayedAt, changes, review);
} }
} }
saveReview(userId, review); saveReview(userId, review);
psnTitleRepository.savePlays(userId, lines.stream().map(line -> new TitlePlay(line.psnKey(), null, List<TitlePlay> plays = lines.stream().map(line -> new TitlePlay(line.psnKey(), null,
line.platform(), line.playtimeMinutes(), lastPlayed.get(line.psnKey()))).toList()); line.platform(), line.playtimeMinutes(), lastPlayed.get(line.psnKey()))).toList();
int added = changes.size(); int added = changes.size();
changes.addAll(trophyStep.run(userId, found.get().accessToken(), titles)); changes.addAll(trophyStep.run(userId, found.get().accessToken(), titles));
log.info("PSN sync user={} added {} titles, {} wait for review, {} games gained trophies", userId, added, log.info("PSN sync user={} added {} titles, {} wait for review, {} games gained trophies", userId, added,
review.size(), changes.size() - added); review.size(), changes.size() - added);
return new Prepared(changes, mergedKeys(playtimes), baseline); return new Prepared(changes, mergedKeys(playtimes), plays, baseline);
} catch (RuntimeException e) { } catch (RuntimeException e) {
log.error("PSN step failed for user {}", userId, e); log.error("PSN step failed for user {}", userId, e);
return new Prepared(changes, playtimes, baseline); return new Prepared(changes, playtimes, List.of(), baseline);
} }
} }
// Only the title's own new play counts: no stored play is a start point, and a total that grew without a newer
// last played is no play of this sync.
private static int played(PsnTitle seen, int minutes, Instant lastPlayedAt) {
if (seen.playtimeMinutes() == null || lastPlayedAt == null
|| (seen.lastPlayedAt() != null && !lastPlayedAt.isAfter(seen.lastPlayedAt()))) {
return 0;
}
return Math.max(0, minutes - seen.playtimeMinutes());
}
private Optional<PsnAccountService.FetchResult.Titles> fetch(Long userId) { private Optional<PsnAccountService.FetchResult.Titles> fetch(Long userId) {
switch (accountService.fetchTitles(userId)) { switch (accountService.fetchTitles(userId)) {
case PsnAccountService.FetchResult.Titles found -> { case PsnAccountService.FetchResult.Titles found -> {
@@ -194,7 +212,7 @@ public class PsnSyncStep {
private List<Playtime> mergedKeys(List<Playtime> playtimes) { private List<Playtime> mergedKeys(List<Playtime> playtimes) {
Map<Long, Long> targets = mergeRepository.findTargets(playtimes.stream().map(Playtime::appid).toList()); Map<Long, Long> targets = mergeRepository.findTargets(playtimes.stream().map(Playtime::appid).toList());
return playtimes.stream().map(playtime -> new Playtime(targets.getOrDefault(playtime.appid(), playtime.appid()), return playtimes.stream().map(playtime -> new Playtime(targets.getOrDefault(playtime.appid(), playtime.appid()),
playtime.minutes(), playtime.lastPlayedAt())).toList(); playtime.minutes(), playtime.lastPlayedAt(), playtime.played())).toList();
} }
// Several titles can be one game (PS4 and PS5 versions, a review pick): its playtime is their sum, so the row // Several titles can be one game (PS4 and PS5 versions, a review pick): its playtime is their sum, so the row
@@ -205,33 +223,35 @@ public class PsnSyncStep {
byGame.merge(playtime.appid(), playtime, (a, b) -> new Playtime(a.appid(), byGame.merge(playtime.appid(), playtime, (a, b) -> new Playtime(a.appid(),
(int) Math.min((long) a.minutes() + b.minutes(), Integer.MAX_VALUE), (int) Math.min((long) a.minutes() + b.minutes(), Integer.MAX_VALUE),
a.lastPlayedAt() == null || (b.lastPlayedAt() != null && b.lastPlayedAt().isAfter(a.lastPlayedAt())) a.lastPlayedAt() == null || (b.lastPlayedAt() != null && b.lastPlayedAt().isAfter(a.lastPlayedAt()))
? b.lastPlayedAt() : a.lastPlayedAt())); ? b.lastPlayedAt() : a.lastPlayedAt(),
(int) Math.min((long) a.played() + b.played(), Integer.MAX_VALUE)));
} }
return byGame.values(); return byGame.values();
} }
/** /**
* In the sync transaction, after the Steam games are saved: playtime, last played and two-weeks minutes of the * In the sync transaction, after the Steam games are saved: playtime, last played and two-weeks minutes of the
* PSN rows. No status change: PSN rows have statusSetByUser. * PSN rows. The played minutes of a game are booked on the day it was last played, in the given zone. No status
* change: PSN rows have statusSetByUser.
* *
* @return all PSN changes of the run (ADDED and PLAYTIME) * @return all PSN changes of the run (ADDED and PLAYTIME)
*/ */
public List<SyncChange> apply(Long userId, Prepared prepared, Instant now, LocalDate day) { public List<SyncChange> apply(Long userId, Prepared prepared, Instant now, LocalDate day, ZoneId zone) {
List<SyncChange> changes = new ArrayList<>(prepared.changes()); List<SyncChange> changes = new ArrayList<>(prepared.changes());
Map<Long, UserGame> touched = new HashMap<>(); Map<Long, UserGame> touched = new HashMap<>();
for (Playtime playtime : perGame(prepared.playtimes())) { for (Playtime playtime : perGame(prepared.playtimes())) {
userGameRepository.findById(new UserGameId(userId, playtime.appid())) userGameRepository.findById(new UserGameId(userId, playtime.appid()))
.filter(game -> game.getSource() == GameSource.PSN && game.getRemovedAt() == null) .filter(game -> game.getSource() == GameSource.PSN && game.getRemovedAt() == null)
.ifPresent(game -> { .ifPresent(game -> {
int growth = playtime.minutes() - game.getPlaytimeMinutes();
game.setPlaytimeMinutes(playtime.minutes()); game.setPlaytimeMinutes(playtime.minutes());
game.setLastSeenAt(now); game.setLastSeenAt(now);
if (playtime.lastPlayedAt() != null) { if (playtime.lastPlayedAt() != null) {
game.setLastPlayedAt(playtime.lastPlayedAt()); game.setLastPlayedAt(playtime.lastPlayedAt());
} }
if (growth > 0) { if (playtime.played() > 0) {
changes.add(SyncChange.playtime(game.getAppid(), growth)); LocalDate playedOn = LocalDate.ofInstant(playtime.lastPlayedAt(), zone);
playtimeDayRepository.addMinutes(userId, game.getAppid(), day, growth); changes.add(SyncChange.playtime(game.getAppid(), playtime.played()));
playtimeDayRepository.addMinutes(userId, game.getAppid(), playedOn, playtime.played());
} }
touched.put(game.getAppid(), game); touched.put(game.getAppid(), game);
}); });
@@ -243,6 +263,9 @@ public class PsnSyncStep {
touched.forEach((appid, game) -> touched.forEach((appid, game) ->
game.setPlaytime2Weeks((int) Math.min(twoWeeks.getOrDefault(appid, 0L), Integer.MAX_VALUE))); game.setPlaytime2Weeks((int) Math.min(twoWeeks.getOrDefault(appid, 0L), Integer.MAX_VALUE)));
} }
if (!prepared.plays().isEmpty()) {
psnTitleRepository.savePlays(userId, prepared.plays());
}
return changes; return changes;
} }
} }
@@ -6,21 +6,33 @@ import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import java.util.Collection; import java.util.Collection;
import java.util.HashSet; import java.util.HashSet;
import java.util.List; import java.util.List;
import java.util.Optional;
import java.util.Set; import java.util.Set;
import java.util.regex.Matcher; import java.util.regex.Matcher;
import java.util.regex.Pattern; import java.util.regex.Pattern;
/** /**
* Finds a library game among speedrun.com's search hits. speedrun.com has no Steam or IGDB link, so the name must be * Finds a library game among speedrun.com's search hits, or none: a wrong link is worse than no link. speedrun.com has
* the same and, when both years are known, the release year may differ by at most one. * no Steam or IGDB link, so the name must be the same (case, punctuation, trademark signs and roman numerals do not
* count). With known years the year of the hit must be within one of ours. Without a year to decide, only a single
* same-name hit counts.
*/ */
final class SpeedrunMatch { final class SpeedrunMatch {
static final int MAX_CANDIDATES = 2;
static final String WEBLINK_PREFIX = "https://www.speedrun.com/"; static final String WEBLINK_PREFIX = "https://www.speedrun.com/";
// speedrun.com tells remakes apart by the year: "Prey (2017)" next to "Prey". // speedrun.com tells remakes apart by the year: "Prey (2017)" next to "Prey".
private static final Pattern YEAR_SUFFIX = Pattern.compile("^(.*?)\\s*\\((\\d{4})\\)$"); private static final Pattern YEAR_SUFFIX = Pattern.compile("^(.*?)\\s*\\((\\d{4})\\)$");
// Other pages on speedrun.com ("Remastered", "Remake") are left alone on purpose.
private static final Pattern EDITION_SUFFIX = Pattern.compile(
"\\s*[-:\u2013]?\\s*(?:the\\s+)?(?:(?:game of the year|goty|special|definitive|complete|deluxe|digital deluxe|enhanced|gold|"
+ "ultimate|anniversary|legendary|premium) edition|goty|director'?s cut)\\s*$",
Pattern.CASE_INSENSITIVE);
// Index + 2 is the number.
private static final List<String> ROMAN_NUMERALS = List.of("ii", "iii", "iv", "v", "vi", "vii", "viii", "ix", "x",
"xi", "xii", "xiii", "xiv", "xv", "xvi", "xvii", "xviii", "xix", "xx");
// "i" and "x" stay: "Mega Man X" is not "Mega Man 10".
private static final Pattern ROMAN_NUMERAL = Pattern.compile("\\b(?!x\\b)(" + String.join("|", ROMAN_NUMERALS) + ")\\b");
// The column sizes of game_speedrun. // The column sizes of game_speedrun.
private static final int MAX_GAME_ID_LENGTH = 16; private static final int MAX_GAME_ID_LENGTH = 16;
private static final int MAX_WEBLINK_LENGTH = 512; private static final int MAX_WEBLINK_LENGTH = 512;
@@ -39,23 +51,40 @@ final class SpeedrunMatch {
/** /**
* @param years our release years of the game; empty when unknown * @param years our release years of the game; empty when unknown
* @param hits in speedrun.com's order * @param hits in speedrun.com's order
* @return the hits worth asking for runs, best first, at most {@link #MAX_CANDIDATES} * @return the one hit that is this game; empty when no hit is, or when several could be
*/ */
static List<SpeedrunApiService.Game> candidates(String name, Collection<Integer> years, List<SpeedrunApiService.Game> hits) { static Optional<SpeedrunApiService.Game> sure(String name, Collection<Integer> years, List<SpeedrunApiService.Game> hits) {
NameAndYear ours = split(name); NameAndYear ours = split(name);
Set<Integer> ourYears = new HashSet<>(years); Set<Integer> ourYears = new HashSet<>(years);
if (ours.year() != null) ourYears.add(ours.year()); if (ours.year() != null) ourYears.add(ours.year());
return hits.stream() List<SpeedrunApiService.Game> same = hits.stream()
.filter(hit -> hit.id() != null && hit.id().length() <= MAX_GAME_ID_LENGTH) .filter(hit -> hit.id() != null && hit.id().length() <= MAX_GAME_ID_LENGTH)
.filter(hit -> isWeblink(hit.weblink())) .filter(hit -> isWeblink(hit.weblink()))
.filter(hit -> { .filter(hit -> split(hit.name()).name().equals(ours.name()))
NameAndYear theirs = split(hit.name());
Integer year = theirs.year() != null ? theirs.year() : hit.year();
return theirs.name().equals(ours.name()) && (year == null || ourYears.isEmpty()
|| ourYears.stream().anyMatch(ourYear -> Math.abs(ourYear - year) <= 1));
})
.limit(MAX_CANDIDATES)
.toList(); .toList();
if (ourYears.isEmpty()) {
return only(same);
}
List<SpeedrunApiService.Game> dated = same.stream()
.filter(hit -> {
Integer year = yearOf(hit);
return year != null && ourYears.stream().anyMatch(ourYear -> Math.abs(ourYear - year) <= 1);
})
.toList();
if (!dated.isEmpty()) {
return only(dated);
}
return only(same).filter(hit -> yearOf(hit) == null);
}
/**
* @return the name without "Special Edition", "Director's Cut" and the like; empty when it has no such suffix or
* nothing is left
*/
static Optional<String> withoutEditionSuffix(String name) {
String trimmed = name.trim();
String stripped = EDITION_SUFFIX.matcher(trimmed).replaceFirst("").trim();
return stripped.isEmpty() || stripped.equals(trimmed) ? Optional.empty() : Optional.of(stripped);
} }
/** /**
@@ -65,11 +94,25 @@ final class SpeedrunMatch {
return name.replaceAll(GameNames.TRADEMARK_SIGNS, "").trim(); return name.replaceAll(GameNames.TRADEMARK_SIGNS, "").trim();
} }
private static Optional<SpeedrunApiService.Game> only(List<SpeedrunApiService.Game> games) {
return games.size() == 1 ? Optional.of(games.get(0)) : Optional.empty();
}
private static Integer yearOf(SpeedrunApiService.Game hit) {
Integer fromName = split(hit.name()).year();
return fromName != null ? fromName : hit.year();
}
private static String key(String name) {
return ROMAN_NUMERAL.matcher(GameNames.normalize(name)).replaceAll(
match -> String.valueOf(ROMAN_NUMERALS.indexOf(match.group()) + 2));
}
private static NameAndYear split(String name) { private static NameAndYear split(String name) {
Matcher matcher = YEAR_SUFFIX.matcher(name.trim()); Matcher matcher = YEAR_SUFFIX.matcher(name.trim());
return matcher.matches() return matcher.matches()
? new NameAndYear(GameNames.normalize(matcher.group(1)), Integer.parseInt(matcher.group(2))) ? new NameAndYear(key(matcher.group(1)), Integer.parseInt(matcher.group(2)))
: new NameAndYear(GameNames.normalize(name), null); : new NameAndYear(key(name), null);
} }
private record NameAndYear(String name, Integer year) {} private record NameAndYear(String name, Integer year) {}
@@ -1,17 +1,22 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameEdition;
import core_service.modules.library.domain.GameKeys; import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.Mood; import core_service.modules.library.domain.Mood;
import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.SteamGame;
import core_service.modules.library.repo.GameEditionRepository;
import core_service.modules.library.repo.GameSpeedrunRepository; import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService; import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.modules.speedrun_adapter.service.SpeedrunThrottledException;
import core_service.shared.background.BackgroundPools; import core_service.shared.background.BackgroundPools;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.core.task.TaskExecutor; import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.annotation.Async; import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -32,7 +37,17 @@ import java.util.concurrent.atomic.AtomicBoolean;
/** /**
* Stores which games have a speedrun.com page with verified runs, for {@link Mood#SPEEDRUN} and the link on the * Stores which games have a speedrun.com page with verified runs, for {@link Mood#SPEEDRUN} and the link on the
* detail page. speedrun.com is searched by name, one game at a time, so all callers share one queue and one thread * detail page. speedrun.com is searched by name, one game at a time, so all callers share one queue and one thread
* works it off: a game in several backlogs is asked once, and the rate limit has one waiter. * (the speedrun executor) works it off: a game in several backlogs is asked once, and the rate limit has one waiter.
* <p>
* A game is looked up by its own name and years; when that gives no sure hit, an edition is looked up by the name and
* year of its IGDB parent game, a game without a parent by its name without an edition suffix. A sure hit without
* verified runs ends the lookup. See {@link SpeedrunMatch} for "sure".
* <p>
* A rate limit answer (429, 420) pauses the thread (Retry-After, else 1 min doubling up to 30 min) and asks the same
* game again; it never drops the queues.
* <p>
* {@link #recheckMarked} asks the rows a migration marked (checked_at = epoch) again. Those games wait behind every
* game a user asked for.
*/ */
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
@@ -42,27 +57,55 @@ public class SpeedrunService {
static final int BATCH = 50; static final int BATCH = 50;
// speedrun.com is down or blocks us; the rest of the queue is asked on the next trigger. // speedrun.com is down or blocks us; the rest of the queue is asked on the next trigger.
static final int MAX_ERRORS_IN_A_ROW = 3; static final int MAX_ERRORS_IN_A_ROW = 3;
// A small batch, so a user request waits for a few games and not for 50.
static final int RECHECK_BATCH = 10;
static final Duration FIRST_PAUSE = Duration.ofMinutes(1);
static final Duration MAX_PAUSE = Duration.ofMinutes(30);
private final SpeedrunApiService speedrunApiService; private final SpeedrunApiService speedrunApiService;
private final IgdbApiService igdbApiService; private final IgdbApiService igdbApiService;
private final GameSpeedrunRepository gameSpeedrunRepository; private final GameSpeedrunRepository gameSpeedrunRepository;
private final GameEditionRepository gameEditionRepository;
private final SteamGameRepository steamGameRepository; private final SteamGameRepository steamGameRepository;
private final UserGameRepository userGameRepository; private final UserGameRepository userGameRepository;
private final TaskExecutor syncFollowUpExecutor; private final TaskExecutor syncFollowUpExecutor;
// Works both queues off on its own thread, so the follow-up pool never waits for speedrun.com.
private final TaskExecutor speedrunExecutor;
private final ConcurrentSkipListSet<Long> pending = new ConcurrentSkipListSet<>(); private final ConcurrentSkipListSet<Long> pending = new ConcurrentSkipListSet<>();
private final ConcurrentSkipListSet<Long> recheck = new ConcurrentSkipListSet<>();
private final AtomicBoolean running = new AtomicBoolean(); private final AtomicBoolean running = new AtomicBoolean();
@Value("${app.library.speedrun-ttl-days:30}") @Value("${app.library.speedrun-ttl-days:30}")
private long ttlDays; private long ttlDays;
/** /**
* Queues the games without a fresh row and, unless another thread already does it, works the queue off on this * Queues the games without a fresh row and returns; the speedrun thread works the queue off. A game takes up to 3
* thread. A game takes up to 3 requests, and there is 1 request per second. * requests, and there is 1 request every 2 seconds.
*/ */
public void refresh(Collection<Long> appids) { public void refresh(Collection<Long> appids) {
if (!speedrunApiService.isEnabled() || appids.isEmpty()) return; if (!speedrunApiService.isEnabled() || appids.isEmpty()) return;
pending.addAll(appids); pending.addAll(appids);
speedrunExecutor.execute(this::work);
}
/**
* Queues the rows the V47 migration marked, behind the games users asked for.
*/
@EventListener(ApplicationReadyEvent.class)
public void recheckMarked() {
if (!speedrunApiService.isEnabled()) return;
syncFollowUpExecutor.execute(() -> Refreshes.quietly("Speedrun re-check", () -> {
List<Long> marked = gameSpeedrunRepository.findAppidsToRecheck(Instant.EPOCH);
if (marked.isEmpty()) return;
log.info("Speedrun re-check of {} games", marked.size());
recheck.addAll(marked);
speedrunExecutor.execute(this::work);
}));
}
private void work() {
// Another work() is draining: it also takes what was queued meanwhile.
while (running.compareAndSet(false, true)) { while (running.compareAndSet(false, true)) {
try { try {
drain(); drain();
@@ -70,7 +113,7 @@ public class SpeedrunService {
running.set(false); running.set(false);
} }
// Games can be queued after the last poll and before the flag was cleared. // Games can be queued after the last poll and before the flag was cleared.
if (pending.isEmpty()) return; if (Thread.currentThread().isInterrupted() || (pending.isEmpty() && recheck.isEmpty())) return;
} }
} }
@@ -99,12 +142,10 @@ public class SpeedrunService {
private void drain() { private void drain() {
int errorsInARow = 0; int errorsInARow = 0;
while (!pending.isEmpty()) { int throttlesInARow = 0;
List<Long> batch = new ArrayList<>(); while (!pending.isEmpty() || !recheck.isEmpty()) {
Long next; boolean forUsers = !pending.isEmpty();
while (batch.size() < BATCH && (next = pending.pollFirst()) != null) { List<Long> batch = forUsers ? poll(pending, BATCH) : poll(recheck, RECHECK_BATCH);
batch.add(next);
}
batch.removeAll(gameSpeedrunRepository.findFreshAppids(batch, Instant.now().minus(Duration.ofDays(ttlDays)))); batch.removeAll(gameSpeedrunRepository.findFreshAppids(batch, Instant.now().minus(Duration.ofDays(ttlDays))));
if (batch.isEmpty()) continue; if (batch.isEmpty()) continue;
@@ -113,26 +154,47 @@ public class SpeedrunService {
names.put(game.getAppid(), game.getName()); names.put(game.getAppid(), game.getName());
} }
Optional<Map<Long, Set<Integer>>> years = releaseYears(batch); Optional<Map<Long, Set<Integer>>> years = releaseYears(batch);
if (years.isEmpty()) { Optional<Map<Long, IgdbApiService.NameAndYear>> parents = years.isEmpty() ? Optional.empty() : parentNames(batch);
if (parents.isEmpty()) {
// Matching by name only would store a wrong remake for 30 days. // Matching by name only would store a wrong remake for 30 days.
log.warn("Speedrun games of {} not refreshed, IGDB did not answer", batch.size()); log.warn("Speedrun games of {} not refreshed, IGDB did not answer", batch.size());
continue; continue;
} }
for (Long appid : batch) { for (int i = 0; i < batch.size(); i++) {
if (!forUsers && !pending.isEmpty()) {
// A user asked in the meantime: the rest of the re-check goes behind it.
recheck.addAll(batch.subList(i, batch.size()));
break;
}
Long appid = batch.get(i);
String name = names.get(appid); String name = names.get(appid);
if (name == null || name.equals(SteamGame.placeholderName(appid))) continue; if (name == null || name.equals(SteamGame.placeholderName(appid))) continue;
boolean stored; boolean stored;
try { try {
stored = checkAndStore(appid, name, years.get().getOrDefault(appid, Set.of())); stored = checkAndStore(appid, name, years.get().getOrDefault(appid, Set.of()), parents.get().get(appid));
} catch (SpeedrunThrottledException e) {
Duration pause = pauseFor(e.getRetryAfter(), ++throttlesInARow);
log.warn("speedrun.com rate limit, pausing {} ({} games queued)", pause,
pending.size() + recheck.size() + batch.size() - i);
try {
pause(pause);
} catch (InterruptedException stop) {
Thread.currentThread().interrupt();
return;
}
i--; // same game again, after the user check at the loop head
continue;
} catch (Exception e) { } catch (Exception e) {
log.warn("Speedrun game of appid={} not stored: {}", appid, e.getMessage()); log.warn("Speedrun game of appid={} not stored: {}", appid, e.getMessage());
stored = false; stored = false;
} }
if (stored) { if (stored) {
errorsInARow = 0; errorsInARow = 0;
throttlesInARow = 0;
} else if (++errorsInARow >= MAX_ERRORS_IN_A_ROW) { } else if (++errorsInARow >= MAX_ERRORS_IN_A_ROW) {
log.warn("Speedrun check failed {} times in a row, {} queued games dropped", errorsInARow, pending.size()); log.warn("Speedrun check failed {} times in a row, {} queued games dropped", errorsInARow, pending.size());
pending.clear(); pending.clear();
recheck.clear();
return; return;
} }
} }
@@ -140,24 +202,86 @@ public class SpeedrunService {
} }
/** /**
* @param retryAfter the pause speedrun.com asked for; null when it did not say
* @param throttlesInARow throttled answers since the last stored game, at least 1
*/
static Duration pauseFor(Duration retryAfter, int throttlesInARow) {
if (retryAfter != null) {
return Duration.ofSeconds(Math.clamp(retryAfter.toSeconds(), 1, MAX_PAUSE.toSeconds()));
}
Duration backoff = FIRST_PAUSE.multipliedBy(1L << Math.min(throttlesInARow - 1, 5));
return backoff.compareTo(MAX_PAUSE) > 0 ? MAX_PAUSE : backoff;
}
void pause(Duration pause) throws InterruptedException {
Thread.sleep(pause);
}
private static List<Long> poll(ConcurrentSkipListSet<Long> queue, int size) {
List<Long> batch = new ArrayList<>();
Long next;
while (batch.size() < size && (next = queue.pollFirst()) != null) {
batch.add(next);
}
return batch;
}
/**
* @param parent the IGDB game this one is an edition, remaster or port of; null when there is none
* @return false when a speedrun.com call failed; then nothing is stored and the game is asked again later * @return false when a speedrun.com call failed; then nothing is stored and the game is asked again later
*/ */
private boolean checkAndStore(Long appid, String name, Set<Integer> years) { private boolean checkAndStore(Long appid, String name, Set<Integer> years, IgdbApiService.NameAndYear parent) {
Optional<List<SpeedrunApiService.Game>> hits = speedrunApiService.findGames(SpeedrunMatch.searchText(name));
if (hits.isEmpty()) return false;
SpeedrunApiService.Game match = null; SpeedrunApiService.Game match = null;
for (SpeedrunApiService.Game candidate : SpeedrunMatch.candidates(name, years, hits.get())) { for (Lookup lookup : lookups(name, years, parent)) {
Optional<Boolean> hasRuns = speedrunApiService.hasVerifiedRuns(candidate.id()); Optional<List<SpeedrunApiService.Game>> hits = speedrunApiService.findGames(SpeedrunMatch.searchText(lookup.name()));
if (hits.isEmpty()) return false;
Optional<SpeedrunApiService.Game> sure = SpeedrunMatch.sure(lookup.name(), lookup.years(), hits.get());
if (sure.isEmpty()) continue;
Optional<Boolean> hasRuns = speedrunApiService.hasVerifiedRuns(sure.get().id());
if (hasRuns.isEmpty()) return false; if (hasRuns.isEmpty()) return false;
if (hasRuns.get()) { if (hasRuns.get()) match = sure.get();
match = candidate; break;
break;
}
} }
gameSpeedrunRepository.upsert(appid, match == null ? null : match.id(), match == null ? null : match.weblink(), Instant.now()); gameSpeedrunRepository.upsert(appid, match == null ? null : match.id(), match == null ? null : match.weblink(), Instant.now());
return true; return true;
} }
private static List<Lookup> lookups(String name, Set<Integer> years, IgdbApiService.NameAndYear parent) {
List<Lookup> lookups = new ArrayList<>();
lookups.add(new Lookup(name, years));
if (parent != null && !SpeedrunMatch.searchText(parent.name()).equalsIgnoreCase(SpeedrunMatch.searchText(name))) {
lookups.add(new Lookup(parent.name(), parent.year() == null ? Set.of() : Set.of(parent.year())));
} else {
SpeedrunMatch.withoutEditionSuffix(name).ifPresent(plain -> lookups.add(new Lookup(plain, years)));
}
return lookups;
}
private record Lookup(String name, Set<Integer> years) {}
/**
* @return library key to the IGDB game it is an edition, remaster or port of; a game that is its own same game
* has no parent. Empty when IGDB failed.
*/
private Optional<Map<Long, IgdbApiService.NameAndYear>> parentNames(List<Long> appids) {
Map<Long, List<Long>> appidsByParent = new HashMap<>();
for (GameEdition edition : gameEditionRepository.findAllById(appids)) {
if (edition.getIgdbId() != null && edition.getSameGameId() != null
&& !edition.getSameGameId().equals(edition.getIgdbId())) {
appidsByParent.computeIfAbsent(edition.getSameGameId(), id -> new ArrayList<>()).add(edition.getAppid());
}
}
if (appidsByParent.isEmpty() || !igdbApiService.isEnabled()) {
return Optional.of(Map.of());
}
Optional<Map<Long, IgdbApiService.NameAndYear>> found = igdbApiService.getNamesAndYears(appidsByParent.keySet());
if (found.isEmpty()) return Optional.empty();
Map<Long, IgdbApiService.NameAndYear> parents = new HashMap<>();
found.get().forEach((parentId, parent) ->
appidsByParent.get(parentId).forEach(appid -> parents.put(appid, parent)));
return Optional.of(parents);
}
/** /**
* @return empty when IGDB failed. A game IGDB does not know, and every game when IGDB is off, has no years and is * @return empty when IGDB failed. A game IGDB does not know, and every game when IGDB is off, has no years and is
* matched by name only. * matched by name only.
@@ -6,14 +6,17 @@ import core_service.modules.rewards.domain.RewardKind;
import core_service.modules.rewards.service.RewardService; import core_service.modules.rewards.service.RewardService;
import java.time.Instant; import java.time.Instant;
import java.time.LocalDate;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
/** /**
* @param paid the game rows of the game and the other editions of its set * @param paid the game rows of the game and the other editions of its set
* @param earlySince app.rewards.early-since, for the text of COMPLETION_TOO_OLD
* @param missing why a kind is not paid; STARTED, FINISHED (both finish kinds) and COMPLETED * @param missing why a kind is not paid; STARTED, FINISHED (both finish kinds) and COMPLETED
*/ */
public record GamePointsDto(long balance, List<PaidDto> paid, Map<RewardKind, NoPointsReason> missing) { public record GamePointsDto(long balance, List<PaidDto> paid, Map<RewardKind, NoPointsReason> missing,
LocalDate earlySince) {
/** /**
* @param appid the edition the row was paid for * @param appid the edition the row was paid for
@@ -27,6 +30,7 @@ public record GamePointsDto(long balance, List<PaidDto> paid, Map<RewardKind, No
} }
public static GamePointsDto of(RewardService.GamePoints points) { public static GamePointsDto of(RewardService.GamePoints points) {
return new GamePointsDto(points.balance(), points.paid().stream().map(PaidDto::of).toList(), points.missing()); return new GamePointsDto(points.balance(), points.paid().stream().map(PaidDto::of).toList(), points.missing(),
points.earlySince());
} }
} }
@@ -7,6 +7,7 @@ import core_service.modules.rewards.domain.Track;
import core_service.modules.rewards.service.RewardService; import core_service.modules.rewards.service.RewardService;
import java.time.Instant; import java.time.Instant;
import java.time.LocalDate;
import java.util.Arrays; import java.util.Arrays;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@@ -16,9 +17,10 @@ import java.util.Set;
* @param owned the item codes the user may wear * @param owned the item codes the user may wear
* @param equipped the worn item code per slot * @param equipped the worn item code per slot
* @param progress the value of each unlock track * @param progress the value of each unlock track
* @param earlySince app.rewards.early-since, for the rules text
*/ */
public record WalletDto(long balance, List<ItemDto> catalog, Set<String> owned, Map<LookSlot, String> equipped, public record WalletDto(long balance, List<ItemDto> catalog, Set<String> owned, Map<LookSlot, String> equipped,
List<EntryDto> history, Map<Track, Integer> progress) { List<EntryDto> history, Map<Track, Integer> progress, LocalDate earlySince) {
/** /**
* @param track null when the item is buyable * @param track null when the item is buyable
@@ -38,6 +40,6 @@ public record WalletDto(long balance, List<ItemDto> catalog, Set<String> owned,
wallet.owned(), wallet.equipped(), wallet.owned(), wallet.equipped(),
wallet.history().stream() wallet.history().stream()
.map(h -> new EntryDto(h.kind(), h.ref(), h.gameName(), h.points(), h.eventAt())).toList(), .map(h -> new EntryDto(h.kind(), h.ref(), h.gameName(), h.points(), h.eventAt())).toList(),
wallet.progress()); wallet.progress(), wallet.earlySince());
} }
} }
@@ -8,6 +8,8 @@ package core_service.modules.rewards.domain;
* @param startLogged a BACKLOG to PLAYING change is in the status log * @param startLogged a BACKLOG to PLAYING change is in the status log
* @param finishLogged a change to FINISHED or COMPLETED is in the status log * @param finishLogged a change to FINISHED or COMPLETED is in the status log
* @param linkedTitle a PlayStation title is linked to the game, so its playtime days prove nothing * @param linkedTitle a PlayStation title is linked to the game, so its playtime days prove nothing
* @param allAchievements the game has achievements and the user unlocked all of them
*/ */
public record GameRewardFacts(boolean manual, boolean finished, boolean completed, boolean hidden, boolean playstation, public record GameRewardFacts(boolean manual, boolean finished, boolean completed, boolean hidden, boolean playstation,
boolean startLogged, boolean finishLogged, boolean linkedTitle) {} boolean startLogged, boolean finishLogged, boolean linkedTitle,
boolean allAchievements) {}
@@ -30,8 +30,13 @@ public enum NoPointsReason {
WEEKLY_LIMIT, WEEKLY_LIMIT,
/** The weekly cap is used up, so no early points; proof still pays the full finish. */ /** The weekly cap is used up, so no early points; proof still pays the full finish. */
WEEKLY_LIMIT_NO_PROOF, WEEKLY_LIMIT_NO_PROOF,
/** COMPLETED needs a FINISHED or FINISHED_TOPUP row first. */ /**
* COMPLETED needs a FINISHED or FINISHED_TOPUP row on the game or its set first, or a FINISHED_MANUAL row on the
* set when the game reached 100 % on or after early-since.
*/
NEEDS_PROVEN_FINISH, NEEDS_PROVEN_FINISH,
/** A FINISHED_MANUAL row is paid, but the game's last achievement unlock is before early-since. */
COMPLETION_TOO_OLD,
/** COMPLETED needs all achievements. */ /** COMPLETED needs all achievements. */
NOT_ALL_ACHIEVEMENTS NOT_ALL_ACHIEVEMENTS
} }
@@ -30,6 +30,11 @@ public enum RewardKind {
*/ */
public static final List<RewardKind> FINISH_KINDS = List.of(FINISHED, FINISHED_MANUAL, FINISHED_EARLY); public static final List<RewardKind> FINISH_KINDS = List.of(FINISHED, FINISHED_MANUAL, FINISHED_EARLY);
/**
* The finish kinds that pay the full finish: a proven Steam finish, its top-up, or a manual finish. They open COMPLETED.
*/
public static final List<RewardKind> FULL_FINISH_KINDS = List.of(FINISHED, FINISHED_MANUAL, FINISHED_TOPUP);
/** /**
* The finish kinds that share the weekly cap: a manual finish and an unproven Steam finish. * The finish kinds that share the weekly cap: a manual finish and an unproven Steam finish.
*/ */
Loaded 100 of 161 files, more files were not shown because too many files have changed in this diff. Show more