Compare commits

...
8 Commits
Author SHA1 Message Date
lobo 3512159b41 test: background work inline, shared mock bases, lock-wait instead of sleep
Build, Test & Push Core Service / docker (push) Successful in 1m31s
Build, Test & Push Core Service / quality-check (push) Successful in 1m8s
2026-09-30 12:08:57 +02:00
lobo 7fe2823519 refactor(library): detail page refreshes, then assembles 2026-09-30 12:08:57 +02:00
lobo 2c97e2ae96 refactor(library): two bounded background pools
Every @Async names a pool: with the scheduler there are two TaskExecutor beans, and a bare @Async ran on an unbounded executor. Page views queue only stale games; the long runs after a sync get their own pool.
2026-09-30 12:08:57 +02:00
lobo 9a09d535fc refactor(library): listed_user_game view and one shared schema (V24) 2026-09-30 09:37:16 +02:00
lobo b499e79c41 refactor(library): one IGDB split and merge for time to beat and moods 2026-09-30 09:37:16 +02:00
lobo c1472c7d59 refactor(library): own IgdbGame record, one owned rule, layer rules
Stored import candidates keep their JSON shape, so READY imports from before still read.
2026-09-30 09:37:16 +02:00
lobo 1abc03d2c1 refactor(library): auditor fields only through applyCorrection 2026-09-30 09:37:16 +02:00
lobo 79d7bf17f2 feat(library): PATCH backlog saves only the sent fields 2026-09-30 07:36:08 +02:00
59 changed files with 1089 additions and 427 deletions

No files matched your search

+62 -10
View File
@@ -52,7 +52,8 @@ To Be Played is split into three repositories:
40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres 40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres
and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the
status history); the Pile of Shame trend (90 days of `backlog_snapshot`, plus "this month: added / done", where the status history); the Pile of Shame trend (90 days of `backlog_snapshot`, plus "this month: added / done", where the
first sync of a Steam account is no "added", see `user_steam_sync.history_since`). Every number counts only listed games. first sync of a Steam account is no "added", see `user_steam_sync.history_since`). Every number counts only listed games (still owned, not hidden: the view
`listed_user_game`, V24).
- **Status history**: a database trigger logs every status change of a library game in `user_game_status_change` - **Status history**: a database trigger logs every status change of a library game in `user_game_status_change`
(old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older (old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older
changes are not known. changes are not known.
@@ -97,7 +98,7 @@ only guard.
Every service method a controller calls carries one role marker, on the method or on its class (the method wins): Every service method a controller calls carries one role marker, on the method or on its class (the method wins):
`@RequiresRole(Role.X)`, `@Public` (no login, e.g. landing stats; the public profile checks its ticket) or `@RequiresRole(Role.X)`, `@Public` (no login, e.g. landing stats; the public profile checks its ticket) or
`@Internal` (only Auth's service token). A Spring Security method interceptor checks it while a request is served; `@Internal` (only Auth's service token). A Spring Security method interceptor checks it while a request is served;
scheduled jobs and `@Async` work run without a user and are not checked. A denied call answers `401` without a scheduled jobs and background work run without a user and are not checked. A denied call answers `401` without a
login, else `403`. The URL rules in `SecurityConfig` stay as the outer guard. ArchUnit fails the build for an login, else `403`. The URL rules in `SecurityConfig` stay as the outer guard. ArchUnit fails the build for an
unmarked method, two markers, a controller calling an `@Async` method (the check would never run), and a service unmarked method, two markers, a controller calling an `@Async` method (the check would never run), and a service
method calling another service's method that needs more. method calling another service's method that needs more.
@@ -107,7 +108,8 @@ method calling another service's method that needs more.
| Method | Path | Auth | | Method | Path | Auth |
|---|---|---| |---|---|---|
| `GET` | `/api/library/games`, `/api/library/games/{appid}` | JWT | | `GET` | `/api/library/games`, `/api/library/games/{appid}` | JWT |
| `PUT` | `/api/library/games/{appid}/backlog` | JWT | | `PATCH` | `/api/library/games/{appid}/backlog` (only the sent fields; `null` clears one) | JWT |
| `PUT` | `/api/library/games/{appid}/backlog` (replaces all four; removed in item 29) | JWT |
| `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR | | `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR |
| `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR | | `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
| `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT | | `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT |
@@ -171,9 +173,9 @@ flowchart LR
Igdb -->|"OAuth token"| Twitch["Twitch"] Igdb -->|"OAuth token"| Twitch["Twitch"]
Speedrun --> SpeedrunApi["speedrun.com API"] Speedrun --> SpeedrunApi["speedrun.com API"]
LibRepo --> Db[("PostgreSQL tbp_core")] LibRepo --> Db[("PostgreSQL tbp_core")]
Stats --> Db Stats -->|"listed_user_game, deleted_user"| Db
Auth -->|"service token<br/>/internal/accounts"| Account Auth -->|"service token<br/>/internal/accounts"| Account
Account --> Db Account -->|"per-user rows, deleted_user"| Db
``` ```
| Module | Content | | Module | Content |
@@ -186,6 +188,13 @@ flowchart LR
| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search | | `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search |
| `speedrun_adapter` | speedrun.com client: game search by name, verified runs | | `speedrun_adapter` | speedrun.com client: game search by name, verified runs |
**One shared schema.** All modules use one database `tbp_core`, and some read or write another module's tables in
SQL, which ArchUnit cannot see: `stats` reads `listed_user_game`, `steam_game` and `deleted_user`; `account` deletes
the per-user rows of the library tables (`AccountCleanupRepository`; a new library table with `user_id` goes into
its list); `library` reads `deleted_user`. A module that moves to its own service takes these reads along as calls.
The listed rule ("still owned, not hidden") is `UserGameRepository.LISTED` for JPQL and the view
`listed_user_game` for native SQL; V24 says what a migration of `user_game` must do with the view.
Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` (JPA entities) and `repo` Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` (JPA entities) and `repo`
(repositories). `src/test/java/core_service/architecture/ArchitectureTest.java` checks these rules on every build: (repositories). `src/test/java/core_service/architecture/ArchitectureTest.java` checks these rules on every build:
@@ -193,6 +202,10 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
`speedrun_adapter`. `speedrun_adapter`.
- No cycles between `modules`, `shared` and `configuration`. - No cycles between `modules`, `shared` and `configuration`.
- `api` never uses `repo`, and `repo` never uses `api`. - `api` never uses `repo`, and `repo` never uses `api`.
- `domain` never uses `api`, `service`, `repo` or an adapter: the import stores IGDB hits as the own record
`IgdbGame`, so a change of the adapter's `IgdbApiService.Game` cannot break stored imports.
- `service` never uses `api`, except the adapters' response DTOs and a fixed list of 8 services that still build
their module's response DTOs. The list only gets shorter.
- Every class in `repo` ends with `Repository`. - Every class in `repo` ends with `Repository`.
- `shared` never uses `modules`, and `modules` never use `configuration`. - `shared` never uses `modules`, and `modules` never use `configuration`.
- A `*Service` class in a module's `service` package has `@Service`. A `*Controller` class in `api` has - A `*Service` class in a module's `service` package has `@Service`. A `*Controller` class in `api` has
@@ -208,7 +221,7 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and
dates finished games by the unlock of their ending achievement dates finished games by the unlock of their ending achievement
(`FinishedEstimateService`, `user_game.finished_at_estimate`). (`FinishedEstimateService`, `user_game.finished_at_estimate`).
3. `LibrarySyncWorker` runs in the background. `user_steam_sync` lets only one run per user start, and none for 3. `LibrarySyncWorker` runs on the background pool. `user_steam_sync` lets only one run per user start, and none for
a user in `deleted_user`. a user in `deleted_user`.
4. The worker loads the wishlist and the owned games from Steam. It updates `steam_game` and `user_game`, adds 4. The worker loads the wishlist and the owned games from Steam. It updates `steam_game` and `user_game`, adds
the playtime growth of the day to `user_game_playtime_day`, and soft-deletes Steam games the user no longer the playtime growth of the day to `user_game_playtime_day`, and soft-deletes Steam games the user no longer
@@ -226,16 +239,50 @@ The frontend polls `GET /api/library/sync-status` until the run ends.
At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/library/broken-images` (table At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/library/broken-images` (table
`broken_image`, one row per game and kind). Reports only count for Steam games the user owns. `broken_image`, one row per game and kind). Reports only count for Steam games the user owns.
### Background work
Background work runs on two pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it):
- The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view
starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with
a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user
starts it again.
- The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun
refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page
views' threads. A task is small (a user or a few appids), and none is dropped.
Imports have their own two threads, and `game_import` is their queue. Scheduled jobs (nightly sync, image repair,
import watchdog) run on the scheduler.
```mermaid
flowchart LR
View["page view, add, merge"] -->|"stale games only"| Pool["default pool<br/>4 threads, queue 100"]
Sync["sync request"] --> Pool
Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
View -->|"speedrun"| FollowUp
Pool --> Apis["Steam, IGDB, speedrun.com"]
FollowUp --> Apis
Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"]
```
Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks
live in memory: a second instance would ask Steam and IGDB twice, and the database would stay consistent (sync
claim, upserts). A restart drops what waits in the pools; it is all refresh work and is asked again. Imports survive
a restart.
### Corrections by an auditor ### Corrections by an auditor
An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag and genres for An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag and genres for
every user (`game_correction`, V20). The value is also written into the cached row (`steam_game`, `game_speedrun`, every user (`game_correction`, V20). The value is also written into the cached row (`steam_game`, `game_speedrun`,
`game_mood`), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name), `game_mood`), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name),
the image repair (image), the IGDB add (name, image), the mood and speedrun upserts (SQL). **A new writer of these the image repair (image), the IGDB add (name, image), the mood and speedrun upserts (SQL). `SteamGame` has no
fields must do the same.** "Back to source" (`DELETE`) writes back the source value from the first correction and setter for name and image: `applyCorrection` is the only Java writer, a refresh uses the guarded updates in
`SteamGameRepository`. "Back to source" (`DELETE`) writes back the source value from the first correction and
lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP. lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP.
Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the
source value once (its guard sees the state from before the save); the auditor saves again. source value once (its guard sees the state from before the save); the auditor saves again. `save(new SteamGame(...))` on a
stored key also overwrites a corrected name; the sync and the IGDB add do it only after `findById` found no row.
### Merging duplicates ### Merging duplicates
@@ -390,11 +437,16 @@ Configuration comes from environment variables:
``` ```
Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers. Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers.
In integration tests background work runs on the calling thread, so a test reads the result at once.
`AbstractMockedAdaptersTest` mocks Steam, IGDB and speedrun.com, `AbstractMockedLibraryServicesTest` the sync and
the refreshes of the picker and stats pages; each is one shared Spring context. Race tests wait until the other
connection waits for the row lock (`awaitLockWaiter`), not for a fixed time.
## Roadmap ## Roadmap
- Sale alerts for wishlist games. - Sale alerts for wishlist games.
- Move `steam_adapter` DTOs out of `api`, then turn on strict module independence in ArchUnit. - Move the adapter DTOs out of `api`, then turn on strict module independence in ArchUnit and drop the adapter
exception of the "`service` never uses `api`" rule.
## License ## License
@@ -1,9 +1,46 @@
package core_service.configuration; package core_service.configuration;
import core_service.shared.background.BackgroundPools;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync; import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
/**
* The two pools of {@link BackgroundPools}. A dropped task is a refresh of data with a TTL: the next view or sync
* asks again.
*/
@Configuration @Configuration
@EnableAsync @EnableAsync
@Slf4j
public class AsyncConfig { public class AsyncConfig {
// A task holds a connection only for its short transactions. At most 6 background tasks, plus the scheduler and
// the 2 import lanes, use Hikari's 10 connections at once; requests share the rest.
static final int THREADS = 4;
static final int QUEUE = 100;
// The achievement runs share one Steam pace and speedrun has one drainer: more threads would only wait.
static final int FOLLOW_UP_THREADS = 2;
@Bean(name = BackgroundPools.DEFAULT)
public ThreadPoolTaskExecutor applicationTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(THREADS);
executor.setMaxPoolSize(THREADS);
executor.setQueueCapacity(QUEUE);
executor.setThreadNamePrefix("background-");
executor.setRejectedExecutionHandler((task, pool) -> log.warn(pool.isShutdown()
? "Background pool stopped, task dropped" : "Background queue full, task dropped"));
return executor;
}
@Bean(name = BackgroundPools.SYNC_FOLLOW_UP)
public ThreadPoolTaskExecutor syncFollowUpExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(FOLLOW_UP_THREADS);
executor.setMaxPoolSize(FOLLOW_UP_THREADS);
executor.setThreadNamePrefix("sync-follow-up-");
return executor;
}
} }
@@ -53,7 +53,7 @@ public class SecurityConfig {
CorsConfiguration configuration = new CorsConfiguration(); CorsConfiguration configuration = new CorsConfiguration();
// Exact origins only: with credentials, a pattern would send the login cookie to every matching site. // Exact origins only: with credentials, a pattern would send the login cookie to every matching site.
configuration.setAllowedOrigins(Arrays.asList(allowedOrigins)); configuration.setAllowedOrigins(Arrays.asList(allowedOrigins));
configuration.setAllowedMethods(List.of("GET", "POST", "PUT", "DELETE", "OPTIONS")); configuration.setAllowedMethods(List.of("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"));
configuration.setAllowedHeaders(List.of( configuration.setAllowedHeaders(List.of(
"Content-Type", "Authorization", ClientHeaderFilter.CLIENT_HEADER, "X-Profile-Ticket")); "Content-Type", "Authorization", ClientHeaderFilter.CLIENT_HEADER, "X-Profile-Ticket"));
configuration.setAllowCredentials(true); configuration.setAllowCredentials(true);
@@ -4,6 +4,7 @@ import core_service.shared.api.ErrorResponse;
import core_service.modules.library.api.dto.AchievementSummaryDto; import core_service.modules.library.api.dto.AchievementSummaryDto;
import core_service.modules.library.api.dto.ActivityDto; import core_service.modules.library.api.dto.ActivityDto;
import core_service.modules.library.api.dto.BacklogEntryDto; import core_service.modules.library.api.dto.BacklogEntryDto;
import core_service.modules.library.api.dto.BacklogPatch;
import core_service.modules.library.api.dto.BrokenImageReport; import core_service.modules.library.api.dto.BrokenImageReport;
import core_service.modules.library.api.dto.CorrectionDto; import core_service.modules.library.api.dto.CorrectionDto;
import core_service.modules.library.api.dto.FinishedHintDismissal; import core_service.modules.library.api.dto.FinishedHintDismissal;
@@ -47,6 +48,7 @@ import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping; import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.PatchMapping;
import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RequestParam;
@@ -231,6 +233,15 @@ public class LibraryController {
.body(GAME_NOT_FOUND)); .body(GAME_NOT_FOUND));
} }
@PatchMapping("/games/{appid}/backlog")
public ResponseEntity<?> patchBacklog(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") Long appid,
@Valid @RequestBody BacklogPatch patch) {
return backlogService.patch(principal.getIdAsLong(), appid, patch)
.<ResponseEntity<?>>map(ResponseEntity::ok)
.orElseGet(LibraryController::unknownGame);
}
@PutMapping("/games/{appid}/hidden") @PutMapping("/games/{appid}/hidden")
public ResponseEntity<?> updateHidden(@AuthenticationPrincipal UserPrincipal principal, public ResponseEntity<?> updateHidden(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") Long appid, @PathVariable("appid") Long appid,
@@ -9,6 +9,7 @@ import jakarta.validation.constraints.Size;
/** /**
* Request and response of the backlog update. The request replaces all four fields; null clears an optional one. * Request and response of the backlog update. The request replaces all four fields; null clears an optional one.
* The PUT goes away in item 29; the Frontend sends a {@link BacklogPatch}.
*/ */
public record BacklogEntryDto( public record BacklogEntryDto(
@NotNull GameStatus status, @NotNull GameStatus status,
@@ -0,0 +1,26 @@
package core_service.modules.library.api.dto;
import core_service.modules.library.domain.GamePriority;
import core_service.modules.library.domain.GameStatus;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import lombok.Getter;
import lombok.Setter;
import java.util.Optional;
/**
* The backlog fields the request sends; a field it leaves out stays as it is. A field is null when it is not in the
* JSON and an empty Optional when it is null there, which clears it.
* A class, not a record: for a record Jackson gives an empty Optional for a missing field too.
*/
@Getter
@Setter
public class BacklogPatch {
private Optional<@NotNull GameStatus> status;
private Optional<@Min(1) @Max(5) Integer> rating;
private Optional<@Size(max = 2000) String> notes;
private Optional<GamePriority> priority;
}
@@ -1,7 +1,5 @@
package core_service.modules.library.domain; package core_service.modules.library.domain;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import java.time.Instant; import java.time.Instant;
import java.util.List; import java.util.List;
@@ -46,7 +44,7 @@ public record GameImport(long id, long userId, Source source, Status status, int
* @param exact the first candidate has the line's name (case and punctuation ignored) * @param exact the first candidate has the line's name (case and punctuation ignored)
* @param candidates best first, at most five * @param candidates best first, at most five
*/ */
public record Row(Line line, boolean searchFailed, boolean exact, List<IgdbApiService.Game> candidates) {} public record Row(Line line, boolean searchFailed, boolean exact, List<IgdbGame> candidates) {}
/** /**
* The user's pick for a line in the review. * The user's pick for a line in the review.
@@ -0,0 +1,9 @@
package core_service.modules.library.domain;
import java.util.List;
/**
* An IGDB game as the library needs it, and the stored shape of an import candidate (V22 {@code candidates}): the
* component names are the JSON keys of rows already stored.
*/
public record IgdbGame(long igdbId, String name, Integer year, String imageUrl, List<String> platforms, Long steamAppid) {}
@@ -17,7 +17,6 @@ import java.time.Instant;
@DynamicUpdate @DynamicUpdate
@Table(name = "steam_game") @Table(name = "steam_game")
@Getter @Getter
@Setter
@NoArgsConstructor(access = AccessLevel.PROTECTED) @NoArgsConstructor(access = AccessLevel.PROTECTED)
public class SteamGame { public class SteamGame {
@@ -28,9 +27,11 @@ public class SteamGame {
@Column(name = "name", nullable = false, length = 512) @Column(name = "name", nullable = false, length = 512)
private String name; private String name;
@Setter
@Column(name = "img_icon_url", length = 512) @Column(name = "img_icon_url", length = 512)
private String imgIconUrl; private String imgIconUrl;
@Setter
@Column(name = "last_seen_at", nullable = false) @Column(name = "last_seen_at", nullable = false)
private Instant lastSeenAt; private Instant lastSeenAt;
@@ -45,9 +46,26 @@ public class SteamGame {
} }
public SteamGame(Long appid, String name, String imgIconUrl, Instant lastSeenAt) { public SteamGame(Long appid, String name, String imgIconUrl, Instant lastSeenAt) {
this(appid, name, imgIconUrl, null, lastSeenAt);
}
public SteamGame(Long appid, String name, String imgIconUrl, String imageUrl, Instant lastSeenAt) {
this.appid = appid; this.appid = appid;
this.name = name; this.name = name;
this.imgIconUrl = imgIconUrl; this.imgIconUrl = imgIconUrl;
this.imageUrl = imageUrl;
this.lastSeenAt = lastSeenAt; this.lastSeenAt = lastSeenAt;
} }
/**
* The only Java writer of name and image on a stored game. Refreshes from a source use the guarded updates in
* {@code SteamGameRepository}, so they never overwrite a correction.
*/
public void applyCorrection(CorrectionField field, String value) {
switch (field) {
case NAME -> name = value;
case IMAGE -> imageUrl = value;
default -> throw new IllegalArgumentException(field + " is not a column of steam_game");
}
}
} }
@@ -22,8 +22,7 @@ public class BacklogSnapshotRepository {
count(*) filter (where ug.playtime_minutes = 0) as never_played, count(*) filter (where ug.playtime_minutes = 0) as never_played,
coalesce(sum(t.normally_minutes) filter (where ug.status = 'BACKLOG'), 0) as backlog_minutes, coalesce(sum(t.normally_minutes) filter (where ug.status = 'BACKLOG'), 0) as backlog_minutes,
count(*) as listed_games count(*) as listed_games
from user_game ug left join game_time_to_beat t on t.appid = ug.appid from listed_user_game ug left join game_time_to_beat t on t.appid = ug.appid
where ug.removed_at is null and ug.hidden_at is null
"""; """;
private final NamedParameterJdbcTemplate jdbc; private final NamedParameterJdbcTemplate jdbc;
@@ -37,7 +36,7 @@ public class BacklogSnapshotRepository {
return jdbc.update(""" return jdbc.update("""
insert into backlog_snapshot (user_id, day, backlog_games, never_played, backlog_minutes, listed_games) insert into backlog_snapshot (user_id, day, backlog_games, never_played, backlog_minutes, listed_games)
select ug.user_id, :day, %s select ug.user_id, :day, %s
and ug.user_id not in (select user_id from deleted_user) where ug.user_id not in (select user_id from deleted_user)
group by ug.user_id group by ug.user_id
on conflict (user_id, day) do update set backlog_games = excluded.backlog_games, on conflict (user_id, day) do update set backlog_games = excluded.backlog_games,
never_played = excluded.never_played, backlog_minutes = excluded.backlog_minutes, never_played = excluded.never_played, backlog_minutes = excluded.backlog_minutes,
@@ -61,7 +60,7 @@ public class BacklogSnapshotRepository {
* The pile of now, for the day that has no row yet. * The pile of now, for the day that has no row yet.
*/ */
public UserStats.PileDay live(long userId, LocalDate day) { public UserStats.PileDay live(long userId, LocalDate day) {
return jdbc.queryForObject("select " + PILE_COLUMNS + "and ug.user_id = :userId", return jdbc.queryForObject("select " + PILE_COLUMNS + "where ug.user_id = :userId",
new MapSqlParameterSource("userId", userId), new MapSqlParameterSource("userId", userId),
(rs, i) -> new UserStats.PileDay(day, rs.getLong("backlog_games"), rs.getLong("never_played"), (rs, i) -> new UserStats.PileDay(day, rs.getLong("backlog_games"), rs.getLong("never_played"),
rs.getLong("backlog_minutes"))); rs.getLong("backlog_minutes")));
@@ -1,6 +1,6 @@
package core_service.modules.library.repo; package core_service.modules.library.repo;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameImport; import core_service.modules.library.domain.GameImport;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -189,11 +189,11 @@ public class GameImportRepository {
new MapSqlParameterSource("id", importId), (rs, row) -> new GameImport.Row(line(rs), new MapSqlParameterSource("id", importId), (rs, row) -> new GameImport.Row(line(rs),
rs.getBoolean("search_failed"), rs.getBoolean("exact"), rs.getBoolean("search_failed"), rs.getBoolean("exact"),
jsonMapper.readValue(rs.getString("candidates"), jsonMapper.readValue(rs.getString("candidates"),
jsonMapper.getTypeFactory().constructCollectionType(List.class, IgdbApiService.Game.class)))); jsonMapper.getTypeFactory().constructCollectionType(List.class, IgdbGame.class))));
} }
public void saveMatch(long importId, int lineNo, boolean searchFailed, boolean exact, public void saveMatch(long importId, int lineNo, boolean searchFailed, boolean exact,
List<IgdbApiService.Game> candidates) { List<IgdbGame> candidates) {
jdbc.update("update game_import_row set matched_at = now(), search_failed = :failed, exact = :exact, " jdbc.update("update game_import_row set matched_at = now(), search_failed = :failed, exact = :exact, "
+ "candidates = cast(:candidates as jsonb) where import_id = :id and line_no = :line", + "candidates = cast(:candidates as jsonb) where import_id = :id and line_no = :line",
new MapSqlParameterSource("id", importId).addValue("line", lineNo).addValue("failed", searchFailed) new MapSqlParameterSource("id", importId).addValue("line", lineNo).addValue("failed", searchFailed)
@@ -39,7 +39,8 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*/ */
String LISTED = "ug.userId = :userId and ug.removedAt is null and ug.hiddenAt is null "; String LISTED = "ug.userId = :userId and ug.removedAt is null and ug.hiddenAt is null ";
List<UserGame> findByUserIdAndRemovedAtIsNull(Long userId); @Query("select ug from UserGame ug where " + LISTED + "and ug.source = core_service.modules.library.domain.GameSource.STEAM")
List<UserGame> findListedSteamGames(@Param("userId") Long userId);
List<UserGame> findByUserId(Long userId); List<UserGame> findByUserId(Long userId);
@@ -103,8 +104,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
* *
* @param excludeAppid never picked; {@link #NO_GAME} excludes nothing * @param excludeAppid never picked; {@link #NO_GAME} excludes nothing
*/ */
@Query(value = "select * from user_game where user_id = :userId and removed_at is null and hidden_at is null " + @Query(value = "select * from listed_user_game where user_id = :userId and status = 'BACKLOG' " +
"and status = 'BACKLOG' " +
"and appid <> :excludeAppid " + "and appid <> :excludeAppid " +
// Weighted random pick: the smallest -ln(u) / weight wins. // Weighted random pick: the smallest -ln(u) / weight wins.
"order by -ln(1 - random()) / case priority when 'HIGH' then 4 when 'MEDIUM' then 2 else 1 end " + "order by -ln(1 - random()) / case priority when 'HIGH' then 4 when 'MEDIUM' then 2 else 1 end " +
@@ -136,8 +136,8 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
return queryPickerGames(userId, maxMinutes, storedMood, speedrun, excluded, count); return queryPickerGames(userId, maxMinutes, storedMood, speedrun, excluded, count);
} }
@Query(value = "select ug.* from user_game ug where ug.user_id = :userId and ug.removed_at is null " + @Query(value = "select ug.* from listed_user_game ug where ug.user_id = :userId " +
"and ug.hidden_at is null and ug.status = 'BACKLOG' and ug.appid not in (:excludeAppids) " + "and ug.status = 'BACKLOG' and ug.appid not in (:excludeAppids) " +
"and (cast(:maxMinutes as integer) is null or exists (select 1 from game_time_to_beat t " + "and (cast(:maxMinutes as integer) is null or exists (select 1 from game_time_to_beat t " +
" where t.appid = ug.appid and t.normally_minutes <= :maxMinutes)) " + " where t.appid = ug.appid and t.normally_minutes <= :maxMinutes)) " +
"and (cast(:mood as varchar) is null or exists (select 1 from game_mood m " + "and (cast(:mood as varchar) is null or exists (select 1 from game_mood m " +
@@ -220,7 +220,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
@Modifying(clearAutomatically = true) @Modifying(clearAutomatically = true)
@Transactional @Transactional
@Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.FINISHED " + @Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.FINISHED " +
"where ug.userId = :userId and ug.appid = :appid and ug.removedAt is null and ug.hiddenAt is null " + "where " + LISTED + "and ug.appid = :appid " +
"and ug.statusSetByUser = false and ug.finishedHintDismissed is null " + "and ug.statusSetByUser = false and ug.finishedHintDismissed is null " +
"and ug.source = core_service.modules.library.domain.GameSource.STEAM " + "and ug.source = core_service.modules.library.domain.GameSource.STEAM " +
"and ug.status in (core_service.modules.library.domain.GameStatus.BACKLOG, " + "and ug.status in (core_service.modules.library.domain.GameStatus.BACKLOG, " +
@@ -253,7 +253,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*/ */
@Modifying @Modifying
@Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.PLAYING " + @Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.PLAYING " +
"where ug.userId = :userId and ug.appid in :appids and ug.removedAt is null and ug.hiddenAt is null " + "where " + LISTED + "and ug.appid in :appids " +
"and ug.statusSetByUser = false " + "and ug.statusSetByUser = false " +
"and ug.status = core_service.modules.library.domain.GameStatus.BACKLOG") "and ug.status = core_service.modules.library.domain.GameStatus.BACKLOG")
int playFromPileIfUntouched(@Param("userId") Long userId, @Param("appids") Collection<Long> appids); int playFromPileIfUntouched(@Param("userId") Long userId, @Param("appids") Collection<Long> appids);
@@ -23,7 +23,7 @@ import java.util.Optional;
@RequiredArgsConstructor @RequiredArgsConstructor
public class UserStatsRepository { public class UserStatsRepository {
private static final String LISTED = "ug.user_id = :userId and ug.removed_at is null and ug.hidden_at is null "; private static final String OF_USER = "ug.user_id = :userId ";
private static final String FINISHED = "('FINISHED', 'COMPLETED')"; private static final String FINISHED = "('FINISHED', 'COMPLETED')";
/** A status change that finished the game: from another status; not a game that came in finished. */ /** A status change that finished the game: from another status; not a game that came in finished. */
private static final String FINISH_CHANGE = private static final String FINISH_CHANGE =
@@ -46,9 +46,9 @@ public class UserStatsRepository {
return jdbc.queryForObject(""" return jdbc.queryForObject("""
select coalesce(sum(t.normally_minutes), 0) as normal, coalesce(sum(t.completely_minutes), 0) as complete, select coalesce(sum(t.normally_minutes), 0) as normal, coalesce(sum(t.completely_minutes), 0) as complete,
count(t.normally_minutes) as known count(t.normally_minutes) as known
from user_game ug join game_time_to_beat t on t.appid = ug.appid from listed_user_game ug join game_time_to_beat t on t.appid = ug.appid
where %s and ug.status = 'BACKLOG' where %s and ug.status = 'BACKLOG'
""".formatted(LISTED), user(userId), """.formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.PileTimes(rs.getLong("normal"), rs.getLong("complete"), rs.getLong("known"))); (rs, i) -> new UserStats.PileTimes(rs.getLong("normal"), rs.getLong("complete"), rs.getLong("known")));
} }
@@ -62,10 +62,10 @@ public class UserStatsRepository {
public Optional<UserStats.OldestGame> oldestUnplayed(long userId) { public Optional<UserStats.OldestGame> oldestUnplayed(long userId) {
return jdbc.query(""" return jdbc.query("""
select ug.appid, sg.name, sg.image_url, ug.first_seen_at select ug.appid, sg.name, sg.image_url, ug.first_seen_at
from user_game ug join steam_game sg on sg.appid = ug.appid from listed_user_game ug join steam_game sg on sg.appid = ug.appid
where %s and ug.status = 'BACKLOG' and ug.playtime_minutes = 0 where %s and ug.status = 'BACKLOG' and ug.playtime_minutes = 0
order by ug.first_seen_at, ug.appid limit 1 order by ug.first_seen_at, ug.appid limit 1
""".formatted(LISTED), user(userId), """.formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.OldestGame(rs.getLong("appid"), rs.getString("name"), (rs, i) -> new UserStats.OldestGame(rs.getLong("appid"), rs.getString("name"),
rs.getString("image_url"), rs.getTimestamp("first_seen_at").toInstant())) rs.getString("image_url"), rs.getTimestamp("first_seen_at").toInstant()))
.stream().findFirst(); .stream().findFirst();
@@ -74,10 +74,10 @@ public class UserStatsRepository {
public List<UserStats.GameMinutes> mostPlayed(long userId, int limit) { public List<UserStats.GameMinutes> mostPlayed(long userId, int limit) {
return jdbc.query(""" return jdbc.query("""
select ug.appid, sg.name, sg.image_url, ug.playtime_minutes select ug.appid, sg.name, sg.image_url, ug.playtime_minutes
from user_game ug join steam_game sg on sg.appid = ug.appid from listed_user_game ug join steam_game sg on sg.appid = ug.appid
where %s and ug.playtime_minutes > 0 where %s and ug.playtime_minutes > 0
order by ug.playtime_minutes desc, ug.appid limit :limit order by ug.playtime_minutes desc, ug.appid limit :limit
""".formatted(LISTED), user(userId).addValue("limit", limit), """.formatted(OF_USER), user(userId).addValue("limit", limit),
(rs, i) -> new UserStats.GameMinutes(rs.getLong("appid"), rs.getString("name"), (rs, i) -> new UserStats.GameMinutes(rs.getLong("appid"), rs.getString("name"),
rs.getString("image_url"), rs.getLong("playtime_minutes"))); rs.getString("image_url"), rs.getLong("playtime_minutes")));
} }
@@ -90,7 +90,7 @@ public class UserStatsRepository {
select l.appid, sg.name, sg.image_url, l.months, l.minutes select l.appid, sg.name, sg.image_url, l.months, l.minutes
from (select p.appid, count(distinct date_trunc('month', p.day)) as months, sum(p.minutes) as minutes from (select p.appid, count(distinct date_trunc('month', p.day)) as months, sum(p.minutes) as minutes
%s group by p.appid) l %s group by p.appid) l
join user_game ug on ug.user_id = :userId and ug.appid = l.appid and ug.removed_at is null join listed_user_game ug on ug.user_id = :userId and ug.appid = l.appid
join steam_game sg on sg.appid = l.appid join steam_game sg on sg.appid = l.appid
order by l.months desc, l.minutes desc, l.appid limit 1 order by l.months desc, l.minutes desc, l.appid limit 1
""".formatted(PLAYTIME_DAYS_NOT_HIDDEN), user(userId), """.formatted(PLAYTIME_DAYS_NOT_HIDDEN), user(userId),
@@ -110,10 +110,10 @@ public class UserStatsRepository {
private List<UserStats.TagCount> tagCounts(long userId, String column, int limit) { private List<UserStats.TagCount> tagCounts(long userId, String column, int limit) {
return jdbc.query(""" return jdbc.query("""
select tag.name, count(*) as games, coalesce(sum(ug.playtime_minutes), 0) as minutes select tag.name, count(*) as games, coalesce(sum(ug.playtime_minutes), 0) as minutes
from user_game ug join game_mood m on m.appid = ug.appid cross join unnest(m.%s) as tag(name) from listed_user_game ug join game_mood m on m.appid = ug.appid cross join unnest(m.%s) as tag(name)
where %s where %s
group by tag.name order by minutes desc, games desc, tag.name limit :limit group by tag.name order by minutes desc, games desc, tag.name limit :limit
""".formatted(column, LISTED), user(userId).addValue("limit", limit), """.formatted(column, OF_USER), user(userId).addValue("limit", limit),
(rs, i) -> new UserStats.TagCount(rs.getString("name"), rs.getLong("games"), rs.getLong("minutes"))); (rs, i) -> new UserStats.TagCount(rs.getString("name"), rs.getLong("games"), rs.getLong("minutes")));
} }
@@ -128,8 +128,8 @@ public class UserStatsRepository {
return jdbc.queryForObject(""" return jdbc.queryForObject("""
select count(*) filter (where ug.playtime_minutes > 0) as started, select count(*) filter (where ug.playtime_minutes > 0) as started,
count(*) filter (where ug.playtime_minutes between 1 and 59) as under_one_hour count(*) filter (where ug.playtime_minutes between 1 and 59) as under_one_hour
from user_game ug where %s from listed_user_game ug where %s
""".formatted(LISTED), user(userId), """.formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.PlayedCounts(rs.getLong("started"), rs.getLong("under_one_hour"))); (rs, i) -> new UserStats.PlayedCounts(rs.getLong("started"), rs.getLong("under_one_hour")));
} }
@@ -141,20 +141,20 @@ public class UserStatsRepository {
return jdbc.query(""" return jdbc.query("""
select cast(extract(year from coalesce(ug.finished_at_estimate, f.finished_at) at time zone :zone) select cast(extract(year from coalesce(ug.finished_at_estimate, f.finished_at) at time zone :zone)
as integer) as year, count(*) as games as integer) as year, count(*) as games
from user_game ug left join (%s) f on f.appid = ug.appid from listed_user_game ug left join (%s) f on f.appid = ug.appid
where %s and ug.status in %s and coalesce(ug.finished_at_estimate, f.finished_at) is not null where %s and ug.status in %s and coalesce(ug.finished_at_estimate, f.finished_at) is not null
group by year order by year group by year order by year
""".formatted(FIRST_FINISH, LISTED, FINISHED), user(userId).addValue("zone", zone.getId()), """.formatted(FIRST_FINISH, OF_USER, FINISHED), user(userId).addValue("zone", zone.getId()),
(rs, i) -> new UserStats.YearCount(rs.getInt("year"), rs.getLong("games"))); (rs, i) -> new UserStats.YearCount(rs.getInt("year"), rs.getLong("games")));
} }
public long finishedBeforeTracking(long userId) { public long finishedBeforeTracking(long userId) {
Long games = jdbc.queryForObject(""" Long games = jdbc.queryForObject("""
select count(*) from user_game ug select count(*) from listed_user_game ug
where %s and ug.status in %s and ug.finished_at_estimate is null where %s and ug.status in %s and ug.finished_at_estimate is null
and not exists (select 1 from user_game_status_change sc and not exists (select 1 from user_game_status_change sc
where sc.user_id = ug.user_id and sc.appid = ug.appid and %s) where sc.user_id = ug.user_id and sc.appid = ug.appid and %s)
""".formatted(LISTED, FINISHED, FINISH_CHANGE), user(userId), Long.class); """.formatted(OF_USER, FINISHED, FINISH_CHANGE), user(userId), Long.class);
return games != null ? games : 0L; return games != null ? games : 0L;
} }
@@ -170,10 +170,10 @@ public class UserStatsRepository {
or sc.from_status is not null or sc.changed_at > uss.history_since)) as added, or sc.from_status is not null or sc.changed_at > uss.history_since)) as added,
count(distinct sc.appid) filter (where sc.from_status = 'BACKLOG' and sc.to_status <> 'BACKLOG') count(distinct sc.appid) filter (where sc.from_status = 'BACKLOG' and sc.to_status <> 'BACKLOG')
as done as done
from user_game_status_change sc join user_game ug on ug.user_id = sc.user_id and ug.appid = sc.appid from user_game_status_change sc join listed_user_game ug on ug.user_id = sc.user_id and ug.appid = sc.appid
left join user_steam_sync uss on uss.user_id = sc.user_id left join user_steam_sync uss on uss.user_id = sc.user_id
where %s and sc.changed_at >= :monthStart where %s and sc.changed_at >= :monthStart
""".formatted(LISTED), user(userId).addValue("monthStart", Timestamp.from(monthStart)), """.formatted(OF_USER), user(userId).addValue("monthStart", Timestamp.from(monthStart)),
(rs, i) -> new UserStats.MonthMoves(rs.getLong("added"), rs.getLong("done"))); (rs, i) -> new UserStats.MonthMoves(rs.getLong("added"), rs.getLong("done")));
} }
@@ -3,7 +3,6 @@ package core_service.modules.library.service;
import core_service.modules.library.domain.GameAchievementSchema; import core_service.modules.library.domain.GameAchievementSchema;
import core_service.modules.library.domain.EndingAchievement; import core_service.modules.library.domain.EndingAchievement;
import core_service.modules.library.domain.GameAchievementSchemaMeta; import core_service.modules.library.domain.GameAchievementSchemaMeta;
import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameAchievement; import core_service.modules.library.domain.UserGameAchievement;
@@ -16,6 +15,7 @@ import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse; import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse; import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.background.BackgroundPools;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async; import org.springframework.scheduling.annotation.Async;
@@ -94,7 +94,7 @@ public class AchievementSyncService {
} }
} }
@Async @Async(BackgroundPools.SYNC_FOLLOW_UP)
@TransactionalEventListener @TransactionalEventListener
public void onLibrarySynced(LibrarySyncedEvent event) { public void onLibrarySynced(LibrarySyncedEvent event) {
syncLibrary(event.userId(), event.steamId()); syncLibrary(event.userId(), event.steamId());
@@ -163,8 +163,7 @@ public class AchievementSyncService {
} }
private void runLibrarySync(Long userId, String steamId) { private void runLibrarySync(Long userId, String steamId) {
List<UserGame> games = userGameRepository.findByUserIdAndRemovedAtIsNull(userId).stream() List<UserGame> games = userGameRepository.findListedSteamGames(userId).stream()
.filter(game -> game.getSource() == GameSource.STEAM && game.getHiddenAt() == null)
.sorted(Comparator.comparingInt(UserGame::getPlaytimeMinutes).reversed()) .sorted(Comparator.comparingInt(UserGame::getPlaytimeMinutes).reversed())
.toList(); .toList();
Map<Long, GameAchievementSchemaMeta> schemas = schemaMetaRepository Map<Long, GameAchievementSchemaMeta> schemas = schemaMetaRepository
@@ -1,6 +1,7 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.library.api.dto.BacklogEntryDto; import core_service.modules.library.api.dto.BacklogEntryDto;
import core_service.modules.library.api.dto.BacklogPatch;
import core_service.modules.library.domain.FinishedHintType; import core_service.modules.library.domain.FinishedHintType;
import core_service.modules.library.domain.GameSource; import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
@@ -40,6 +41,28 @@ public class BacklogService {
}); });
} }
/**
* Changes only the fields the patch sends. Only a sent status counts as set by the user, so a save of the notes
* alone still lets the sync move the game from BACKLOG to PLAYING.
*
* @return empty when the game is not in the user's active library
*/
@Transactional
public Optional<BacklogEntryDto> patch(Long userId, Long appid, BacklogPatch patch) {
return userGameRepository.findById(new UserGameId(userId, appid))
.filter(game -> game.getRemovedAt() == null)
.map(game -> {
if (patch.getStatus() != null) {
game.setStatus(patch.getStatus().orElseThrow());
game.setStatusSetByUser(true);
}
if (patch.getRating() != null) game.setRating(patch.getRating().orElse(null));
if (patch.getNotes() != null) game.setNotes(patch.getNotes().filter(notes -> !notes.isBlank()).orElse(null));
if (patch.getPriority() != null) game.setPriority(patch.getPriority().orElse(null));
return toDto(game);
});
}
/** /**
* Changes only the status and keeps rating, notes and priority. * Changes only the status and keeps rating, notes and priority.
* *
@@ -115,8 +115,7 @@ public class GameCorrectionService {
private void write(CorrectionField field, SteamGame game, Object value) { private void write(CorrectionField field, SteamGame game, Object value) {
switch (field) { switch (field) {
case NAME -> game.setName((String) value); case NAME, IMAGE -> game.applyCorrection(field, (String) value);
case IMAGE -> game.setImageUrl((String) value);
// A new row has only the corrected link, so the next check would ask speedrun.com for nothing. // A new row has only the corrected link, so the next check would ask speedrun.com for nothing.
case SPEEDRUN -> gameSpeedrunRepository.writeWeblink(game.getAppid(), (String) value, Instant.now()); case SPEEDRUN -> gameSpeedrunRepository.writeWeblink(game.getAppid(), (String) value, Instant.now());
case MOODS -> gameMoodsRepository.writeMoods(game.getAppid(), strings(value), LONG_AGO); case MOODS -> gameMoodsRepository.writeMoods(game.getAppid(), strings(value), LONG_AGO);
@@ -77,8 +77,20 @@ public class GameDetailService {
if (userGameOpt.isEmpty() || userGameOpt.get().getRemovedAt() != null) { if (userGameOpt.isEmpty() || userGameOpt.get().getRemovedAt() != null) {
return Optional.empty(); return Optional.empty();
} }
UserGame userGame = userGameOpt.get();
SteamGame meta = steamGameRepository.findById(appid).orElse(null); SteamGame meta = steamGameRepository.findById(appid).orElse(null);
Refreshed refreshed = refresh(userGameOpt.get(), steamId);
return Optional.of(assemble(refreshed.userGame(), meta, refreshed.hasSchema()));
}
/**
* @param userGame the game after the refresh, re-read when the finish rule changed its status
*/
private record Refreshed(UserGame userGame, boolean hasSchema) {}
// May call Steam, IGDB and speedrun.com and write.
private Refreshed refresh(UserGame userGame, String steamId) {
Long userId = userGame.getUserId();
Long appid = userGame.getAppid();
// A hidden game gets no calls to Steam, IGDB or speedrun.com; it shows what is stored. // A hidden game gets no calls to Steam, IGDB or speedrun.com; it shows what is stored.
boolean hidden = userGame.getHiddenAt() != null; boolean hidden = userGame.getHiddenAt() != null;
if (!hidden) { if (!hidden) {
@@ -89,7 +101,7 @@ public class GameDetailService {
// Users without a Steam ID never sync, so the time to beat of their games is asked again here. // Users without a Steam ID never sync, so the time to beat of their games is asked again here.
timeToBeatService.refreshInBackground(List.of(appid)); timeToBeatService.refreshInBackground(List.of(appid));
moodService.refreshInBackground(List.of(appid)); moodService.refreshInBackground(List.of(appid));
return Optional.of(buildDto(userGame, meta, true, 0, 0, List.of())); return new Refreshed(userGame, false);
} }
if (!hidden) { if (!hidden) {
@@ -100,9 +112,8 @@ public class GameDetailService {
} }
} }
boolean hasSchema = schemaMetaRepository.findById(appid).map(GameAchievementSchemaMeta::isHasSchema).orElse(false); boolean hasSchema = schemaMetaRepository.findById(appid).map(GameAchievementSchemaMeta::isHasSchema).orElse(false);
if (!hasSchema) { if (!hasSchema) {
return Optional.of(buildDto(userGame, meta, true, 0, 0, List.of())); return new Refreshed(userGame, false);
} }
if (steamId != null && !hidden) { if (steamId != null && !hidden) {
@@ -117,7 +128,16 @@ public class GameDetailService {
if (!hidden && !userGame.isStatusSetByUser() && achievementSyncService.finishIfEndingUnlocked(userId, appid)) { if (!hidden && !userGame.isStatusSetByUser() && achievementSyncService.finishIfEndingUnlocked(userId, appid)) {
userGame = userGameRepository.findById(new UserGameId(userId, appid)).orElse(userGame); userGame = userGameRepository.findById(new UserGameId(userId, appid)).orElse(userGame);
} }
return new Refreshed(userGame, true);
}
// Reads only.
private GameDetailDto assemble(UserGame userGame, SteamGame meta, boolean hasSchema) {
if (!hasSchema) {
return buildDto(userGame, meta, true, 0, 0, List.of(), List.of());
}
Long userId = userGame.getUserId();
Long appid = userGame.getAppid();
UserGameAchievementMeta userMeta = userAchievementMetaRepository.findByUserIdAndAppid(userId, appid).orElse(null); UserGameAchievementMeta userMeta = userAchievementMetaRepository.findByUserIdAndAppid(userId, appid).orElse(null);
boolean statsVisible = userMeta == null || userMeta.isStatsVisible(); boolean statsVisible = userMeta == null || userMeta.isStatsVisible();
int unlocked = userMeta != null ? userMeta.getAchievementsUnlocked() : 0; int unlocked = userMeta != null ? userMeta.getAchievementsUnlocked() : 0;
@@ -145,13 +165,8 @@ public class GameDetailService {
}) })
.toList(); .toList();
return Optional.of(buildDto(userGame, meta, statsVisible, schemaList.size(), return buildDto(userGame, meta, statsVisible, schemaList.size(),
AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList)); AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList);
}
private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible,
int total, int unlocked, List<AchievementDto> achievements) {
return buildDto(userGame, meta, statsVisible, total, unlocked, achievements, List.of());
} }
private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible, int total, int unlocked, private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible, int total, int unlocked,
@@ -1,7 +1,7 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameImport; import core_service.modules.library.domain.GameImport;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.repo.GameImportRepository; import core_service.modules.library.repo.GameImportRepository;
import jakarta.annotation.PreDestroy; import jakarta.annotation.PreDestroy;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
@@ -49,7 +49,6 @@ public class GameImportWorker {
private final GameImportRepository importRepository; private final GameImportRepository importRepository;
private final ManualGameService manualGameService; private final ManualGameService manualGameService;
private final IgdbApiService igdbApiService;
private final TimeToBeatService timeToBeatService; private final TimeToBeatService timeToBeatService;
private final MoodService moodService; private final MoodService moodService;
private final Duration stallTime; private final Duration stallTime;
@@ -59,11 +58,10 @@ public class GameImportWorker {
private final Lane saving; private final Lane saving;
public GameImportWorker(GameImportRepository importRepository, ManualGameService manualGameService, public GameImportWorker(GameImportRepository importRepository, ManualGameService manualGameService,
IgdbApiService igdbApiService, TimeToBeatService timeToBeatService, MoodService moodService, TimeToBeatService timeToBeatService, MoodService moodService,
@Value("${app.library.import-stall-minutes:5}") long stallMinutes) { @Value("${app.library.import-stall-minutes:5}") long stallMinutes) {
this.importRepository = importRepository; this.importRepository = importRepository;
this.manualGameService = manualGameService; this.manualGameService = manualGameService;
this.igdbApiService = igdbApiService;
this.timeToBeatService = timeToBeatService; this.timeToBeatService = timeToBeatService;
this.moodService = moodService; this.moodService = moodService;
this.stallTime = Duration.ofMinutes(stallMinutes); this.stallTime = Duration.ofMinutes(stallMinutes);
@@ -140,11 +138,11 @@ public class GameImportWorker {
Map<Integer, GameImport.Row> rows = importRepository.findRows(importId).stream() Map<Integer, GameImport.Row> rows = importRepository.findRows(importId).stream()
.collect(Collectors.toMap(row -> row.line().lineNo(), row -> row)); .collect(Collectors.toMap(row -> row.line().lineNo(), row -> row));
for (GameImport.Choice choice : choices) { for (GameImport.Choice choice : choices) {
IgdbApiService.Game game = rows.get(choice.lineNo()).candidates().stream() IgdbGame game = rows.get(choice.lineNo()).candidates().stream()
.filter(candidate -> candidate.igdbId() == choice.igdbId()).findFirst() .filter(candidate -> candidate.igdbId() == choice.igdbId()).findFirst()
// A game from a new search in the review. IGDB knew it minutes ago, so no answer is an IGDB // A game from a new search in the review. IGDB knew it minutes ago, so no answer is an IGDB
// failure: the try stops and the line waits for the next one. // failure: the try stops and the line waits for the next one.
.or(() -> igdbApiService.getGame(choice.igdbId())) .or(() -> manualGameService.findGame(choice.igdbId()))
.orElseThrow(() -> new IllegalStateException("IGDB did not answer for game " + choice.igdbId())); .orElseThrow(() -> new IllegalStateException("IGDB did not answer for game " + choice.igdbId()));
if (Thread.currentThread().isInterrupted()) { if (Thread.currentThread().isInterrupted()) {
return; return;
@@ -170,11 +168,11 @@ public class GameImportWorker {
importRepository.markSaved(importId); importRepository.markSaved(importId);
} }
private record Match(boolean searchFailed, boolean exact, List<IgdbApiService.Game> candidates) {} private record Match(boolean searchFailed, boolean exact, List<IgdbGame> candidates) {}
private Match match(GameImport.Line line) { private Match match(GameImport.Line line) {
if (line.igdbId() != null) { if (line.igdbId() != null) {
Optional<IgdbApiService.Game> game = igdbApiService.getGame(line.igdbId()); Optional<IgdbGame> game = manualGameService.findGame(line.igdbId());
return new Match(game.isEmpty(), game.isPresent(), game.stream().toList()); return new Match(game.isEmpty(), game.isPresent(), game.stream().toList());
} }
return manualGameService.findGames(line.title()) return manualGameService.findGames(line.title())
@@ -185,9 +183,9 @@ public class GameImportWorker {
} }
// Same name first, among them one on the line's platform; else IGDB's order. // Same name first, among them one on the line's platform; else IGDB's order.
static List<IgdbApiService.Game> rank(GameImport.Line line, List<IgdbApiService.Game> found) { static List<IgdbGame> rank(GameImport.Line line, List<IgdbGame> found) {
Comparator<IgdbApiService.Game> order = Comparator Comparator<IgdbGame> order = Comparator
.comparing((IgdbApiService.Game game) -> !sameName(game.name(), line.title())) .comparing((IgdbGame game) -> !sameName(game.name(), line.title()))
.thenComparing(game -> line.platform() == null .thenComparing(game -> line.platform() == null
|| game.platforms().stream().noneMatch(line.platform()::equalsIgnoreCase)); || game.platforms().stream().noneMatch(line.platform()::equalsIgnoreCase));
return found.stream().sorted(order).limit(MAX_CANDIDATES).toList(); return found.stream().sorted(order).limit(MAX_CANDIDATES).toList();
@@ -0,0 +1,54 @@
package core_service.modules.library.service;
import core_service.modules.library.domain.GameKeys;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.TreeSet;
import java.util.function.Function;
/**
* IGDB finds a Steam game by its appid and a game off Steam by its IGDB id: one call per key kind, answers by appid.
*/
final class IgdbLookup {
/**
* @param answered the appids whose call worked, sorted; an appid without a value in {@code found} has no data
*/
record Answer<T>(Set<Long> answered, Map<Long, T> found) {}
private IgdbLookup() {
}
static <T> Answer<T> byAppid(Collection<Long> appids,
Function<Set<Long>, Optional<Map<Long, T>>> bySteamAppid,
Function<Set<Long>, Optional<Map<Long, T>>> byIgdbId) {
Set<Long> steamAppids = new TreeSet<>();
Set<Long> igdbIds = new TreeSet<>();
for (Long appid : appids) {
if (GameKeys.isSteam(appid)) {
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, T> found = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
bySteamAppid.apply(steamAppids).ifPresent(values -> {
found.putAll(values);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
byIgdbId.apply(igdbIds).ifPresent(values -> {
values.forEach((igdbId, value) -> found.put(GameKeys.ofIgdb(igdbId), value));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
return new Answer<>(answered, found);
}
}
@@ -37,6 +37,7 @@ import java.util.Set;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import core_service.modules.library.domain.UserSteamSync.Status; import core_service.modules.library.domain.UserSteamSync.Status;
import core_service.shared.background.BackgroundPools;
/** /**
* Worker bean. Runs the actual Steam fetch + DB upserts. Separate bean so the Spring proxy applies @Async * Worker bean. Runs the actual Steam fetch + DB upserts. Separate bean so the Spring proxy applies @Async
@@ -68,7 +69,7 @@ public class LibrarySyncWorker {
@Value("${app.library.zone:Europe/Berlin}") @Value("${app.library.zone:Europe/Berlin}")
private ZoneId zone; private ZoneId zone;
@Async @Async(BackgroundPools.DEFAULT)
public void runSync(Long userId, String steamId) { public void runSync(Long userId, String steamId) {
sync(userId, steamId); sync(userId, steamId);
} }
@@ -4,6 +4,7 @@ import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.api.dto.GameSearchDto; import core_service.modules.library.api.dto.GameSearchDto;
import core_service.modules.library.api.dto.ManualGameUpdate; import core_service.modules.library.api.dto.ManualGameUpdate;
import core_service.modules.library.domain.GameKeys; import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameSource; import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.SteamGame;
@@ -93,24 +94,22 @@ public class ManualGameService {
// Games already in ToBePlayed first, with their catalog name and image (an auditor's correction included); then // Games already in ToBePlayed first, with their catalog name and image (an auditor's correction included); then
// games added by hand whose name matches (also a duplicate of a game IGDB now knows on Steam, so an auditor can // games added by hand whose name matches (also a duplicate of a game IGDB now knows on Steam, so an auditor can
// merge it); then the rest in IGDB's order. One hit per key. // merge it); then the rest in IGDB's order. One hit per key.
private List<GameSearchDto.Hit> hits(Long userId, String query, List<IgdbApiService.Game> games) { private List<GameSearchDto.Hit> hits(Long userId, String query, List<IgdbGame> games) {
Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList()); Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList());
Map<Long, SteamGame> catalog = steamGameRepository.findAllById(games.stream().map(game -> key(game, targets)).toList()) Map<Long, SteamGame> catalog = steamGameRepository.findAllById(games.stream().map(game -> key(game, targets)).toList())
.stream().collect(Collectors.toMap(SteamGame::getAppid, game -> game)); .stream().collect(Collectors.toMap(SteamGame::getAppid, game -> game));
List<SteamGame> addedByOthers = steamGameRepository.findManualByName(UserGameRepository.containsPattern(query), List<SteamGame> addedByOthers = steamGameRepository.findManualByName(UserGameRepository.containsPattern(query),
PageRequest.of(0, MAX_CATALOG_HITS)); PageRequest.of(0, MAX_CATALOG_HITS));
Set<Long> owned = new HashSet<>(userGameRepository.findActiveAppids(userId, Stream.concat( Set<Long> owned = ownedAppids(userId, games, targets, addedByOthers.stream().map(SteamGame::getAppid).toList());
games.stream().flatMap(game -> possibleKeys(game, targets).stream()),
addedByOthers.stream().map(SteamGame::getAppid)).toList()));
List<GameSearchDto.Hit> inCatalog = new ArrayList<>(); List<GameSearchDto.Hit> inCatalog = new ArrayList<>();
List<GameSearchDto.Hit> rest = new ArrayList<>(); List<GameSearchDto.Hit> rest = new ArrayList<>();
for (IgdbApiService.Game game : games) { for (IgdbGame game : games) {
long appid = key(game, targets); long appid = key(game, targets);
SteamGame entry = catalog.get(appid); SteamGame entry = catalog.get(appid);
GameSearchDto.Hit hit = new GameSearchDto.Hit(appid, game.igdbId(), entry != null ? entry.getName() : game.name(), GameSearchDto.Hit hit = new GameSearchDto.Hit(appid, game.igdbId(), entry != null ? entry.getName() : game.name(),
game.year(), entry != null && entry.getImageUrl() != null ? entry.getImageUrl() : game.imageUrl(), game.year(), entry != null && entry.getImageUrl() != null ? entry.getImageUrl() : game.imageUrl(),
fittingPlatforms(game), possibleKeys(game, targets).stream().anyMatch(owned::contains), entry != null); fittingPlatforms(game), isOwned(game, targets, owned), entry != null);
(entry != null ? inCatalog : rest).add(hit); (entry != null ? inCatalog : rest).add(hit);
} }
Stream<GameSearchDto.Hit> others = addedByOthers.stream().map(game -> new GameSearchDto.Hit(game.getAppid(), Stream<GameSearchDto.Hit> others = addedByOthers.stream().map(game -> new GameSearchDto.Hit(game.getAppid(),
@@ -125,7 +124,7 @@ public class ManualGameService {
// Few hits often mean a typo. IGDB's own hits keep their order; the variant hits follow, closest name first. // Few hits often mean a typo. IGDB's own hits keep their order; the variant hits follow, closest name first.
// The variants go through the same cache, and one that fails adds nothing. // The variants go through the same cache, and one that fails adds nothing.
private List<IgdbApiService.Game> withTypoHelp(String query, List<IgdbApiService.Game> found) { private List<IgdbGame> withTypoHelp(String query, List<IgdbGame> found) {
if (found.size() >= MIN_HITS) { if (found.size() >= MIN_HITS) {
return found; return found;
} }
@@ -137,12 +136,12 @@ public class ManualGameService {
Set<Long> seen = new HashSet<>(); Set<Long> seen = new HashSet<>();
found.forEach(game -> seen.add(game.igdbId())); found.forEach(game -> seen.add(game.igdbId()));
List<IgdbApiService.Game> extra = new ArrayList<>(); List<IgdbGame> extra = new ArrayList<>();
for (String variant : variants) { for (String variant : variants) {
if (found.size() + extra.size() >= MIN_HITS) { if (found.size() + extra.size() >= MIN_HITS) {
break; break;
} }
igdbApiService.searchGames(variant).orElse(List.of()).stream() igdbApiService.searchGames(variant).orElse(List.of()).stream().map(ManualGameService::toIgdbGame)
.filter(game -> seen.add(game.igdbId())) .filter(game -> seen.add(game.igdbId()))
.forEach(extra::add); .forEach(extra::add);
} }
@@ -162,7 +161,7 @@ public class ManualGameService {
return new AddResult.SearchUnavailable(); return new AddResult.SearchUnavailable();
} }
// Outside the transaction: no database connection waits for IGDB. // Outside the transaction: no database connection waits for IGDB.
Optional<IgdbApiService.Game> found = igdbApiService.getGame(igdbId); Optional<IgdbGame> found = findGame(igdbId);
if (found.isEmpty()) { if (found.isEmpty()) {
return new AddResult.UnknownGame(); return new AddResult.UnknownGame();
} }
@@ -181,7 +180,7 @@ public class ManualGameService {
* @param importId the import that adds the game, so its undo can remove it; null for a single add. A removed * @param importId the import that adds the game, so its undo can remove it; null for a single add. A removed
* row that comes back keeps no import id. * row that comes back keeps no import id.
*/ */
public AddResult addGame(Long userId, IgdbApiService.Game game, String platform, GameStatus status, public AddResult addGame(Long userId, IgdbGame game, String platform, GameStatus status,
int playtimeMinutes, Long importId) { int playtimeMinutes, Long importId) {
AddResult result; AddResult result;
try { try {
@@ -201,35 +200,34 @@ public class ManualGameService {
/** /**
* IGDB's hits for a title, with the same typo help as {@link #search}; empty when IGDB failed or is off. * IGDB's hits for a title, with the same typo help as {@link #search}; empty when IGDB failed or is off.
*/ */
public Optional<List<IgdbApiService.Game>> findGames(String query) { public Optional<List<IgdbGame>> findGames(String query) {
if (!igdbApiService.isEnabled()) { if (!igdbApiService.isEnabled()) {
return Optional.empty(); return Optional.empty();
} }
return igdbApiService.searchGames(query).map(found -> withTypoHelp(query, found)); return igdbApiService.searchGames(query)
.map(found -> withTypoHelp(query, found.stream().map(ManualGameService::toIgdbGame).toList()));
} }
/** /**
* The IGDB ids of the games the user already has, under any of their keys (a merge included). * The IGDB ids of the games the user already has, under any of their keys (a merge included).
*/ */
public Set<Long> ownedIgdbIds(Long userId, Collection<IgdbApiService.Game> games) { public Set<Long> ownedIgdbIds(Long userId, Collection<IgdbGame> games) {
if (games.isEmpty()) { if (games.isEmpty()) {
return Set.of(); return Set.of();
} }
Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList()); Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList());
Set<Long> owned = new HashSet<>(userGameRepository.findActiveAppids(userId, Set<Long> owned = ownedAppids(userId, games, targets, List.of());
games.stream().flatMap(game -> possibleKeys(game, targets).stream()).distinct().toList())); return games.stream().filter(game -> isOwned(game, targets, owned))
return games.stream().filter(game -> possibleKeys(game, targets).stream().anyMatch(owned::contains)) .map(IgdbGame::igdbId).collect(Collectors.toSet());
.map(IgdbApiService.Game::igdbId).collect(Collectors.toSet());
} }
private AddResult addInTransaction(Long userId, IgdbApiService.Game game, String platform, GameStatus status, private AddResult addInTransaction(Long userId, IgdbGame game, String platform, GameStatus status,
int playtimeMinutes, Long importId) { int playtimeMinutes, Long importId) {
return transaction.execute(tx -> { return transaction.execute(tx -> {
Instant now = Instant.now(); Instant now = Instant.now();
Map<Long, Long> targets = mergeRepository.findTargets(ownKeys(game)); Map<Long, Long> targets = mergeRepository.findTargets(ownKeys(game));
long appid = key(game, targets); long appid = key(game, targets);
List<Long> possibleKeys = possibleKeys(game, targets); if (!ownedAppids(userId, List.of(game), targets, List.of()).isEmpty()) {
if (!userGameRepository.findActiveAppids(userId, possibleKeys).isEmpty()) {
return new AddResult.AlreadyInLibrary(); return new AddResult.AlreadyInLibrary();
} }
Optional<UserGame> existing = userGameRepository.findById(new UserGameId(userId, appid)); Optional<UserGame> existing = userGameRepository.findById(new UserGameId(userId, appid));
@@ -287,14 +285,11 @@ public class ManualGameService {
// A Steam row belongs to the sync, which has the better name and art; only a game off Steam takes IGDB's, and // A Steam row belongs to the sync, which has the better name and art; only a game off Steam takes IGDB's, and
// only its own: a merged game must not rename the game it went into. // only its own: a merged game must not rename the game it went into.
private void saveCatalogEntry(long appid, IgdbApiService.Game game, Instant now) { private void saveCatalogEntry(long appid, IgdbGame game, Instant now) {
Optional<SteamGame> existing = steamGameRepository.findById(appid); Optional<SteamGame> existing = steamGameRepository.findById(appid);
if (existing.isEmpty()) { if (existing.isEmpty()) {
SteamGame created = new SteamGame(appid, game.name(), null, now); String imageUrl = GameKeys.isSteam(appid) ? null : game.imageUrl();
if (!GameKeys.isSteam(appid)) { steamGameRepository.save(new SteamGame(appid, game.name(), null, imageUrl, now));
created.setImageUrl(game.imageUrl());
}
steamGameRepository.save(created);
} else if (appid == GameKeys.ofIgdb(game.igdbId())) { } else if (appid == GameKeys.ofIgdb(game.igdbId())) {
steamGameRepository.setSourceName(appid, game.name()); steamGameRepository.setSourceName(appid, game.name());
steamGameRepository.setImageUrl(appid, game.imageUrl()); steamGameRepository.setImageUrl(appid, game.imageUrl());
@@ -303,25 +298,47 @@ public class ManualGameService {
} }
// A longer name would not fit the column; with no platform left, the user types one. // A longer name would not fit the column; with no platform left, the user types one.
static List<String> fittingPlatforms(IgdbApiService.Game game) { static List<String> fittingPlatforms(IgdbGame game) {
return game.platforms().stream().filter(platform -> platform.length() <= ManualGameUpdate.MAX_PLATFORM_LENGTH).toList(); return game.platforms().stream().filter(platform -> platform.length() <= ManualGameUpdate.MAX_PLATFORM_LENGTH).toList();
} }
/**
* One game by its IGDB id; empty when IGDB does not know it or failed.
*/
public Optional<IgdbGame> findGame(long igdbId) {
return igdbApiService.getGame(igdbId).map(ManualGameService::toIgdbGame);
}
private static IgdbGame toIgdbGame(IgdbApiService.Game game) {
return new IgdbGame(game.igdbId(), game.name(), game.year(), game.imageUrl(), game.platforms(), game.steamAppid());
}
// IGDB can learn a game's Steam appid after the user added it under minus its IGDB id. Each key goes through its // IGDB can learn a game's Steam appid after the user added it under minus its IGDB id. Each key goes through its
// merge, if an auditor merged it. // merge, if an auditor merged it.
private static List<Long> possibleKeys(IgdbApiService.Game game, Map<Long, Long> targets) { private static List<Long> possibleKeys(IgdbGame game, Map<Long, Long> targets) {
return ownKeys(game).stream().map(own -> targets.getOrDefault(own, own)).distinct().toList(); return ownKeys(game).stream().map(own -> targets.getOrDefault(own, own)).distinct().toList();
} }
private Set<Long> ownedAppids(Long userId, Collection<IgdbGame> games, Map<Long, Long> targets,
Collection<Long> extraAppids) {
return new HashSet<>(userGameRepository.findActiveAppids(userId, Stream.concat(
games.stream().flatMap(game -> possibleKeys(game, targets).stream()), extraAppids.stream())
.distinct().toList()));
}
private static boolean isOwned(IgdbGame game, Map<Long, Long> targets, Set<Long> ownedAppids) {
return possibleKeys(game, targets).stream().anyMatch(ownedAppids::contains);
}
/** /**
* @param targets {@link GameMergeRepository#findTargets} of the {@link #ownKeys} * @param targets {@link GameMergeRepository#findTargets} of the {@link #ownKeys}
*/ */
private static long key(IgdbApiService.Game game, Map<Long, Long> targets) { private static long key(IgdbGame game, Map<Long, Long> targets) {
return possibleKeys(game, targets).getFirst(); return possibleKeys(game, targets).getFirst();
} }
// The key IGDB gives today first, then minus the IGDB id when the game is on Steam. // The key IGDB gives today first, then minus the IGDB id when the game is on Steam.
private static List<Long> ownKeys(IgdbApiService.Game game) { private static List<Long> ownKeys(IgdbGame game) {
long igdbKey = GameKeys.ofIgdb(game.igdbId()); long igdbKey = GameKeys.ofIgdb(game.igdbId());
return game.steamAppid() != null ? List.of(game.steamAppid(), igdbKey) : List.of(igdbKey); return game.steamAppid() != null ? List.of(game.steamAppid(), igdbKey) : List.of(igdbKey);
} }
@@ -1,23 +1,19 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.Mood; import core_service.modules.library.domain.Mood;
import core_service.modules.library.repo.GameMoodsRepository; import core_service.modules.library.repo.GameMoodsRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async; import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import java.time.Duration;
import java.time.Instant; import java.time.Instant;
import java.util.Collection; import java.util.Collection;
import java.util.HashMap;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
import java.util.TreeSet;
/** /**
* Stores the IGDB genres of games and the {@link Mood}s from IGDB's genres, themes and game modes, the same way * Stores the IGDB genres of games and the {@link Mood}s from IGDB's genres, themes and game modes, the same way
@@ -31,40 +27,20 @@ public class MoodService {
private final IgdbApiService igdbApiService; private final IgdbApiService igdbApiService;
private final GameMoodsRepository gameMoodsRepository; private final GameMoodsRepository gameMoodsRepository;
private final UserGameRepository userGameRepository; private final UserGameRepository userGameRepository;
private final TaskExecutor applicationTaskExecutor;
@Value("${app.library.mood-ttl-days:30}") @Value("${app.library.mood-ttl-days:30}")
private long ttlDays; private long ttlDays;
public void refresh(Collection<Long> appids) { public void refresh(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return; if (!igdbApiService.isEnabled() || appids.isEmpty()) return;
// Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock. Set<Long> stale = stale(appids);
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(gameMoodsRepository.findFreshAppids(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
if (stale.isEmpty()) return; if (stale.isEmpty()) return;
Set<Long> steamAppids = new TreeSet<>(); IgdbLookup.Answer<IgdbApiService.GameTags> answer = IgdbLookup.byAppid(stale, igdbApiService::getGameTags,
Set<Long> igdbIds = new TreeSet<>(); igdbApiService::getGameTagsByGameIds);
for (Long appid : stale) { Set<Long> answered = answer.answered();
if (GameKeys.isSteam(appid)) { Map<Long, IgdbApiService.GameTags> tags = answer.found();
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, IgdbApiService.GameTags> tags = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
igdbApiService.getGameTags(steamAppids).ifPresent(found -> {
tags.putAll(found);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
igdbApiService.getGameTagsByGameIds(igdbIds).ifPresent(found -> {
found.forEach((igdbId, gameTags) -> tags.put(GameKeys.ofIgdb(igdbId), gameTags));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
if (answered.size() < stale.size()) { if (answered.size() < stale.size()) {
log.warn("Moods of {} games not refreshed, IGDB did not answer", stale.size() - answered.size()); log.warn("Moods of {} games not refreshed, IGDB did not answer", stale.size() - answered.size());
} }
@@ -83,36 +59,31 @@ public class MoodService {
/** /**
* Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it. * Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it.
*/ */
@Async
public void refreshInBackground(Collection<Long> appids) { public void refreshInBackground(Collection<Long> appids) {
try { if (!igdbApiService.isEnabled()) return;
refresh(appids); Refreshes.staleInBackground(applicationTaskExecutor, "Moods of " + appids, () -> appids, this::stale,
} catch (Exception e) { this::refresh);
log.warn("Moods of {} not refreshed: {}", appids, e.getMessage());
}
} }
/** /**
* For the genres and moods of the stats page: a user without a Steam sync gets them only here. * For the genres and moods of the stats page: a user without a Steam sync gets them only here.
*/ */
@Async
public void refreshListedInBackground(Long userId) { public void refreshListedInBackground(Long userId) {
try { if (!igdbApiService.isEnabled()) return;
refresh(userGameRepository.findAllListedAppids(userId)); Refreshes.staleInBackground(applicationTaskExecutor, "Moods of the library of user " + userId,
} catch (Exception e) { () -> userGameRepository.findAllListedAppids(userId), this::stale, this::refresh);
log.warn("Moods of the library of user {} not refreshed: {}", userId, e.getMessage());
}
} }
/** /**
* For the picker: a user without a Steam sync gets moods only here, and a Steam user before the next sync. * For the picker: a user without a Steam sync gets moods only here, and a Steam user before the next sync.
*/ */
@Async
public void refreshBacklogInBackground(Long userId) { public void refreshBacklogInBackground(Long userId) {
try { if (!igdbApiService.isEnabled()) return;
refresh(userGameRepository.findBacklogAppids(userId)); Refreshes.staleInBackground(applicationTaskExecutor, "Backlog moods of user " + userId,
} catch (Exception e) { () -> userGameRepository.findBacklogAppids(userId), this::stale, this::refresh);
log.warn("Backlog moods of user {} not refreshed: {}", userId, e.getMessage()); }
}
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameMoodsRepository::findFreshAppids, ttlDays);
} }
} }
@@ -0,0 +1,64 @@
package core_service.modules.library.service;
import lombok.extern.slf4j.Slf4j;
import org.springframework.core.task.TaskExecutor;
import java.time.Duration;
import java.time.Instant;
import java.util.Collection;
import java.util.Set;
import java.util.TreeSet;
import java.util.function.BiFunction;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.function.Supplier;
/**
* The shared steps of the IGDB and speedrun.com refreshes: which games have no fresh row, and the background run.
*/
@Slf4j
final class Refreshes {
private Refreshes() {
}
/**
* Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock.
*
* @param findFreshAppids the appids among the given ones with a row newer than the instant
*/
static Set<Long> stale(Collection<Long> appids, BiFunction<Collection<Long>, Instant, Set<Long>> findFreshAppids,
long ttlDays) {
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(findFreshAppids.apply(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
return stale;
}
/**
* Checks on the calling thread which games are stale and refreshes only those in the background, so a fresh game
* takes no place in the queue. No error reaches the caller: the next view or sync asks again.
*
* @param appids read on the calling thread
*/
static void staleInBackground(TaskExecutor executor, String what, Supplier<Collection<Long>> appids,
Function<Collection<Long>, Set<Long>> stale, Consumer<Set<Long>> refresh) {
try {
Collection<Long> candidates = appids.get();
if (candidates.isEmpty()) return;
Set<Long> staleAppids = stale.apply(candidates);
if (staleAppids.isEmpty()) return;
// ponytail: views of one stale game in the same second ask once each; an in-flight set if that hurts
executor.execute(() -> quietly(what, () -> refresh.accept(staleAppids)));
} catch (Exception e) {
log.warn("{} not refreshed: {}", what, e.getMessage());
}
}
static void quietly(String what, Runnable refresh) {
try {
refresh.run();
} catch (Exception e) {
log.warn("{} not refreshed: {}", what, e.getMessage());
}
}
}
@@ -8,9 +8,11 @@ import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService; import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.shared.background.BackgroundPools;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.annotation.Async; import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import org.springframework.transaction.event.TransactionalEventListener; import org.springframework.transaction.event.TransactionalEventListener;
@@ -46,6 +48,7 @@ public class SpeedrunService {
private final GameSpeedrunRepository gameSpeedrunRepository; private final GameSpeedrunRepository gameSpeedrunRepository;
private final SteamGameRepository steamGameRepository; private final SteamGameRepository steamGameRepository;
private final UserGameRepository userGameRepository; private final UserGameRepository userGameRepository;
private final TaskExecutor syncFollowUpExecutor;
private final ConcurrentSkipListSet<Long> pending = new ConcurrentSkipListSet<>(); private final ConcurrentSkipListSet<Long> pending = new ConcurrentSkipListSet<>();
private final AtomicBoolean running = new AtomicBoolean(); private final AtomicBoolean running = new AtomicBoolean();
@@ -71,28 +74,27 @@ public class SpeedrunService {
} }
} }
@Async @Async(BackgroundPools.SYNC_FOLLOW_UP)
@TransactionalEventListener @TransactionalEventListener
public void onLibrarySynced(LibrarySyncedEvent event) { public void onLibrarySynced(LibrarySyncedEvent event) {
refreshBacklogInBackground(event.userId()); Refreshes.quietly("Speedrun games of the backlog of user " + event.userId(),
() -> refresh(userGameRepository.findBacklogAppids(event.userId())));
} }
@Async
public void refreshBacklogInBackground(Long userId) { public void refreshBacklogInBackground(Long userId) {
try { if (!speedrunApiService.isEnabled()) return;
refresh(userGameRepository.findBacklogAppids(userId)); Refreshes.staleInBackground(syncFollowUpExecutor, "Speedrun games of the backlog of user " + userId,
} catch (Exception e) { () -> userGameRepository.findBacklogAppids(userId), this::stale, this::refresh);
log.warn("Speedrun games of the backlog of user {} not refreshed: {}", userId, e.getMessage());
}
} }
@Async
public void refreshInBackground(Collection<Long> appids) { public void refreshInBackground(Collection<Long> appids) {
try { if (!speedrunApiService.isEnabled()) return;
refresh(appids); Refreshes.staleInBackground(syncFollowUpExecutor, "Speedrun games of " + appids, () -> appids, this::stale,
} catch (Exception e) { this::refresh);
log.warn("Speedrun games of {} not refreshed: {}", appids, e.getMessage()); }
}
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameSpeedrunRepository::findFreshAppids, ttlDays);
} }
private void drain() { private void drain() {
@@ -1,21 +1,17 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value; import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async; import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
import java.time.Duration;
import java.time.Instant; import java.time.Instant;
import java.util.Collection; import java.util.Collection;
import java.util.HashMap;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
import java.util.TreeSet;
@Service @Service
@RequiredArgsConstructor @RequiredArgsConstructor
@@ -24,20 +20,19 @@ public class TimeToBeatService {
private final IgdbApiService igdbApiService; private final IgdbApiService igdbApiService;
private final GameTimeToBeatRepository gameTimeToBeatRepository; private final GameTimeToBeatRepository gameTimeToBeatRepository;
private final TaskExecutor applicationTaskExecutor;
@Value("${app.library.time-to-beat-ttl-days:30}") @Value("${app.library.time-to-beat-ttl-days:30}")
private long ttlDays; private long ttlDays;
/** /**
* Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it. * Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it. Only stale apps
* take a place in the background queue.
*/ */
@Async
public void refreshInBackground(Collection<Long> appids) { public void refreshInBackground(Collection<Long> appids) {
try { if (!igdbApiService.isEnabled()) return;
refresh(appids); Refreshes.staleInBackground(applicationTaskExecutor, "Time to beat of " + appids, () -> appids, this::stale,
} catch (Exception e) { this::refresh);
log.warn("Time to beat of {} not refreshed: {}", appids, e.getMessage());
}
} }
/** /**
@@ -46,34 +41,13 @@ public class TimeToBeatService {
*/ */
public void refresh(Collection<Long> appids) { public void refresh(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return; if (!igdbApiService.isEnabled() || appids.isEmpty()) return;
// Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock. Set<Long> stale = stale(appids);
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(gameTimeToBeatRepository.findFreshAppids(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
if (stale.isEmpty()) return; if (stale.isEmpty()) return;
Set<Long> steamAppids = new TreeSet<>(); IgdbLookup.Answer<IgdbApiService.TimeToBeat> answer = IgdbLookup.byAppid(stale, igdbApiService::getTimeToBeat,
Set<Long> igdbIds = new TreeSet<>(); igdbApiService::getTimeToBeatByGameIds);
for (Long appid : stale) { Set<Long> answered = answer.answered();
if (GameKeys.isSteam(appid)) { Map<Long, IgdbApiService.TimeToBeat> times = answer.found();
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, IgdbApiService.TimeToBeat> times = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
igdbApiService.getTimeToBeat(steamAppids).ifPresent(found -> {
times.putAll(found);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
igdbApiService.getTimeToBeatByGameIds(igdbIds).ifPresent(found -> {
found.forEach((igdbId, time) -> times.put(GameKeys.ofIgdb(igdbId), time));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
if (answered.size() < stale.size()) { if (answered.size() < stale.size()) {
log.warn("Time to beat for {} apps not refreshed, IGDB did not answer", stale.size() - answered.size()); log.warn("Time to beat for {} apps not refreshed, IGDB did not answer", stale.size() - answered.size());
} }
@@ -87,4 +61,8 @@ public class TimeToBeatService {
} }
} }
} }
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameTimeToBeatRepository::findFreshAppids, ttlDays);
}
} }
@@ -10,8 +10,7 @@ import java.util.Map;
@Repository @Repository
public class StatsRepository { public class StatsRepository {
private static final String LISTED = private static final String NOT_DELETED = "user_id NOT IN (SELECT user_id FROM deleted_user)";
"removed_at IS NULL AND hidden_at IS NULL AND user_id NOT IN (SELECT user_id FROM deleted_user)";
private final JdbcTemplate jdbc; private final JdbcTemplate jdbc;
@@ -22,77 +21,75 @@ public class StatsRepository {
public long countDistinctGames() { public long countDistinctGames() {
Long v = jdbc.queryForObject( Long v = jdbc.queryForObject(
"SELECT COUNT(DISTINCT appid) FROM user_game WHERE " + LISTED, Long.class); "SELECT COUNT(DISTINCT appid) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long countLibraryEntries() { public long countLibraryEntries() {
Long v = jdbc.queryForObject( Long v = jdbc.queryForObject(
"SELECT COUNT(*) FROM user_game WHERE " + LISTED, Long.class); "SELECT COUNT(*) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long countGamesPlayed() { public long countGamesPlayed() {
Long v = jdbc.queryForObject( Long v = jdbc.queryForObject(
"SELECT COUNT(*) FROM user_game WHERE " + LISTED + " AND playtime_minutes > 0", Long.class); "SELECT COUNT(*) FROM listed_user_game WHERE " + NOT_DELETED + " AND playtime_minutes > 0", Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long sumPlaytimeMinutes() { public long sumPlaytimeMinutes() {
Long v = jdbc.queryForObject( Long v = jdbc.queryForObject(
"SELECT COALESCE(SUM(playtime_minutes), 0) FROM user_game WHERE " + LISTED, Long.class); "SELECT COALESCE(SUM(playtime_minutes), 0) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long countUsersWithLibrary() { public long countUsersWithLibrary() {
Long v = jdbc.queryForObject( Long v = jdbc.queryForObject(
"SELECT COUNT(DISTINCT user_id) FROM user_game WHERE " + LISTED, Long.class); "SELECT COUNT(DISTINCT user_id) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long biggestLibrary() { public long biggestLibrary() {
Long v = jdbc.queryForObject(""" Long v = jdbc.queryForObject("""
SELECT COALESCE(MAX(c), 0) FROM ( SELECT COALESCE(MAX(c), 0) FROM (
SELECT COUNT(*) AS c FROM user_game SELECT COUNT(*) AS c FROM listed_user_game
WHERE %s GROUP BY user_id WHERE %s GROUP BY user_id
) t ) t
""".formatted(LISTED), Long.class); """.formatted(NOT_DELETED), Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public long biggestBacklog() { public long biggestBacklog() {
Long v = jdbc.queryForObject(""" Long v = jdbc.queryForObject("""
SELECT COALESCE(MAX(c), 0) FROM ( SELECT COALESCE(MAX(c), 0) FROM (
SELECT COUNT(*) AS c FROM user_game SELECT COUNT(*) AS c FROM listed_user_game
WHERE %s AND playtime_minutes = 0 GROUP BY user_id WHERE %s AND playtime_minutes = 0 GROUP BY user_id
) t ) t
""".formatted(LISTED), Long.class); """.formatted(NOT_DELETED), Long.class);
return v != null ? v : 0L; return v != null ? v : 0L;
} }
public List<Map<String, Object>> topOwnedGames(int limit) { public List<Map<String, Object>> topOwnedGames(int limit) {
return jdbc.queryForList(""" return jdbc.queryForList("""
SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, COUNT(*) AS value SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, COUNT(*) AS value
FROM user_game ug FROM listed_user_game ug
LEFT JOIN steam_game sg ON sg.appid = ug.appid LEFT JOIN steam_game sg ON sg.appid = ug.appid
WHERE ug.removed_at IS NULL AND ug.hidden_at IS NULL WHERE %s
AND ug.user_id NOT IN (SELECT user_id FROM deleted_user)
GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url
ORDER BY value DESC ORDER BY value DESC
LIMIT ? LIMIT ?
""", limit); """.formatted(NOT_DELETED), limit);
} }
public List<Map<String, Object>> topPlayedGames(int limit) { public List<Map<String, Object>> topPlayedGames(int limit) {
return jdbc.queryForList(""" return jdbc.queryForList("""
SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, SUM(ug.playtime_minutes) AS value SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, SUM(ug.playtime_minutes) AS value
FROM user_game ug FROM listed_user_game ug
LEFT JOIN steam_game sg ON sg.appid = ug.appid LEFT JOIN steam_game sg ON sg.appid = ug.appid
WHERE ug.removed_at IS NULL AND ug.hidden_at IS NULL WHERE %s
AND ug.user_id NOT IN (SELECT user_id FROM deleted_user)
GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url
ORDER BY value DESC ORDER BY value DESC
LIMIT ? LIMIT ?
""", limit); """.formatted(NOT_DELETED), limit);
} }
} }
@@ -0,0 +1,22 @@
package core_service.shared.background;
import org.springframework.boot.autoconfigure.task.TaskExecutionAutoConfiguration;
/**
* The bean names of the two background pools (configuration.AsyncConfig). Every {@code @Async} names one of them:
* with the scheduler there are two TaskExecutor beans, and a bare {@code @Async} would run unbounded.
*/
public final class BackgroundPools {
/** Syncs and the refreshes a page view starts: short tasks, a full queue drops the task. */
public static final String DEFAULT = TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME;
/**
* The achievement runs after a sync and every speedrun refresh: they hold a thread for minutes (Steam and
* speedrun.com pace), so page views keep the default pool. Small tasks, nothing is dropped.
*/
public static final String SYNC_FOLLOW_UP = "syncFollowUpExecutor";
private BackgroundPools() {
}
}
@@ -10,7 +10,7 @@ import java.lang.annotation.Target;
* The lowest active role that may call this service method, or every method of this class. A method marker * The lowest active role that may call this service method, or every method of this class. A method marker
* replaces the class marker. Every service method a controller calls carries one of {@link RequiresRole}, * replaces the class marker. Every service method a controller calls carries one of {@link RequiresRole},
* {@link Public} or {@link Internal} (ArchitectureTest). Checked only while an HTTP request is served: scheduled * {@link Public} or {@link Internal} (ArchitectureTest). Checked only while an HTTP request is served: scheduled
* jobs and {@code @Async} work run without a user. * jobs and background work (the pools in {@code AsyncConfig}) run without a user.
*/ */
@Documented @Documented
@Retention(RetentionPolicy.RUNTIME) @Retention(RetentionPolicy.RUNTIME)
@@ -0,0 +1,6 @@
-- The games a user's lists and numbers show: still owned and not hidden. Native SQL reads this view; JPQL uses
-- UserGameRepository.LISTED. Postgres fixes the columns of "select *" at creation and blocks dropping or retyping a
-- column the view uses: a migration that adds, drops or retypes a column of user_game drops this view first and
-- creates it again.
CREATE VIEW listed_user_game AS
SELECT * FROM user_game WHERE removed_at IS NULL AND hidden_at IS NULL;
@@ -1,14 +1,24 @@
package core_service.architecture; package core_service.architecture;
import core_service.shared.background.BackgroundPools;
import core_service.shared.test.AbstractArchitectureTest; import core_service.shared.test.AbstractArchitectureTest;
import com.tngtech.archunit.base.DescribedPredicate;
import com.tngtech.archunit.core.domain.JavaClass; import com.tngtech.archunit.core.domain.JavaClass;
import com.tngtech.archunit.core.domain.JavaMethod;
import com.tngtech.archunit.core.importer.ImportOption; import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.junit.AnalyzeClasses; import com.tngtech.archunit.junit.AnalyzeClasses;
import com.tngtech.archunit.junit.ArchTest; import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.lang.ArchCondition;
import com.tngtech.archunit.lang.ArchRule; import com.tngtech.archunit.lang.ArchRule;
import com.tngtech.archunit.lang.ConditionEvents;
import com.tngtech.archunit.lang.SimpleConditionEvent;
import com.tngtech.archunit.library.dependencies.SlicesRuleDefinition; import com.tngtech.archunit.library.dependencies.SlicesRuleDefinition;
import org.springframework.scheduling.annotation.Async;
import java.util.Set;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.classes; import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.classes;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.methods;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses; import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses;
/** /**
@@ -59,12 +69,57 @@ class ArchitectureTest extends AbstractArchitectureTest {
.should().dependOnClassesThat().resideInAPackage("..api..") .should().dependOnClassesThat().resideInAPackage("..api..")
.because("Persistence layer must be independent of the API layer."); .because("Persistence layer must be independent of the API layer.");
@ArchTest
static final ArchRule domain_should_not_depend_on_other_layers = noClasses()
.that().resideInAPackage("core_service.modules..domain..")
.should().dependOnClassesThat().resideInAnyPackage("..api..", "..service..", "..repo..",
"core_service.modules.steam_adapter..", "core_service.modules.igdb_adapter..",
"core_service.modules.speedrun_adapter..")
.because("Entities and stored shapes must not change when a controller, service or external API changes.");
// The listed services build their module's response DTOs; the list only gets shorter.
@ArchTest
static final ArchRule services_should_not_depend_on_apis = noClasses()
.that().resideInAPackage("core_service.modules..service..")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.BacklogService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.GameDetailService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.GameImportService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.LibraryQueryService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.ManualGameService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.SteamProfileService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.UserStatsService")
.and().doNotHaveFullyQualifiedName("core_service.modules.stats.service.PublicStatsService")
.should().dependOnClassesThat(JavaClass.Predicates.resideInAPackage("..api..")
.and(DescribedPredicate.not(JavaClass.Predicates.resideInAnyPackage(
"core_service.modules.steam_adapter.api.dto.response..",
"core_service.modules.igdb_adapter.api.dto.response..",
"core_service.modules.speedrun_adapter.api.dto.response.."))))
.because("Services must not depend on the HTTP layer; the adapters' response DTOs are the external APIs' shapes.");
@ArchTest @ArchTest
static final ArchRule repos_should_be_interfaces_named_repository = classes() static final ArchRule repos_should_be_interfaces_named_repository = classes()
.that().resideInAPackage("..repo..") .that().resideInAPackage("..repo..")
.should().haveSimpleNameEndingWith("Repository") .should().haveSimpleNameEndingWith("Repository")
.because("Repository interfaces are the persistence boundary; naming makes them explicit."); .because("Repository interfaces are the persistence boundary; naming makes them explicit.");
@ArchTest
static final ArchRule async_methods_should_name_a_pool = methods()
.that().areAnnotatedWith(Async.class)
.should(new ArchCondition<>("name a pool of BackgroundPools") {
@Override
public void check(JavaMethod method, ConditionEvents events) {
String pool = method.getAnnotationOfType(Async.class).value();
boolean named = Set.of(BackgroundPools.DEFAULT, BackgroundPools.SYNC_FOLLOW_UP).contains(pool);
events.add(new SimpleConditionEvent(method, named, method.getFullName() + " runs on '" + pool + "'"));
}
})
.because("A bare @Async runs on an unbounded executor (BackgroundPools).");
@ArchTest
static final ArchRule no_class_level_async = noClasses()
.should().beAnnotatedWith(Async.class)
.because("@Async on a class names no pool for its methods; put it on each method.");
// =================================================================================== // ===================================================================================
// 3. SHARED / CONFIGURATION RULES // 3. SHARED / CONFIGURATION RULES
// =================================================================================== // ===================================================================================
@@ -0,0 +1,72 @@
package core_service.configuration;
import core_service.shared.test.AbstractUnitTest;
import org.junit.jupiter.api.Test;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
class AsyncConfigTest extends AbstractUnitTest {
@Test
void fullQueue_DropsTheTaskWithoutFailingTheCaller() throws InterruptedException {
ThreadPoolTaskExecutor executor = new AsyncConfig().applicationTaskExecutor();
executor.initialize();
CountDownLatch release = new CountDownLatch(1);
CountDownLatch started = new CountDownLatch(AsyncConfig.THREADS);
AtomicBoolean droppedRan = new AtomicBoolean();
try {
for (int i = 0; i < AsyncConfig.THREADS; i++) {
executor.execute(() -> {
started.countDown();
awaitQuietly(release);
});
}
assertTrue(started.await(5, TimeUnit.SECONDS));
for (int i = 0; i < AsyncConfig.QUEUE; i++) {
executor.execute(() -> { });
}
assertDoesNotThrow(() -> executor.execute(() -> droppedRan.set(true)));
assertEquals(AsyncConfig.QUEUE, executor.getQueueSize());
} finally {
release.countDown();
executor.shutdown();
}
assertTrue(executor.getThreadPoolExecutor().awaitTermination(5, TimeUnit.SECONDS));
assertFalse(droppedRan.get());
}
@Test
void followUpPool_KeepsEveryTask() throws InterruptedException {
ThreadPoolTaskExecutor executor = new AsyncConfig().syncFollowUpExecutor();
executor.initialize();
int tasks = AsyncConfig.QUEUE * 3;
CountDownLatch done = new CountDownLatch(tasks);
try {
for (int i = 0; i < tasks; i++) {
executor.execute(done::countDown);
}
assertTrue(done.await(5, TimeUnit.SECONDS));
} finally {
executor.shutdown();
}
}
private static void awaitQuietly(CountDownLatch latch) {
try {
latch.await(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
@@ -1,6 +1,6 @@
package core_service.configuration.security; package core_service.configuration.security;
import core_service.shared.test.AbstractIntegrationTest; import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
@@ -16,7 +16,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile logs every request in, so only the CSRF header and CORS decide here. * The dev profile logs every request in, so only the CSRF header and CORS decide here.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class ClientHeaderFilterTest extends AbstractIntegrationTest { class ClientHeaderFilterTest extends AbstractMockedAdaptersTest {
private static final String FRONTEND = "https://tbp.andreas-dahm.eu"; private static final String FRONTEND = "https://tbp.andreas-dahm.eu";
@@ -48,6 +48,13 @@ class ClientHeaderFilterTest extends AbstractIntegrationTest {
.andExpect(header().string("Access-Control-Allow-Origin", FRONTEND)) .andExpect(header().string("Access-Control-Allow-Origin", FRONTEND))
.andExpect(header().string("Access-Control-Allow-Credentials", "true")); .andExpect(header().string("Access-Control-Allow-Credentials", "true"));
mockMvc.perform(options("/api/library/games/220/backlog")
.header("Origin", FRONTEND)
.header("Access-Control-Request-Method", "PATCH")
.header("Access-Control-Request-Headers", "x-tbp-client,content-type"))
.andExpect(status().isOk())
.andExpect(header().string("Access-Control-Allow-Origin", FRONTEND));
mockMvc.perform(options("/api/library/sync") mockMvc.perform(options("/api/library/sync")
.header("Origin", "https://evil.andreas-dahm.eu") .header("Origin", "https://evil.andreas-dahm.eu")
.header("Access-Control-Request-Method", "POST")) .header("Access-Control-Request-Method", "POST"))
@@ -118,8 +118,11 @@ class AccountCleanupRepositoryTest extends AbstractPersistenceTest {
// A new table with user rows must be in the purge list, or "Delete my account" leaves its rows behind. // A new table with user rows must be in the purge list, or "Delete my account" leaves its rows behind.
@Test @Test
void tables_CoverEveryTableWithAUserId() { void tables_CoverEveryTableWithAUserId() {
List<String> withUserId = jdbc.queryForList("select table_name from information_schema.columns " // Base tables only: a view like listed_user_game has no rows of its own.
+ "where table_schema = 'public' and column_name = 'user_id' order by table_name", String.class); List<String> withUserId = jdbc.queryForList("select c.table_name from information_schema.columns c "
+ "join information_schema.tables t on t.table_schema = c.table_schema and t.table_name = c.table_name "
+ "where c.table_schema = 'public' and c.column_name = 'user_id' and t.table_type = 'BASE TABLE' "
+ "order by c.table_name", String.class);
assertThat(AccountCleanupRepository.TABLES).containsExactlyInAnyOrderElementsOf(withUserId); assertThat(AccountCleanupRepository.TABLES).containsExactlyInAnyOrderElementsOf(withUserId);
} }
@@ -8,7 +8,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId; import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.GameImportWorker; import core_service.modules.library.service.GameImportWorker;
import core_service.shared.test.AbstractIntegrationTest; import core_service.shared.test.AbstractMockedAdaptersTest;
import com.jayway.jsonpath.JsonPath; import com.jayway.jsonpath.JsonPath;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
@@ -16,7 +16,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType; import org.springframework.http.MediaType;
import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions; import org.springframework.test.web.servlet.ResultActions;
@@ -48,7 +47,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class GameImportControllerTest extends AbstractIntegrationTest { class GameImportControllerTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023, private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023,
@@ -70,9 +69,6 @@ class GameImportControllerTest extends AbstractIntegrationTest {
@Autowired @Autowired
private JdbcTemplate jdbc; private JdbcTemplate jdbc;
@MockitoBean
private IgdbApiService igdbApiService;
@BeforeEach @BeforeEach
void seed() { void seed() {
jdbc.update("delete from game_alias"); jdbc.update("delete from game_alias");
@@ -104,7 +100,7 @@ class GameImportControllerTest extends AbstractIntegrationTest {
awaitStatus(id, "READY"); awaitStatus(id, "READY");
} }
// The matching and the save run in the background. // The matching and the save run on the worker's own lane threads, not on the test executor, so the test polls.
private void awaitStatus(long id, String expected) throws InterruptedException { private void awaitStatus(long id, String expected) throws InterruptedException {
for (int attempt = 0; attempt < 50; attempt++) { for (int attempt = 0; attempt < 50; attempt++) {
if (expected.equals(jdbc.queryForObject("select status from game_import where id = ?", String.class, id))) { if (expected.equals(jdbc.queryForObject("select status from game_import where id = ?", String.class, id))) {
@@ -146,6 +142,23 @@ class GameImportControllerTest extends AbstractIntegrationTest {
USER_ID, appid); USER_ID, appid);
} }
@Test
void get_ReadyImportStoredBeforeIgdbGame_ReadsItsCandidates() throws Exception {
long id = insertImport(USER_ID, "READY", "total = 1");
// The JSON the import wrote with IgdbApiService.Game, before the own record IgdbGame.
jdbc.update("insert into game_import_row (import_id, line_no, title, matched_at, exact, candidates) "
+ "values (?, 1, 'Zelda', now(), true, cast(? as jsonb))", id,
"[{\"igdbId\":119388,\"name\":\"Zelda: Tears of the Kingdom\",\"year\":2023,"
+ "\"imageUrl\":null,\"platforms\":[\"Switch\"],\"steamAppid\":null}]");
mockMvc.perform(get("/api/library/imports/" + id))
.andExpect(status().isOk())
.andExpect(jsonPath("$.rows[0].candidates[0].igdbId").value(119388))
.andExpect(jsonPath("$.rows[0].candidates[0].name").value("Zelda: Tears of the Kingdom"))
.andExpect(jsonPath("$.rows[0].candidates[0].year").value(2023))
.andExpect(jsonPath("$.rows[0].candidates[0].platforms[0]").value("Switch"));
}
@Test @Test
void create_MatchesInTheBackground_SameNameFirstAndMarksGamesInTheLibrary() throws Exception { void create_MatchesInTheBackground_SameNameFirstAndMarksGamesInTheLibrary() throws Exception {
jdbc.update("insert into steam_game (appid, name) values (1245620, 'ELDEN RING') on conflict do nothing"); jdbc.update("insert into steam_game (appid, name) values (1245620, 'ELDEN RING') on conflict do nothing");
@@ -10,13 +10,11 @@ import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncWorker; import core_service.modules.library.service.LibrarySyncWorker;
import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse; import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant; import java.time.Instant;
@@ -27,7 +25,6 @@ import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.nullValue; import static org.hamcrest.Matchers.nullValue;
import static org.mockito.ArgumentMatchers.anyLong; import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString; import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.timeout;
import static org.mockito.Mockito.verify; import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when; import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
@@ -38,7 +35,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class LibraryControllerAchievementsTest extends AbstractIntegrationTest { class LibraryControllerAchievementsTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final String REPAIRED_COVER = "https://shared.akamai.steamstatic.com/store_item_assets/steam/apps/1/x/header.jpg"; private static final String REPAIRED_COVER = "https://shared.akamai.steamstatic.com/store_item_assets/steam/apps/1/x/header.jpg";
@@ -46,9 +43,6 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
@Autowired @Autowired
private MockMvc mockMvc; private MockMvc mockMvc;
@MockitoBean
private SteamApiService steamApiService;
@Autowired @Autowired
private SteamGameRepository steamGameRepository; private SteamGameRepository steamGameRepository;
@@ -107,9 +101,7 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
game(70L, 8, null, true); // unlocks never fetched game(70L, 8, null, true); // unlocks never fetched
game(80L, 10, 12, true); // schema lost achievements, still perfect game(80L, 10, 12, true); // schema lost achievements, still perfect
game(90L, 6, 0, true); // not started game(90L, 6, 0, true); // not started
SteamGame repaired = steamGameRepository.findById(20L).orElseThrow(); steamGameRepository.setImageUrl(20L, REPAIRED_COVER);
repaired.setImageUrl(REPAIRED_COVER);
steamGameRepository.save(repaired);
steamGameRepository.save(new SteamGame(95L, "Game 95", null, Instant.now())); steamGameRepository.save(new SteamGame(95L, "Game 95", null, Instant.now()));
UserGame removed = new UserGame(USER_ID, 95L, 60, Instant.now(), Instant.now()); UserGame removed = new UserGame(USER_ID, 95L, 60, Instant.now(), Instant.now());
removed.setRemovedAt(Instant.now()); removed.setRemovedAt(Instant.now());
@@ -163,6 +155,6 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
librarySyncWorker.runSyncNow(USER_ID, "76561198000000001"); librarySyncWorker.runSyncNow(USER_ID, "76561198000000001");
verify(steamApiService, timeout(5_000)).getSchemaForGame(620L); verify(steamApiService).getSchemaForGame(620L);
} }
} }
@@ -53,8 +53,7 @@ class LibraryControllerActivityTest extends AbstractIntegrationTest {
void seed() { void seed() {
userGameRepository.deleteAll(); userGameRepository.deleteAll();
jdbc.update("delete from user_game_playtime_day"); jdbc.update("delete from user_game_playtime_day");
SteamGame repairedPortal = new SteamGame(620L, "Portal 2", null, Instant.now()); SteamGame repairedPortal = new SteamGame(620L, "Portal 2", null, REPAIRED_COVER, Instant.now());
repairedPortal.setImageUrl(REPAIRED_COVER);
steamGameRepository.saveAll(List.of( steamGameRepository.saveAll(List.of(
new SteamGame(220L, "Half-Life 2", null, Instant.now()), new SteamGame(220L, "Half-Life 2", null, Instant.now()),
repairedPortal, repairedPortal,
@@ -17,11 +17,15 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType; import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;
import java.time.Instant; import java.time.Instant;
import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.patch;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.put; import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.put;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
@@ -138,6 +142,87 @@ class LibraryControllerBacklogTest extends AbstractIntegrationTest {
.andExpect(status().isNotFound()); .andExpect(status().isNotFound());
} }
private ResultActions patchBacklog(long appid, String body) throws Exception {
return mockMvc.perform(patch("/api/library/games/" + appid + "/backlog").header(ClientHeaderFilter.CLIENT_HEADER, "test")
.contentType(MediaType.APPLICATION_JSON).content(body));
}
private void storeAllFields() {
UserGame game = stored();
game.setRating(4);
game.setNotes("old notes");
game.setPriority(GamePriority.HIGH);
userGameRepository.save(game);
}
@Test
void patchBacklog_OnlyNotes_KeepsTheOtherFieldsAndTheAutomaticStatus() throws Exception {
storeAllFields();
patchBacklog(APPID, "{\"notes\":\"new notes\"}")
.andExpect(status().isOk())
.andExpect(jsonPath("$.notes").value("new notes"))
.andExpect(jsonPath("$.rating").value(4));
UserGame game = stored();
assertEquals("new notes", game.getNotes());
assertEquals(4, game.getRating());
assertEquals(GamePriority.HIGH, game.getPriority());
assertEquals(GameStatus.BACKLOG, game.getStatus());
assertFalse(game.isStatusSetByUser());
}
@Test
void patchBacklog_Status_IsSetByTheUser() throws Exception {
patchBacklog(APPID, "{\"status\":\"PLAYING\"}").andExpect(status().isOk());
assertEquals(GameStatus.PLAYING, stored().getStatus());
assertTrue(stored().isStatusSetByUser());
}
@Test
void patchBacklog_NullClearsAnOptionalFieldAndBlankNotesAreNull() throws Exception {
storeAllFields();
patchBacklog(APPID, "{\"rating\":null,\"priority\":null,\"notes\":\" \"}").andExpect(status().isOk());
UserGame game = stored();
assertNull(game.getRating());
assertNull(game.getPriority());
assertNull(game.getNotes());
}
@ParameterizedTest
@ValueSource(strings = {
"{\"status\":null}",
"{\"status\":\"WISHLIST\"}",
"{\"rating\":0}",
"{\"rating\":6}",
"{\"rating\":4.7}",
"{\"priority\":\"URGENT\"}"})
void patchBacklog_InvalidBody_Is400AndStoresNothing(String body) throws Exception {
storeAllFields();
patchBacklog(APPID, body).andExpect(status().isBadRequest());
UserGame game = stored();
assertEquals(GameStatus.BACKLOG, game.getStatus());
assertEquals(4, game.getRating());
assertEquals(GamePriority.HIGH, game.getPriority());
}
@Test
void patchBacklog_NotesTooLong_Is400() throws Exception {
patchBacklog(APPID, "{\"notes\":\"" + "x".repeat(2001) + "\"}").andExpect(status().isBadRequest());
}
@Test
void patchBacklog_GameNotOwned_Is404() throws Exception {
patchBacklog(999L, "{\"notes\":\"x\"}")
.andExpect(status().isNotFound())
.andExpect(jsonPath("$.code").value("GAME_NOT_FOUND"));
}
@Test @Test
void updateStatus_KeepsRatingNotesAndPriority() throws Exception { void updateStatus_KeepsRatingNotesAndPriority() throws Exception {
UserGame game = stored(); UserGame game = stored();
@@ -7,7 +7,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId; import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.shared.test.AbstractIntegrationTest; import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.ParameterizedTest;
@@ -34,7 +34,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class LibraryControllerHiddenTest extends AbstractIntegrationTest { class LibraryControllerHiddenTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
@@ -11,19 +11,14 @@ import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncService; import core_service.shared.test.AbstractMockedLibraryServicesTest;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant; import java.time.Instant;
import java.util.List;
import static org.hamcrest.Matchers.containsInAnyOrder; import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.hamcrest.Matchers.hasSize; import static org.hamcrest.Matchers.hasSize;
@@ -37,7 +32,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class LibraryControllerPickerTest extends AbstractIntegrationTest { class LibraryControllerPickerTest extends AbstractMockedLibraryServicesTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
@@ -56,15 +51,6 @@ class LibraryControllerPickerTest extends AbstractIntegrationTest {
@Autowired @Autowired
private GameMoodsRepository gameMoodsRepository; private GameMoodsRepository gameMoodsRepository;
@MockitoBean
private LibrarySyncService syncService;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@Autowired @Autowired
private GameSpeedrunRepository gameSpeedrunRepository; private GameSpeedrunRepository gameSpeedrunRepository;
@@ -7,10 +7,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncService; import core_service.shared.test.AbstractMockedLibraryServicesTest;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
@@ -20,7 +17,6 @@ import org.springframework.boot.test.system.CapturedOutput;
import org.springframework.boot.test.system.OutputCaptureExtension; import org.springframework.boot.test.system.OutputCaptureExtension;
import org.springframework.http.converter.HttpMessageNotWritableException; import org.springframework.http.converter.HttpMessageNotWritableException;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import java.sql.SQLException; import java.sql.SQLException;
@@ -43,7 +39,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class LibraryControllerStatsTest extends AbstractIntegrationTest { class LibraryControllerStatsTest extends AbstractMockedLibraryServicesTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
@@ -59,16 +55,6 @@ class LibraryControllerStatsTest extends AbstractIntegrationTest {
@Autowired @Autowired
private GameTimeToBeatRepository gameTimeToBeatRepository; private GameTimeToBeatRepository gameTimeToBeatRepository;
// The same mocks as LibraryControllerPickerTest, so both share one Spring context.
@MockitoBean
private LibrarySyncService syncService;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@BeforeEach @BeforeEach
void seed() { void seed() {
userGameRepository.deleteAll(); userGameRepository.deleteAll();
@@ -1,12 +1,10 @@
package core_service.modules.library.api; package core_service.modules.library.api;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerSummaryResponse; import core_service.modules.steam_adapter.api.dto.response.SteamPlayerSummaryResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import java.util.Optional; import java.util.Optional;
@@ -21,14 +19,11 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1 with Steam ID 76561198000000001. * The dev profile authenticates every request as user 1 with Steam ID 76561198000000001.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class LibraryControllerSteamProfileTest extends AbstractIntegrationTest { class LibraryControllerSteamProfileTest extends AbstractMockedAdaptersTest {
@Autowired @Autowired
private MockMvc mockMvc; private MockMvc mockMvc;
@MockitoBean
private SteamApiService steamApiService;
@Test @Test
void steamProfile_SendsNameAndAvatar() throws Exception { void steamProfile_SendsNameAndAvatar() throws Exception {
when(steamApiService.getPlayerSummary("76561198000000001")).thenReturn(Optional.of(new SteamPlayerSummaryResponse.Player( when(steamApiService.getPlayerSummary("76561198000000001")).thenReturn(Optional.of(new SteamPlayerSummaryResponse.Player(
@@ -4,14 +4,13 @@ import core_service.configuration.security.ClientHeaderFilter;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameSource; import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus; import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.GameTimeToBeat;
import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.SteamGame;
import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId; import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.shared.test.AbstractIntegrationTest; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.ContractTokens; import core_service.shared.test.ContractTokens;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
@@ -19,7 +18,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType; import org.springframework.http.MediaType;
import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant; import java.time.Instant;
@@ -48,7 +46,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1. * The dev profile authenticates every request as user 1.
*/ */
@AutoConfigureMockMvc @AutoConfigureMockMvc
class ManualGameControllerTest extends AbstractIntegrationTest { class ManualGameControllerTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023, private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023,
@@ -71,9 +69,6 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
@Autowired @Autowired
private JdbcTemplate jdbc; private JdbcTemplate jdbc;
@MockitoBean
private IgdbApiService igdbApiService;
@BeforeEach @BeforeEach
void seed() { void seed() {
jdbc.update("delete from game_alias"); jdbc.update("delete from game_alias");
@@ -115,19 +110,7 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
SteamGame catalog = steamGameRepository.findById(-119388L).orElseThrow(); SteamGame catalog = steamGameRepository.findById(-119388L).orElseThrow();
assertEquals("Zelda: Tears of the Kingdom", catalog.getName()); assertEquals("Zelda: Tears of the Kingdom", catalog.getName());
assertEquals(ZELDA.imageUrl(), catalog.getImageUrl()); assertEquals(ZELDA.imageUrl(), catalog.getImageUrl());
assertEquals(3600, timeToBeatStored(-119388L).getNormallyMinutes()); assertEquals(3600, gameTimeToBeatRepository.findById(-119388L).orElseThrow().getNormallyMinutes());
}
// The add asks IGDB for the time to beat in the background.
private GameTimeToBeat timeToBeatStored(long appid) throws InterruptedException {
for (int attempt = 0; attempt < 50; attempt++) {
Optional<GameTimeToBeat> found = gameTimeToBeatRepository.findById(appid);
if (found.isPresent()) {
return found.get();
}
Thread.sleep(100);
}
throw new AssertionError("No time to beat stored for appid " + appid);
} }
@Test @Test
@@ -333,7 +316,7 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
@Test @Test
void gameAddedByHand_ShowsUpInTheLibraryAndItsDetail() throws Exception { void gameAddedByHand_ShowsUpInTheLibraryAndItsDetail() throws Exception {
add(119388, "Switch"); add(119388, "Switch");
timeToBeatStored(-119388L); assertTrue(gameTimeToBeatRepository.findById(-119388L).isPresent());
mockMvc.perform(get("/api/library/games").param("q", "zelda")) mockMvc.perform(get("/api/library/games").param("q", "zelda"))
.andExpect(status().isOk()) .andExpect(status().isOk())
@@ -0,0 +1,23 @@
package core_service.modules.library.repo;
import core_service.shared.test.AbstractPersistenceTest;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import static org.junit.jupiter.api.Assertions.assertEquals;
class ListedUserGameViewTest extends AbstractPersistenceTest {
private static final String COLUMNS = "select column_name from information_schema.columns "
+ "where table_schema = 'public' and table_name = ? order by ordinal_position";
@Autowired
private JdbcTemplate jdbc;
@Test
void view_HasEveryColumnOfUserGame() {
assertEquals(jdbc.queryForList(COLUMNS, String.class, "user_game"),
jdbc.queryForList(COLUMNS, String.class, "listed_user_game"));
}
}
@@ -124,6 +124,29 @@ class UserGameRepositoryTest extends AbstractPersistenceTest {
assertEquals(Set.of(220L, 380L), Set.copyOf(userGameRepository.findBacklogAppids(USER_ID))); assertEquals(Set.of(220L, 380L), Set.copyOf(userGameRepository.findBacklogAppids(USER_ID)));
} }
@Test
void findListedSteamGames_LeavesOutHiddenRemovedAndHandAddedGames() {
userGameRepository.findById(new UserGameId(USER_ID, 380L)).orElseThrow().setHiddenAt(Instant.now());
userGameRepository.findById(new UserGameId(USER_ID, 700L)).orElseThrow().setRemovedAt(Instant.now());
steamGameRepository.save(new SteamGame(999L, "Elden Ring", null, Instant.now()));
userGameRepository.save(UserGame.manual(USER_ID, 999L, "PS5", GameStatus.PLAYING, 60, Instant.now()));
userGameRepository.flush();
assertEquals(Set.of(220L, 620L),
Set.copyOf(userGameRepository.findListedSteamGames(USER_ID).stream().map(UserGame::getAppid).toList()));
}
@Test
void listedUserGameView_LeavesOutHiddenAndRemovedGames() {
userGameRepository.findById(new UserGameId(USER_ID, 380L)).orElseThrow().setHiddenAt(Instant.now());
userGameRepository.findById(new UserGameId(USER_ID, 700L)).orElseThrow().setRemovedAt(Instant.now());
userGameRepository.flush();
assertEquals(Set.of(220L, 620L), Set.copyOf(entityManager.createNativeQuery(
"select appid from listed_user_game where user_id = :userId", Long.class)
.setParameter("userId", USER_ID).getResultList()));
}
@Test @Test
void findAchievementProgress_LeavesOutGamesAddedByHand() { void findAchievementProgress_LeavesOutGamesAddedByHand() {
steamGameRepository.save(new SteamGame(999L, "Elden Ring", null, Instant.now())); steamGameRepository.save(new SteamGame(999L, "Elden Ring", null, Instant.now()));
@@ -3,13 +3,11 @@ package core_service.modules.library.service;
import core_service.modules.library.repo.UserGameAchievementMetaRepository; import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameAchievementRepository; import core_service.modules.library.repo.UserGameAchievementRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse; import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.transaction.PlatformTransactionManager; import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate; import org.springframework.transaction.support.TransactionTemplate;
@@ -28,7 +26,7 @@ import static org.mockito.Mockito.when;
* The game detail page refreshes inside its own @Transactional while the background sync refreshes the same * The game detail page refreshes inside its own @Transactional while the background sync refreshes the same
* (user, appid) in a transaction of its own, e.g. right after the user pressed "sync" and opened the most played game. * (user, appid) in a transaction of its own, e.g. right after the user pressed "sync" and opened the most played game.
*/ */
class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest { class AchievementSyncServiceConcurrencyTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001"; private static final String STEAM_ID = "76561198000000001";
@@ -45,9 +43,6 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
@Autowired @Autowired
private PlatformTransactionManager transactionManager; private PlatformTransactionManager transactionManager;
@MockitoBean
private SteamApiService steamApiService;
@BeforeEach @BeforeEach
@AfterEach @AfterEach
void clean() { void clean() {
@@ -61,20 +56,20 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
new SteamPlayerAchievementsResponse.PlayerStats(STEAM_ID, "Portal 2", new SteamPlayerAchievementsResponse.PlayerStats(STEAM_ID, "Portal 2",
List.of(new SteamPlayerAchievementsResponse.Achievement("A", 1, 1_700_000_000L)), true, null))); List.of(new SteamPlayerAchievementsResponse.Achievement("A", 1, 1_700_000_000L)), true, null)));
CountDownLatch firstWrote = new CountDownLatch(1); CountDownLatch firstWrote = new CountDownLatch(1);
CountDownLatch secondStarted = new CountDownLatch(1); CountDownLatch secondDone = new CountDownLatch(1);
CompletableFuture<Void> detailPage = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager) CompletableFuture<Void> detailPage = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager)
.executeWithoutResult(tx -> { .executeWithoutResult(tx -> {
achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1)); achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1));
firstWrote.countDown(); firstWrote.countDown();
await(secondStarted); // The page's transaction stays open until the background run wrote the same rows; a run that
// Keep the page's transaction open while the background run writes the same rows. // waited for this transaction would time out the latch.
sleep(); await(secondDone);
})); }));
CompletableFuture<Void> backgroundRun = CompletableFuture.runAsync(() -> { CompletableFuture<Void> backgroundRun = CompletableFuture.runAsync(() -> {
await(firstWrote); await(firstWrote);
secondStarted.countDown();
achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1)); achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1));
secondDone.countDown();
}); });
assertDoesNotThrow(() -> CompletableFuture.allOf(detailPage, backgroundRun).get(30, TimeUnit.SECONDS)); assertDoesNotThrow(() -> CompletableFuture.allOf(detailPage, backgroundRun).get(30, TimeUnit.SECONDS));
@@ -90,11 +85,4 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
} }
} }
private static void sleep() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
} }
@@ -96,7 +96,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
} }
private void library(List<UserGame> games, List<GameAchievementSchemaMeta> schemas, List<UserGameAchievementMeta> metas) { private void library(List<UserGame> games, List<GameAchievementSchemaMeta> schemas, List<UserGameAchievementMeta> metas) {
when(userGameRepository.findByUserIdAndRemovedAtIsNull(USER_ID)).thenReturn(games); when(userGameRepository.findListedSteamGames(USER_ID)).thenReturn(games);
when(schemaMetaRepository.findAllById(any())).thenReturn(schemas); when(schemaMetaRepository.findAllById(any())).thenReturn(schemas);
when(userAchievementMetaRepository.findByUserId(USER_ID)).thenReturn(metas); when(userAchievementMetaRepository.findByUserId(USER_ID)).thenReturn(metas);
} }
@@ -117,28 +117,6 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
verify(userAchievementMetaRepository, never()).save(any()); verify(userAchievementMetaRepository, never()).save(any());
} }
@Test
void syncLibrary_GameAddedByHand_AsksSteamNothing() {
UserGame manual = UserGame.manual(USER_ID, 620L, "PS5", core_service.modules.library.domain.GameStatus.PLAYING,
150, Instant.now());
library(List.of(manual), List.of(schema(620L, 10)), List.of());
service.syncLibrary(USER_ID, STEAM_ID);
verifyNoInteractions(steamApiService);
}
@Test
void syncLibrary_HiddenGame_AsksSteamNothing() {
UserGame hidden = game(620L, 120);
hidden.setHiddenAt(Instant.now());
library(List.of(hidden), List.of(), List.of());
service.syncLibrary(USER_ID, STEAM_ID);
verifyNoInteractions(steamApiService);
}
@Test @Test
void syncLibrary_ChangedPlaytime_StoresUnlocksAndThePlaytime() { void syncLibrary_ChangedPlaytime_StoresUnlocksAndThePlaytime() {
library(List.of(game(620L, 150)), List.of(schema(620L, 10)), List.of(unlocks(620L, 3, 120))); library(List.of(game(620L, 150)), List.of(schema(620L, 10)), List.of(unlocks(620L, 3, 120)));
@@ -358,7 +336,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
@Test @Test
void syncLibrary_CallWhileRunning_RunsOnceMoreAfterwards() throws Exception { void syncLibrary_CallWhileRunning_RunsOnceMoreAfterwards() throws Exception {
when(userGameRepository.findByUserIdAndRemovedAtIsNull(USER_ID)).thenReturn(List.of(game(620L, 60))); when(userGameRepository.findListedSteamGames(USER_ID)).thenReturn(List.of(game(620L, 60)));
when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10))); when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10)));
// The second run sees the row the first run stored. // The second run sees the row the first run stored.
when(userAchievementMetaRepository.findByUserId(USER_ID)) when(userAchievementMetaRepository.findByUserId(USER_ID))
@@ -378,7 +356,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
first.join(5_000); first.join(5_000);
verify(steamApiService, times(1)).getPlayerAchievements(STEAM_ID, 620L); verify(steamApiService, times(1)).getPlayerAchievements(STEAM_ID, 620L);
verify(userGameRepository, times(2)).findByUserIdAndRemovedAtIsNull(USER_ID); verify(userGameRepository, times(2)).findListedSteamGames(USER_ID);
} }
@Test @Test
@@ -399,8 +377,8 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
AchievementSyncService paced = new AchievementSyncService(userGameRepository, schemaRepository, schemaMetaRepository, AchievementSyncService paced = new AchievementSyncService(userGameRepository, schemaRepository, schemaMetaRepository,
userAchievementRepository, userAchievementMetaRepository, steamApiService, userAchievementRepository, userAchievementMetaRepository, steamApiService,
mock(PlatformTransactionManager.class), 30, pauseMillis); mock(PlatformTransactionManager.class), 30, pauseMillis);
when(userGameRepository.findByUserIdAndRemovedAtIsNull(1L)).thenReturn(List.of(new UserGame(1L, 620L, 60, Instant.now(), Instant.now()))); when(userGameRepository.findListedSteamGames(1L)).thenReturn(List.of(new UserGame(1L, 620L, 60, Instant.now(), Instant.now())));
when(userGameRepository.findByUserIdAndRemovedAtIsNull(2L)).thenReturn(List.of(new UserGame(2L, 620L, 60, Instant.now(), Instant.now()))); when(userGameRepository.findListedSteamGames(2L)).thenReturn(List.of(new UserGame(2L, 620L, 60, Instant.now(), Instant.now())));
when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10))); when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10)));
when(userAchievementMetaRepository.findByUserId(anyLong())).thenReturn(List.of()); when(userAchievementMetaRepository.findByUserId(anyLong())).thenReturn(List.of());
List<Long> steamCallsAtNanos = Collections.synchronizedList(new ArrayList<>()); List<Long> steamCallsAtNanos = Collections.synchronizedList(new ArrayList<>());
@@ -55,12 +55,12 @@ class GameCorrectionServiceConcurrencyTest extends AbstractIntegrationTest {
void twoAuditorsCorrectTheSameFieldForTheFirstTime_BothSucceed() { void twoAuditorsCorrectTheSameFieldForTheFirstTime_BothSucceed() {
CountDownLatch firstCorrected = new CountDownLatch(1); CountDownLatch firstCorrected = new CountDownLatch(1);
// The first commits a moment after the second has started, so both see no row before either commits. // The first commits only when the second waits for its row, so both saw no row before either commits.
CompletableFuture<Void> first = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager) CompletableFuture<Void> first = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager)
.executeWithoutResult(tx -> { .executeWithoutResult(tx -> {
gameCorrectionService.correct(GAME, "name", "Portal Two", 1L); gameCorrectionService.correct(GAME, "name", "Portal Two", 1L);
firstCorrected.countDown(); firstCorrected.countDown();
pause(); awaitLockWaiter(jdbcTemplate);
})); }));
CompletableFuture<Void> second = CompletableFuture.runAsync(() -> { CompletableFuture<Void> second = CompletableFuture.runAsync(() -> {
await(firstCorrected); await(firstCorrected);
@@ -86,14 +86,6 @@ class GameCorrectionServiceConcurrencyTest extends AbstractIntegrationTest {
assertThat(steamGameRepository.findById(GAME).orElseThrow().getImageUrl()).isEqualTo(IMAGE); assertThat(steamGameRepository.findById(GAME).orElseThrow().getImageUrl()).isEqualTo(IMAGE);
} }
private static void pause() {
try {
Thread.sleep(300);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private static void await(CountDownLatch latch) { private static void await(CountDownLatch latch) {
try { try {
if (!latch.await(10, TimeUnit.SECONDS)) throw new IllegalStateException("latch timed out"); if (!latch.await(10, TimeUnit.SECONDS)) throw new IllegalStateException("latch timed out");
@@ -11,13 +11,11 @@ import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameAchievementMetaRepository; import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse; import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.TestPropertySource; import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import java.time.Instant; import java.time.Instant;
import java.util.List; import java.util.List;
@@ -32,7 +30,7 @@ import static org.mockito.Mockito.when;
@TestPropertySource(properties = { @TestPropertySource(properties = {
"spring.datasource.hikari.maximum-pool-size=1", "spring.datasource.hikari.maximum-pool-size=1",
"spring.datasource.hikari.connection-timeout=1000"}) "spring.datasource.hikari.connection-timeout=1000"})
class GameDetailServicePoolTest extends AbstractIntegrationTest { class GameDetailServicePoolTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001"; private static final String STEAM_ID = "76561198000000001";
@@ -52,9 +50,6 @@ class GameDetailServicePoolTest extends AbstractIntegrationTest {
@Autowired @Autowired
private UserGameAchievementMetaRepository userAchievementMetaRepository; private UserGameAchievementMetaRepository userAchievementMetaRepository;
@MockitoBean
private SteamApiService steamApiService;
@Autowired @Autowired
private GameAchievementSchemaRepository schemaRepository; private GameAchievementSchemaRepository schemaRepository;
@@ -12,23 +12,25 @@ import core_service.modules.library.repo.GameAchievementSchemaMetaRepository;
import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.GameAchievementSchemaRepository; import core_service.modules.library.repo.GameAchievementSchemaRepository;
import core_service.modules.library.repo.GameSpeedrunRepository; import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.GameMoodsRepository;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.UserGameAchievementMetaRepository; import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameAchievementRepository; import core_service.modules.library.repo.UserGameAchievementRepository;
import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse; import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse; import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.web.client.ResourceAccessException; import org.springframework.web.client.ResourceAccessException;
import java.time.Duration; import java.time.Duration;
import java.time.Instant; import java.time.Instant;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
import java.util.Set;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue; import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -37,7 +39,7 @@ import static org.mockito.Mockito.when;
/** /**
* The detail page refreshes stale rows in a transaction of its own, then reads them in its own read transaction. * The detail page refreshes stale rows in a transaction of its own, then reads them in its own read transaction.
*/ */
class GameDetailServiceTest extends AbstractIntegrationTest { class GameDetailServiceTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L; private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001"; private static final String STEAM_ID = "76561198000000001";
@@ -63,17 +65,11 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
@Autowired @Autowired
private UserGameAchievementMetaRepository userAchievementMetaRepository; private UserGameAchievementMetaRepository userAchievementMetaRepository;
@MockitoBean @Autowired
private SteamApiService steamApiService; private GameTimeToBeatRepository gameTimeToBeatRepository;
@MockitoBean @Autowired
private TimeToBeatService timeToBeatService; private GameMoodsRepository gameMoodsRepository;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@Autowired @Autowired
private GameSpeedrunRepository gameSpeedrunRepository; private GameSpeedrunRepository gameSpeedrunRepository;
@@ -86,6 +82,8 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
userAchievementRepository.deleteAll(); userAchievementRepository.deleteAll();
userAchievementMetaRepository.deleteAll(); userAchievementMetaRepository.deleteAll();
gameSpeedrunRepository.deleteAll(); gameSpeedrunRepository.deleteAll();
gameTimeToBeatRepository.deleteAll();
gameMoodsRepository.deleteAll();
steamGameRepository.save(new SteamGame(620L, "Portal 2", null, Instant.now())); steamGameRepository.save(new SteamGame(620L, "Portal 2", null, Instant.now()));
userGameRepository.save(new UserGame(USER_ID, 620L, 60, Instant.now(), Instant.now())); userGameRepository.save(new UserGame(USER_ID, 620L, 60, Instant.now(), Instant.now()));
} }
@@ -115,7 +113,7 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, STEAM_ID, 620L).orElseThrow(); GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, STEAM_ID, 620L).orElseThrow();
org.mockito.Mockito.verifyNoInteractions(steamApiService, speedrunService, timeToBeatService, moodService); org.mockito.Mockito.verifyNoInteractions(steamApiService, igdbApiService, speedrunApiService);
assertEquals(true, detail.hidden()); assertEquals(true, detail.hidden());
assertEquals(1, detail.achievementsTotal()); assertEquals(1, detail.achievementsTotal());
} }
@@ -124,20 +122,26 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
void getGameDetail_GameAddedByHand_AsksForTheTimeToBeatAndMoodsAgain() { void getGameDetail_GameAddedByHand_AsksForTheTimeToBeatAndMoodsAgain() {
userGameRepository.save(UserGame.manual(USER_ID, 620L, "PS5", GameStatus.PLAYING, 60, Instant.now())); userGameRepository.save(UserGame.manual(USER_ID, 620L, "PS5", GameStatus.PLAYING, 60, Instant.now()));
when(igdbApiService.isEnabled()).thenReturn(true);
when(igdbApiService.getTimeToBeat(Set.of(620L))).thenReturn(Optional.of(Map.of()));
when(igdbApiService.getGameTags(Set.of(620L))).thenReturn(Optional.of(Map.of()));
gameDetailService.getGameDetail(USER_ID, null, 620L); gameDetailService.getGameDetail(USER_ID, null, 620L);
org.mockito.Mockito.verify(timeToBeatService).refreshInBackground(List.of(620L)); assertTrue(gameTimeToBeatRepository.findById(620L).isPresent());
org.mockito.Mockito.verify(moodService).refreshInBackground(List.of(620L)); assertTrue(gameMoodsRepository.findById(620L).isPresent());
} }
@Test @Test
void getGameDetail_ShowsTheSpeedrunLinkAndAsksForIt() { void getGameDetail_ShowsTheSpeedrunLinkAndAsksForIt() {
gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2", Instant.now()); gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2",
Instant.now().minus(Duration.ofDays(31)));
when(speedrunApiService.isEnabled()).thenReturn(true);
GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, null, 620L).orElseThrow(); GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, null, 620L).orElseThrow();
assertEquals("https://www.speedrun.com/portal_2", detail.speedrunUrl()); assertEquals("https://www.speedrun.com/portal_2", detail.speedrunUrl());
org.mockito.Mockito.verify(speedrunService).refreshInBackground(List.of(620L)); org.mockito.Mockito.verify(speedrunApiService).findGames("Portal 2");
} }
@Test @Test
@@ -1,6 +1,6 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameImport; import core_service.modules.library.domain.GameImport;
import core_service.modules.library.repo.GameImportRepository; import core_service.modules.library.repo.GameImportRepository;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
@@ -20,8 +20,8 @@ import static org.mockito.Mockito.when;
class GameImportWorkerTest { class GameImportWorkerTest {
private static IgdbApiService.Game game(long igdbId, String name, String... platforms) { private static IgdbGame game(long igdbId, String name, String... platforms) {
return new IgdbApiService.Game(igdbId, name, null, null, List.of(platforms), null); return new IgdbGame(igdbId, name, null, null, List.of(platforms), null);
} }
private static GameImport.Line line(String title, String platform) { private static GameImport.Line line(String title, String platform) {
@@ -37,12 +37,12 @@ class GameImportWorkerTest {
@Test @Test
void rank_SameNameFirst_ThenTheLinesPlatform_ThenIgdbsOrder_AtMostFive() { void rank_SameNameFirst_ThenTheLinesPlatform_ThenIgdbsOrder_AtMostFive() {
List<IgdbApiService.Game> found = List.of(game(1, "Doom Eternal", "PC"), game(2, "DOOM", "PC"), List<IgdbGame> found = List.of(game(1, "Doom Eternal", "PC"), game(2, "DOOM", "PC"),
game(3, "Doom", "Switch"), game(4, "Doom 3"), game(5, "Doom 64"), game(6, "Doom II")); game(3, "Doom", "Switch"), game(4, "Doom 3"), game(5, "Doom 64"), game(6, "Doom II"));
assertThat(GameImportWorker.rank(line("Doom", "switch"), found)).extracting(IgdbApiService.Game::igdbId) assertThat(GameImportWorker.rank(line("Doom", "switch"), found)).extracting(IgdbGame::igdbId)
.containsExactly(3L, 2L, 1L, 4L, 5L); .containsExactly(3L, 2L, 1L, 4L, 5L);
assertThat(GameImportWorker.rank(line("Doom", null), found)).extracting(IgdbApiService.Game::igdbId) assertThat(GameImportWorker.rank(line("Doom", null), found)).extracting(IgdbGame::igdbId)
.startsWith(2L, 3L); .startsWith(2L, 3L);
assertThat(GameImportWorker.rank(line("x", null), IntStream.range(0, 9) assertThat(GameImportWorker.rank(line("x", null), IntStream.range(0, 9)
.mapToObj(i -> game(i, "Game " + i)).toList())).hasSize(GameImportWorker.MAX_CANDIDATES); .mapToObj(i -> game(i, "Game " + i)).toList())).hasSize(GameImportWorker.MAX_CANDIDATES);
@@ -56,7 +56,7 @@ class GameImportWorkerTest {
.thenReturn(Optional.of(7L), Optional.empty()); .thenReturn(Optional.of(7L), Optional.empty());
when(repository.findStalled(any())).thenReturn(Map.of()); when(repository.findStalled(any())).thenReturn(Map.of());
GameImportWorker worker = new GameImportWorker(repository, mock(ManualGameService.class), GameImportWorker worker = new GameImportWorker(repository, mock(ManualGameService.class),
mock(IgdbApiService.class), mock(TimeToBeatService.class), mock(MoodService.class), 5); mock(TimeToBeatService.class), mock(MoodService.class), 5);
try { try {
worker.wakeMatching(); worker.wakeMatching();
verify(repository, timeout(2000)).findNext(GameImportWorker.MATCHING_QUEUE); verify(repository, timeout(2000)).findNext(GameImportWorker.MATCHING_QUEUE);
@@ -136,8 +136,7 @@ class ImageRepairServiceTest extends AbstractUnitTest {
@Test @Test
void repairNightly_ChecksTheStoredRepairFirst() { void repairNightly_ChecksTheStoredRepairFirst() {
due(row(620L, ImageKind.COVER, Instant.now())); due(row(620L, ImageKind.COVER, Instant.now()));
SteamGame game = new SteamGame(620L, "Portal 2", null, Instant.now()); SteamGame game = new SteamGame(620L, "Portal 2", null, HASHED_620, Instant.now());
game.setImageUrl(HASHED_620);
when(steamGameRepository.findAllById(List.of(620L))).thenReturn(List.of(game)); when(steamGameRepository.findAllById(List.of(620L))).thenReturn(List.of(game));
when(steamCdnService.isAvailable(HASHED_620)).thenReturn(true); when(steamCdnService.isAvailable(HASHED_620)).thenReturn(true);
@@ -1,16 +1,16 @@
package core_service.modules.library.service; package core_service.modules.library.service;
import core_service.modules.library.domain.UserSteamSync; import core_service.modules.library.domain.UserSteamSync;
import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.modules.library.repo.UserSteamSyncRepository; import core_service.modules.library.repo.UserSteamSyncRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse; import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse;
import core_service.modules.steam_adapter.service.SteamApiService; import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.AbstractIntegrationTest;
import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
@@ -27,7 +27,7 @@ import static org.mockito.Mockito.when;
/** /**
* The claim and the finish of a sync are committed on their own, so other requests see RUNNING and FAILED. * The claim and the finish of a sync are committed on their own, so other requests see RUNNING and FAILED.
*/ */
class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest { class LibrarySyncWorkerConcurrencyTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 4711L; private static final long USER_ID = 4711L;
private static final long APPID = 991220L; private static final long APPID = 991220L;
@@ -42,8 +42,8 @@ class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest {
@Autowired @Autowired
private JdbcTemplate jdbcTemplate; private JdbcTemplate jdbcTemplate;
@MockitoBean @Autowired
private SteamApiService steamApiService; private GameSpeedrunRepository gameSpeedrunRepository;
@BeforeEach @BeforeEach
@AfterEach @AfterEach
@@ -53,6 +53,23 @@ class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest {
jdbcTemplate.update("DELETE FROM wishlist_game WHERE user_id = ?", USER_ID); jdbcTemplate.update("DELETE FROM wishlist_game WHERE user_id = ?", USER_ID);
jdbcTemplate.update("DELETE FROM user_steam_sync WHERE user_id = ?", USER_ID); jdbcTemplate.update("DELETE FROM user_steam_sync WHERE user_id = ?", USER_ID);
jdbcTemplate.update("DELETE FROM steam_game WHERE appid = ?", APPID); jdbcTemplate.update("DELETE FROM steam_game WHERE appid = ?", APPID);
jdbcTemplate.update("DELETE FROM game_speedrun WHERE appid = ?", APPID);
}
@Test
void runSyncNow_TheFollowUpStoresTheSpeedrunRow() {
when(steamApiService.getWishlist(STEAM_ID)).thenReturn(Optional.empty());
when(steamApiService.getOwnedGames(STEAM_ID))
.thenReturn(List.of(new SteamOwnedGamesResponse.Game((int) APPID, "Half-Life 2", 0, "icon", true, 0)));
when(speedrunApiService.isEnabled()).thenReturn(true);
when(speedrunApiService.findGames("Half-Life 2")).thenReturn(Optional.of(List.of(
new SpeedrunApiService.Game("hl2", "Half-Life 2", 2004, "https://www.speedrun.com/hl2"))));
when(speedrunApiService.hasVerifiedRuns("hl2")).thenReturn(Optional.of(true));
worker.runSyncNow(USER_ID, STEAM_ID);
verify(speedrunApiService).hasVerifiedRuns("hl2");
assertThat(gameSpeedrunRepository.findById(APPID)).isPresent();
} }
@Test @Test
@@ -8,6 +8,9 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks; import org.mockito.InjectMocks;
import org.mockito.Mock; import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.test.util.ReflectionTestUtils; import org.springframework.test.util.ReflectionTestUtils;
import java.time.Instant; import java.time.Instant;
@@ -38,6 +41,9 @@ class MoodServiceTest extends AbstractUnitTest {
@Mock @Mock
private UserGameRepository userGameRepository; private UserGameRepository userGameRepository;
@Spy
private TaskExecutor applicationTaskExecutor = new SyncTaskExecutor();
@InjectMocks @InjectMocks
private MoodService service; private MoodService service;
@@ -72,6 +78,16 @@ class MoodServiceTest extends AbstractUnitTest {
verify(gameMoodsRepository).upsert(eq(427520L), any(), any(), eq(true), any(Instant.class)); verify(gameMoodsRepository).upsert(eq(427520L), any(), any(), eq(true), any(Instant.class));
} }
@Test
void refreshInBackground_AllFresh_QueuesNothing() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameMoodsRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(620L));
service.refreshInBackground(List.of(620L));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test @Test
void refreshBacklogInBackground_RefreshesTheBacklogOfTheUser() { void refreshBacklogInBackground_RefreshesTheBacklogOfTheUser() {
when(igdbApiService.isEnabled()).thenReturn(true); when(igdbApiService.isEnabled()).thenReturn(true);
@@ -11,6 +11,9 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks; import org.mockito.InjectMocks;
import org.mockito.Mock; import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.dao.DataAccessResourceFailureException; import org.springframework.dao.DataAccessResourceFailureException;
import java.time.Instant; import java.time.Instant;
@@ -44,6 +47,8 @@ class SpeedrunServiceTest extends AbstractUnitTest {
private SteamGameRepository steamGameRepository; private SteamGameRepository steamGameRepository;
@Mock @Mock
private UserGameRepository userGameRepository; private UserGameRepository userGameRepository;
@Spy
private TaskExecutor syncFollowUpExecutor = new SyncTaskExecutor();
@InjectMocks @InjectMocks
private SpeedrunService service; private SpeedrunService service;
@@ -222,4 +227,14 @@ class SpeedrunServiceTest extends AbstractUnitTest {
verifyNoInteractions(gameSpeedrunRepository, steamGameRepository); verifyNoInteractions(gameSpeedrunRepository, steamGameRepository);
} }
@Test
void onLibrarySynced_RefreshesTheBacklogOnTheListenerThread() {
when(userGameRepository.findBacklogAppids(7L)).thenReturn(List.of());
service.onLibrarySynced(new LibrarySyncedEvent(7L, "76561198000000000"));
verify(userGameRepository).findBacklogAppids(7L);
verify(syncFollowUpExecutor, never()).execute(any());
}
} }
@@ -2,12 +2,12 @@ package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService; import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.shared.test.AbstractIntegrationTest; import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean; import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.transaction.PlatformTransactionManager; import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate; import org.springframework.transaction.support.TransactionTemplate;
@@ -26,7 +26,10 @@ import static org.mockito.Mockito.when;
/** /**
* Two users' syncs run in parallel and can miss the same app, e.g. every app right after the first deploy. * Two users' syncs run in parallel and can miss the same app, e.g. every app right after the first deploy.
*/ */
class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest { class TimeToBeatServiceConcurrencyTest extends AbstractMockedAdaptersTest {
@Autowired
private JdbcTemplate jdbc;
@Autowired @Autowired
private TimeToBeatService timeToBeatService; private TimeToBeatService timeToBeatService;
@@ -37,9 +40,6 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
@Autowired @Autowired
private PlatformTransactionManager transactionManager; private PlatformTransactionManager transactionManager;
@MockitoBean
private IgdbApiService igdbApiService;
// Before as well: other integration tests leave rows in this shared table, and a fresh row would hide the race. // Before as well: other integration tests leave rows in this shared table, and a fresh row would hide the race.
@BeforeEach @BeforeEach
@AfterEach @AfterEach
@@ -60,8 +60,8 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
timeToBeatService.refresh(List.of(620L)); timeToBeatService.refresh(List.of(620L));
firstWrote.countDown(); firstWrote.countDown();
await(secondStarted); await(secondStarted);
// Keep the transaction open while the second sync writes the same row. // Keep the transaction open until the second sync waits for the same row.
sleep(); awaitLockWaiter(jdbc);
})); }));
CompletableFuture<Void> second = CompletableFuture.runAsync(() -> { CompletableFuture<Void> second = CompletableFuture.runAsync(() -> {
await(firstWrote); await(firstWrote);
@@ -82,11 +82,4 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
} }
} }
private static void sleep() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
} }
@@ -7,6 +7,10 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks; import org.mockito.InjectMocks;
import org.mockito.Mock; import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.dao.QueryTimeoutException;
import org.springframework.core.task.TaskExecutor;
import org.springframework.test.util.ReflectionTestUtils; import org.springframework.test.util.ReflectionTestUtils;
import java.time.Instant; import java.time.Instant;
@@ -15,6 +19,7 @@ import java.util.Map;
import java.util.Optional; import java.util.Optional;
import java.util.Set; import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyCollection; import static org.mockito.ArgumentMatchers.anyCollection;
import static org.mockito.ArgumentMatchers.eq; import static org.mockito.ArgumentMatchers.eq;
@@ -31,6 +36,9 @@ class TimeToBeatServiceTest extends AbstractUnitTest {
@Mock @Mock
private GameTimeToBeatRepository gameTimeToBeatRepository; private GameTimeToBeatRepository gameTimeToBeatRepository;
@Spy
private TaskExecutor applicationTaskExecutor = new SyncTaskExecutor();
@InjectMocks @InjectMocks
private TimeToBeatService service; private TimeToBeatService service;
@@ -53,6 +61,37 @@ class TimeToBeatServiceTest extends AbstractUnitTest {
verify(gameTimeToBeatRepository, never()).upsert(eq(220L), any(), any(), any(), any()); verify(gameTimeToBeatRepository, never()).upsert(eq(220L), any(), any(), any(), any());
} }
@Test
void refreshInBackground_AllFresh_QueuesNothing() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(220L, 620L));
service.refreshInBackground(List.of(220L, 620L));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test
void refreshInBackground_StaleCheckFails_DoesNotFailTheCaller() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenThrow(new QueryTimeoutException("slow"));
assertDoesNotThrow(() -> service.refreshInBackground(List.of(620L)));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test
void refreshInBackground_QueuesOnlyTheStaleApps() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(220L));
when(igdbApiService.getTimeToBeat(Set.of(620L))).thenReturn(Optional.of(Map.of()));
service.refreshInBackground(List.of(220L, 620L));
verify(igdbApiService).getTimeToBeat(Set.of(620L));
verify(gameTimeToBeatRepository).upsert(eq(620L), isNull(), isNull(), isNull(), any(Instant.class));
}
@Test @Test
void refresh_GamesOffSteam_AreAskedByIgdbIdAndStoredUnderTheirKey() { void refresh_GamesOffSteam_AreAskedByIgdbIdAndStoredUnderTheirKey() {
when(igdbApiService.isEnabled()).thenReturn(true); when(igdbApiService.isEnabled()).thenReturn(true);
@@ -1,17 +1,24 @@
package core_service.shared.test; package core_service.shared.test;
import core_service.shared.background.BackgroundPools;
import org.junit.jupiter.api.Tag; import org.junit.jupiter.api.Tag;
import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.testcontainers.service.connection.ServiceConnection; import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.ActiveProfiles; import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.DynamicPropertyRegistry; import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource; import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.bean.override.convention.TestBean;
import org.testcontainers.containers.PostgreSQLContainer; import org.testcontainers.containers.PostgreSQLContainer;
import java.security.KeyPair; import java.security.KeyPair;
import java.security.KeyPairGenerator; import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException; import java.security.NoSuchAlgorithmException;
import java.time.Duration;
import java.util.Base64; import java.util.Base64;
import java.util.concurrent.atomic.AtomicReference;
/** /**
* Base class for integration tests. * Base class for integration tests.
@@ -37,6 +44,63 @@ public abstract class AbstractIntegrationTest {
postgres.start(); postgres.start();
} }
// Background work is done when the call returns, so a test reads the result at once. The default pool runs it on
// the calling thread. The follow-up pool runs it on a new thread and waits: an after-commit listener then has its
// own transactions, as in production, and does not join the finished one of the sync.
@TestBean(name = BackgroundPools.DEFAULT)
private TaskExecutor applicationTaskExecutor;
@TestBean(name = BackgroundPools.SYNC_FOLLOW_UP)
private TaskExecutor syncFollowUpExecutor;
static TaskExecutor applicationTaskExecutor() {
return new SyncTaskExecutor();
}
static TaskExecutor syncFollowUpExecutor() {
return task -> {
AtomicReference<Throwable> failure = new AtomicReference<>();
Thread thread = Thread.ofVirtual().start(() -> {
try {
task.run();
} catch (Throwable e) {
failure.set(e);
}
});
try {
if (!thread.join(Duration.ofSeconds(30))) {
throw new AssertionError("Follow-up task still runs after 30 s");
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IllegalStateException(e);
}
if (failure.get() != null) {
throw new AssertionError("Follow-up task failed", failure.get());
}
};
}
/**
* Waits until another connection waits for a row lock: the race a concurrency test sets up is then in place.
*/
protected static void awaitLockWaiter(JdbcTemplate jdbc) {
for (int attempt = 0; attempt < 250; attempt++) {
Integer waiting = jdbc.queryForObject(
"select count(*) from pg_stat_activity where wait_event_type = 'Lock'", Integer.class);
if (waiting != null && waiting > 0) {
return;
}
try {
Thread.sleep(20);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IllegalStateException(e);
}
}
throw new AssertionError("No connection waits for a lock");
}
// No key is committed: a fresh one per JVM. Tests that sign tokens use TEST_KEYS.getPrivate(). // No key is committed: a fresh one per JVM. Tests that sign tokens use TEST_KEYS.getPrivate().
protected static final KeyPair TEST_KEYS = generateKeys(); protected static final KeyPair TEST_KEYS = generateKeys();
@@ -0,0 +1,22 @@
package core_service.shared.test;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.modules.steam_adapter.service.SteamApiService;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
/**
* Integration tests with the three external APIs mocked, in one shared context. A mock answers empty, so a sync a
* request starts finds no games and leaves the library as it is.
*/
public abstract class AbstractMockedAdaptersTest extends AbstractIntegrationTest {
@MockitoBean
protected SteamApiService steamApiService;
@MockitoBean
protected IgdbApiService igdbApiService;
@MockitoBean
protected SpeedrunApiService speedrunApiService;
}
@@ -0,0 +1,21 @@
package core_service.shared.test;
import core_service.modules.library.service.LibrarySyncService;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
/**
* Integration tests of the picker and stats pages: the sync and the background refreshes they start are mocked.
*/
public abstract class AbstractMockedLibraryServicesTest extends AbstractIntegrationTest {
@MockitoBean
protected LibrarySyncService syncService;
@MockitoBean
protected MoodService moodService;
@MockitoBean
protected SpeedrunService speedrunService;
}