Compare commits

..
8 Commits
Author SHA1 Message Date
lobo 3512159b41 test: background work inline, shared mock bases, lock-wait instead of sleep
Build, Test & Push Core Service / docker (push) Successful in 1m31s
Build, Test & Push Core Service / quality-check (push) Successful in 1m8s
2026-09-30 12:08:57 +02:00
lobo 7fe2823519 refactor(library): detail page refreshes, then assembles 2026-09-30 12:08:57 +02:00
lobo 2c97e2ae96 refactor(library): two bounded background pools
Every @Async names a pool: with the scheduler there are two TaskExecutor beans, and a bare @Async ran on an unbounded executor. Page views queue only stale games; the long runs after a sync get their own pool.
2026-09-30 12:08:57 +02:00
lobo 9a09d535fc refactor(library): listed_user_game view and one shared schema (V24) 2026-09-30 09:37:16 +02:00
lobo b499e79c41 refactor(library): one IGDB split and merge for time to beat and moods 2026-09-30 09:37:16 +02:00
lobo c1472c7d59 refactor(library): own IgdbGame record, one owned rule, layer rules
Stored import candidates keep their JSON shape, so READY imports from before still read.
2026-09-30 09:37:16 +02:00
lobo 1abc03d2c1 refactor(library): auditor fields only through applyCorrection 2026-09-30 09:37:16 +02:00
lobo 79d7bf17f2 feat(library): PATCH backlog saves only the sent fields 2026-09-30 07:36:08 +02:00
59 changed files with 1089 additions and 427 deletions

No files matched your search

+62 -10
View File
@@ -52,7 +52,8 @@ To Be Played is split into three repositories:
40-hour weeks, oldest unplayed game); most played and most loyal game (most months with playtime); top IGDB genres
and moods; moon trips (Apollo 11, 195.3 h), longest day, share under one hour; finished games per year (from the
status history); the Pile of Shame trend (90 days of `backlog_snapshot`, plus "this month: added / done", where the
first sync of a Steam account is no "added", see `user_steam_sync.history_since`). Every number counts only listed games.
first sync of a Steam account is no "added", see `user_steam_sync.history_since`). Every number counts only listed games (still owned, not hidden: the view
`listed_user_game`, V24).
- **Status history**: a database trigger logs every status change of a library game in `user_game_status_change`
(old and new status, time, whether the user chose it), whatever code path made it. It starts with V14; older
changes are not known.
@@ -97,7 +98,7 @@ only guard.
Every service method a controller calls carries one role marker, on the method or on its class (the method wins):
`@RequiresRole(Role.X)`, `@Public` (no login, e.g. landing stats; the public profile checks its ticket) or
`@Internal` (only Auth's service token). A Spring Security method interceptor checks it while a request is served;
scheduled jobs and `@Async` work run without a user and are not checked. A denied call answers `401` without a
scheduled jobs and background work run without a user and are not checked. A denied call answers `401` without a
login, else `403`. The URL rules in `SecurityConfig` stay as the outer guard. ArchUnit fails the build for an
unmarked method, two markers, a controller calling an `@Async` method (the check would never run), and a service
method calling another service's method that needs more.
@@ -107,7 +108,8 @@ method calling another service's method that needs more.
| Method | Path | Auth |
|---|---|---|
| `GET` | `/api/library/games`, `/api/library/games/{appid}` | JWT |
| `PUT` | `/api/library/games/{appid}/backlog` | JWT |
| `PATCH` | `/api/library/games/{appid}/backlog` (only the sent fields; `null` clears one) | JWT |
| `PUT` | `/api/library/games/{appid}/backlog` (replaces all four; removed in item 29) | JWT |
| `PUT`, `DELETE` | `/api/library/games/{appid}/corrections/{field}` (`{value}`; `name`, `image`, `speedrun`, `moods`, `endless`, `genres`; `204`, `400 INVALID_CORRECTION`) | JWT, AUDITOR |
| `POST` | `/api/library/games/{appid}/merge` (`{otherAppid}`; `200 {appid}` = the key that stays, `400 SAME_GAME` / `STEAM_GAMES`, `404 GAME_NOT_FOUND`) | JWT, AUDITOR |
| `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT |
@@ -171,9 +173,9 @@ flowchart LR
Igdb -->|"OAuth token"| Twitch["Twitch"]
Speedrun --> SpeedrunApi["speedrun.com API"]
LibRepo --> Db[("PostgreSQL tbp_core")]
Stats --> Db
Stats -->|"listed_user_game, deleted_user"| Db
Auth -->|"service token<br/>/internal/accounts"| Account
Account --> Db
Account -->|"per-user rows, deleted_user"| Db
```
| Module | Content |
@@ -186,6 +188,13 @@ flowchart LR
| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search |
| `speedrun_adapter` | speedrun.com client: game search by name, verified runs |
**One shared schema.** All modules use one database `tbp_core`, and some read or write another module's tables in
SQL, which ArchUnit cannot see: `stats` reads `listed_user_game`, `steam_game` and `deleted_user`; `account` deletes
the per-user rows of the library tables (`AccountCleanupRepository`; a new library table with `user_id` goes into
its list); `library` reads `deleted_user`. A module that moves to its own service takes these reads along as calls.
The listed rule ("still owned, not hidden") is `UserGameRepository.LISTED` for JPQL and the view
`listed_user_game` for native SQL; V24 says what a migration of `user_game` must do with the view.
Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` (JPA entities) and `repo`
(repositories). `src/test/java/core_service/architecture/ArchitectureTest.java` checks these rules on every build:
@@ -193,6 +202,10 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
`speedrun_adapter`.
- No cycles between `modules`, `shared` and `configuration`.
- `api` never uses `repo`, and `repo` never uses `api`.
- `domain` never uses `api`, `service`, `repo` or an adapter: the import stores IGDB hits as the own record
`IgdbGame`, so a change of the adapter's `IgdbApiService.Game` cannot break stored imports.
- `service` never uses `api`, except the adapters' response DTOs and a fixed list of 8 services that still build
their module's response DTOs. The list only gets shorter.
- Every class in `repo` ends with `Repository`.
- `shared` never uses `modules`, and `modules` never use `configuration`.
- A `*Service` class in a module's `service` package has `@Service`. A `*Controller` class in `api` has
@@ -208,7 +221,7 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain`
Afterwards it writes every user's pile size of the day to `backlog_snapshot`, also for users without Steam, and
dates finished games by the unlock of their ending achievement
(`FinishedEstimateService`, `user_game.finished_at_estimate`).
3. `LibrarySyncWorker` runs in the background. `user_steam_sync` lets only one run per user start, and none for
3. `LibrarySyncWorker` runs on the background pool. `user_steam_sync` lets only one run per user start, and none for
a user in `deleted_user`.
4. The worker loads the wishlist and the owned games from Steam. It updates `steam_game` and `user_game`, adds
the playtime growth of the day to `user_game_playtime_day`, and soft-deletes Steam games the user no longer
@@ -226,16 +239,50 @@ The frontend polls `GET /api/library/sync-status` until the run ends.
At 01:30 `ImageRepairService` repairs the pictures reported through `POST /api/library/broken-images` (table
`broken_image`, one row per game and kind). Reports only count for Steam games the user owns.
### Background work
Background work runs on two pools (`AsyncConfig`; every `@Async` names one, ArchUnit checks it):
- The default pool (4 threads, queue 100) runs the syncs and the time-to-beat and mood refreshes a page view
starts. A page view queues only games without a fresh row. When the queue is full, the task is dropped with
a warning, and the next view or sync asks again. Known limit: a dropped sync still answers `202`; the user
starts it again.
- The follow-up pool (2 threads, no limit on the queue) runs the achievement runs after a sync and every speedrun
refresh. They hold a thread for minutes because of the Steam and speedrun.com pace, so they do not take the page
views' threads. A task is small (a user or a few appids), and none is dropped.
Imports have their own two threads, and `game_import` is their queue. Scheduled jobs (nightly sync, image repair,
import watchdog) run on the scheduler.
```mermaid
flowchart LR
View["page view, add, merge"] -->|"stale games only"| Pool["default pool<br/>4 threads, queue 100"]
Sync["sync request"] --> Pool
Synced["sync, after commit"] --> FollowUp["follow-up pool<br/>2 threads"]
View -->|"speedrun"| FollowUp
Pool --> Apis["Steam, IGDB, speedrun.com"]
FollowUp --> Apis
Import["POST /imports"] --> Lanes["GameImportWorker<br/>2 own threads"]
Cron["scheduler"] --> Jobs["nightly sync, image repair, import watchdog"]
```
Core runs as one instance. The pools, the speedrun queue, the per-user achievement runs and the achievement locks
live in memory: a second instance would ask Steam and IGDB twice, and the database would stay consistent (sync
claim, upserts). A restart drops what waits in the pools; it is all refresh work and is asked again. Imports survive
a restart.
### Corrections by an auditor
An AUDITOR (active role) corrects a game's name, header image, speedrun link, moods, endless flag and genres for
every user (`game_correction`, V20). The value is also written into the cached row (`steam_game`, `game_speedrun`,
`game_mood`), so no read joins the corrections. The refresh writers skip a corrected field: the Steam sync (name),
the image repair (image), the IGDB add (name, image), the mood and speedrun upserts (SQL). **A new writer of these
fields must do the same.** "Back to source" (`DELETE`) writes back the source value from the first correction and
the image repair (image), the IGDB add (name, image), the mood and speedrun upserts (SQL). `SteamGame` has no
setter for name and image: `applyCorrection` is the only Java writer, a refresh uses the guarded updates in
`SteamGameRepository`. "Back to source" (`DELETE`) writes back the source value from the first correction and
lets the next refresh ask IGDB or speedrun.com again. An image must be on a host of the frontend's CSP.
Known limit: a mood or speedrun refresh that waits on the row lock while an auditor saves can still write the
source value once (its guard sees the state from before the save); the auditor saves again.
source value once (its guard sees the state from before the save); the auditor saves again. `save(new SteamGame(...))` on a
stored key also overwrites a corrected name; the sync and the IGDB add do it only after `findById` found no row.
### Merging duplicates
@@ -390,11 +437,16 @@ Configuration comes from environment variables:
```
Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Needs Docker for Testcontainers.
In integration tests background work runs on the calling thread, so a test reads the result at once.
`AbstractMockedAdaptersTest` mocks Steam, IGDB and speedrun.com, `AbstractMockedLibraryServicesTest` the sync and
the refreshes of the picker and stats pages; each is one shared Spring context. Race tests wait until the other
connection waits for the row lock (`awaitLockWaiter`), not for a fixed time.
## Roadmap
- Sale alerts for wishlist games.
- Move `steam_adapter` DTOs out of `api`, then turn on strict module independence in ArchUnit.
- Move the adapter DTOs out of `api`, then turn on strict module independence in ArchUnit and drop the adapter
exception of the "`service` never uses `api`" rule.
## License
@@ -1,9 +1,46 @@
package core_service.configuration;
import core_service.shared.background.BackgroundPools;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
/**
* The two pools of {@link BackgroundPools}. A dropped task is a refresh of data with a TTL: the next view or sync
* asks again.
*/
@Configuration
@EnableAsync
@Slf4j
public class AsyncConfig {
// A task holds a connection only for its short transactions. At most 6 background tasks, plus the scheduler and
// the 2 import lanes, use Hikari's 10 connections at once; requests share the rest.
static final int THREADS = 4;
static final int QUEUE = 100;
// The achievement runs share one Steam pace and speedrun has one drainer: more threads would only wait.
static final int FOLLOW_UP_THREADS = 2;
@Bean(name = BackgroundPools.DEFAULT)
public ThreadPoolTaskExecutor applicationTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(THREADS);
executor.setMaxPoolSize(THREADS);
executor.setQueueCapacity(QUEUE);
executor.setThreadNamePrefix("background-");
executor.setRejectedExecutionHandler((task, pool) -> log.warn(pool.isShutdown()
? "Background pool stopped, task dropped" : "Background queue full, task dropped"));
return executor;
}
@Bean(name = BackgroundPools.SYNC_FOLLOW_UP)
public ThreadPoolTaskExecutor syncFollowUpExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(FOLLOW_UP_THREADS);
executor.setMaxPoolSize(FOLLOW_UP_THREADS);
executor.setThreadNamePrefix("sync-follow-up-");
return executor;
}
}
@@ -53,7 +53,7 @@ public class SecurityConfig {
CorsConfiguration configuration = new CorsConfiguration();
// Exact origins only: with credentials, a pattern would send the login cookie to every matching site.
configuration.setAllowedOrigins(Arrays.asList(allowedOrigins));
configuration.setAllowedMethods(List.of("GET", "POST", "PUT", "DELETE", "OPTIONS"));
configuration.setAllowedMethods(List.of("GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"));
configuration.setAllowedHeaders(List.of(
"Content-Type", "Authorization", ClientHeaderFilter.CLIENT_HEADER, "X-Profile-Ticket"));
configuration.setAllowCredentials(true);
@@ -4,6 +4,7 @@ import core_service.shared.api.ErrorResponse;
import core_service.modules.library.api.dto.AchievementSummaryDto;
import core_service.modules.library.api.dto.ActivityDto;
import core_service.modules.library.api.dto.BacklogEntryDto;
import core_service.modules.library.api.dto.BacklogPatch;
import core_service.modules.library.api.dto.BrokenImageReport;
import core_service.modules.library.api.dto.CorrectionDto;
import core_service.modules.library.api.dto.FinishedHintDismissal;
@@ -47,6 +48,7 @@ import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.PatchMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
@@ -231,6 +233,15 @@ public class LibraryController {
.body(GAME_NOT_FOUND));
}
@PatchMapping("/games/{appid}/backlog")
public ResponseEntity<?> patchBacklog(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") Long appid,
@Valid @RequestBody BacklogPatch patch) {
return backlogService.patch(principal.getIdAsLong(), appid, patch)
.<ResponseEntity<?>>map(ResponseEntity::ok)
.orElseGet(LibraryController::unknownGame);
}
@PutMapping("/games/{appid}/hidden")
public ResponseEntity<?> updateHidden(@AuthenticationPrincipal UserPrincipal principal,
@PathVariable("appid") Long appid,
@@ -9,6 +9,7 @@ import jakarta.validation.constraints.Size;
/**
* Request and response of the backlog update. The request replaces all four fields; null clears an optional one.
* The PUT goes away in item 29; the Frontend sends a {@link BacklogPatch}.
*/
public record BacklogEntryDto(
@NotNull GameStatus status,
@@ -0,0 +1,26 @@
package core_service.modules.library.api.dto;
import core_service.modules.library.domain.GamePriority;
import core_service.modules.library.domain.GameStatus;
import jakarta.validation.constraints.Max;
import jakarta.validation.constraints.Min;
import jakarta.validation.constraints.NotNull;
import jakarta.validation.constraints.Size;
import lombok.Getter;
import lombok.Setter;
import java.util.Optional;
/**
* The backlog fields the request sends; a field it leaves out stays as it is. A field is null when it is not in the
* JSON and an empty Optional when it is null there, which clears it.
* A class, not a record: for a record Jackson gives an empty Optional for a missing field too.
*/
@Getter
@Setter
public class BacklogPatch {
private Optional<@NotNull GameStatus> status;
private Optional<@Min(1) @Max(5) Integer> rating;
private Optional<@Size(max = 2000) String> notes;
private Optional<GamePriority> priority;
}
@@ -1,7 +1,5 @@
package core_service.modules.library.domain;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import java.time.Instant;
import java.util.List;
@@ -46,7 +44,7 @@ public record GameImport(long id, long userId, Source source, Status status, int
* @param exact the first candidate has the line's name (case and punctuation ignored)
* @param candidates best first, at most five
*/
public record Row(Line line, boolean searchFailed, boolean exact, List<IgdbApiService.Game> candidates) {}
public record Row(Line line, boolean searchFailed, boolean exact, List<IgdbGame> candidates) {}
/**
* The user's pick for a line in the review.
@@ -0,0 +1,9 @@
package core_service.modules.library.domain;
import java.util.List;
/**
* An IGDB game as the library needs it, and the stored shape of an import candidate (V22 {@code candidates}): the
* component names are the JSON keys of rows already stored.
*/
public record IgdbGame(long igdbId, String name, Integer year, String imageUrl, List<String> platforms, Long steamAppid) {}
@@ -17,7 +17,6 @@ import java.time.Instant;
@DynamicUpdate
@Table(name = "steam_game")
@Getter
@Setter
@NoArgsConstructor(access = AccessLevel.PROTECTED)
public class SteamGame {
@@ -28,9 +27,11 @@ public class SteamGame {
@Column(name = "name", nullable = false, length = 512)
private String name;
@Setter
@Column(name = "img_icon_url", length = 512)
private String imgIconUrl;
@Setter
@Column(name = "last_seen_at", nullable = false)
private Instant lastSeenAt;
@@ -45,9 +46,26 @@ public class SteamGame {
}
public SteamGame(Long appid, String name, String imgIconUrl, Instant lastSeenAt) {
this(appid, name, imgIconUrl, null, lastSeenAt);
}
public SteamGame(Long appid, String name, String imgIconUrl, String imageUrl, Instant lastSeenAt) {
this.appid = appid;
this.name = name;
this.imgIconUrl = imgIconUrl;
this.imageUrl = imageUrl;
this.lastSeenAt = lastSeenAt;
}
/**
* The only Java writer of name and image on a stored game. Refreshes from a source use the guarded updates in
* {@code SteamGameRepository}, so they never overwrite a correction.
*/
public void applyCorrection(CorrectionField field, String value) {
switch (field) {
case NAME -> name = value;
case IMAGE -> imageUrl = value;
default -> throw new IllegalArgumentException(field + " is not a column of steam_game");
}
}
}
@@ -22,8 +22,7 @@ public class BacklogSnapshotRepository {
count(*) filter (where ug.playtime_minutes = 0) as never_played,
coalesce(sum(t.normally_minutes) filter (where ug.status = 'BACKLOG'), 0) as backlog_minutes,
count(*) as listed_games
from user_game ug left join game_time_to_beat t on t.appid = ug.appid
where ug.removed_at is null and ug.hidden_at is null
from listed_user_game ug left join game_time_to_beat t on t.appid = ug.appid
""";
private final NamedParameterJdbcTemplate jdbc;
@@ -37,7 +36,7 @@ public class BacklogSnapshotRepository {
return jdbc.update("""
insert into backlog_snapshot (user_id, day, backlog_games, never_played, backlog_minutes, listed_games)
select ug.user_id, :day, %s
and ug.user_id not in (select user_id from deleted_user)
where ug.user_id not in (select user_id from deleted_user)
group by ug.user_id
on conflict (user_id, day) do update set backlog_games = excluded.backlog_games,
never_played = excluded.never_played, backlog_minutes = excluded.backlog_minutes,
@@ -61,7 +60,7 @@ public class BacklogSnapshotRepository {
* The pile of now, for the day that has no row yet.
*/
public UserStats.PileDay live(long userId, LocalDate day) {
return jdbc.queryForObject("select " + PILE_COLUMNS + "and ug.user_id = :userId",
return jdbc.queryForObject("select " + PILE_COLUMNS + "where ug.user_id = :userId",
new MapSqlParameterSource("userId", userId),
(rs, i) -> new UserStats.PileDay(day, rs.getLong("backlog_games"), rs.getLong("never_played"),
rs.getLong("backlog_minutes")));
@@ -1,6 +1,6 @@
package core_service.modules.library.repo;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameImport;
import core_service.modules.library.domain.GameStatus;
import lombok.RequiredArgsConstructor;
@@ -189,11 +189,11 @@ public class GameImportRepository {
new MapSqlParameterSource("id", importId), (rs, row) -> new GameImport.Row(line(rs),
rs.getBoolean("search_failed"), rs.getBoolean("exact"),
jsonMapper.readValue(rs.getString("candidates"),
jsonMapper.getTypeFactory().constructCollectionType(List.class, IgdbApiService.Game.class))));
jsonMapper.getTypeFactory().constructCollectionType(List.class, IgdbGame.class))));
}
public void saveMatch(long importId, int lineNo, boolean searchFailed, boolean exact,
List<IgdbApiService.Game> candidates) {
List<IgdbGame> candidates) {
jdbc.update("update game_import_row set matched_at = now(), search_failed = :failed, exact = :exact, "
+ "candidates = cast(:candidates as jsonb) where import_id = :id and line_no = :line",
new MapSqlParameterSource("id", importId).addValue("line", lineNo).addValue("failed", searchFailed)
@@ -39,7 +39,8 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*/
String LISTED = "ug.userId = :userId and ug.removedAt is null and ug.hiddenAt is null ";
List<UserGame> findByUserIdAndRemovedAtIsNull(Long userId);
@Query("select ug from UserGame ug where " + LISTED + "and ug.source = core_service.modules.library.domain.GameSource.STEAM")
List<UserGame> findListedSteamGames(@Param("userId") Long userId);
List<UserGame> findByUserId(Long userId);
@@ -103,8 +104,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*
* @param excludeAppid never picked; {@link #NO_GAME} excludes nothing
*/
@Query(value = "select * from user_game where user_id = :userId and removed_at is null and hidden_at is null " +
"and status = 'BACKLOG' " +
@Query(value = "select * from listed_user_game where user_id = :userId and status = 'BACKLOG' " +
"and appid <> :excludeAppid " +
// Weighted random pick: the smallest -ln(u) / weight wins.
"order by -ln(1 - random()) / case priority when 'HIGH' then 4 when 'MEDIUM' then 2 else 1 end " +
@@ -136,8 +136,8 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
return queryPickerGames(userId, maxMinutes, storedMood, speedrun, excluded, count);
}
@Query(value = "select ug.* from user_game ug where ug.user_id = :userId and ug.removed_at is null " +
"and ug.hidden_at is null and ug.status = 'BACKLOG' and ug.appid not in (:excludeAppids) " +
@Query(value = "select ug.* from listed_user_game ug where ug.user_id = :userId " +
"and ug.status = 'BACKLOG' and ug.appid not in (:excludeAppids) " +
"and (cast(:maxMinutes as integer) is null or exists (select 1 from game_time_to_beat t " +
" where t.appid = ug.appid and t.normally_minutes <= :maxMinutes)) " +
"and (cast(:mood as varchar) is null or exists (select 1 from game_mood m " +
@@ -220,7 +220,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
@Modifying(clearAutomatically = true)
@Transactional
@Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.FINISHED " +
"where ug.userId = :userId and ug.appid = :appid and ug.removedAt is null and ug.hiddenAt is null " +
"where " + LISTED + "and ug.appid = :appid " +
"and ug.statusSetByUser = false and ug.finishedHintDismissed is null " +
"and ug.source = core_service.modules.library.domain.GameSource.STEAM " +
"and ug.status in (core_service.modules.library.domain.GameStatus.BACKLOG, " +
@@ -253,7 +253,7 @@ public interface UserGameRepository extends JpaRepository<UserGame, UserGameId>
*/
@Modifying
@Query("update UserGame ug set ug.status = core_service.modules.library.domain.GameStatus.PLAYING " +
"where ug.userId = :userId and ug.appid in :appids and ug.removedAt is null and ug.hiddenAt is null " +
"where " + LISTED + "and ug.appid in :appids " +
"and ug.statusSetByUser = false " +
"and ug.status = core_service.modules.library.domain.GameStatus.BACKLOG")
int playFromPileIfUntouched(@Param("userId") Long userId, @Param("appids") Collection<Long> appids);
@@ -23,7 +23,7 @@ import java.util.Optional;
@RequiredArgsConstructor
public class UserStatsRepository {
private static final String LISTED = "ug.user_id = :userId and ug.removed_at is null and ug.hidden_at is null ";
private static final String OF_USER = "ug.user_id = :userId ";
private static final String FINISHED = "('FINISHED', 'COMPLETED')";
/** A status change that finished the game: from another status; not a game that came in finished. */
private static final String FINISH_CHANGE =
@@ -46,9 +46,9 @@ public class UserStatsRepository {
return jdbc.queryForObject("""
select coalesce(sum(t.normally_minutes), 0) as normal, coalesce(sum(t.completely_minutes), 0) as complete,
count(t.normally_minutes) as known
from user_game ug join game_time_to_beat t on t.appid = ug.appid
from listed_user_game ug join game_time_to_beat t on t.appid = ug.appid
where %s and ug.status = 'BACKLOG'
""".formatted(LISTED), user(userId),
""".formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.PileTimes(rs.getLong("normal"), rs.getLong("complete"), rs.getLong("known")));
}
@@ -62,10 +62,10 @@ public class UserStatsRepository {
public Optional<UserStats.OldestGame> oldestUnplayed(long userId) {
return jdbc.query("""
select ug.appid, sg.name, sg.image_url, ug.first_seen_at
from user_game ug join steam_game sg on sg.appid = ug.appid
from listed_user_game ug join steam_game sg on sg.appid = ug.appid
where %s and ug.status = 'BACKLOG' and ug.playtime_minutes = 0
order by ug.first_seen_at, ug.appid limit 1
""".formatted(LISTED), user(userId),
""".formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.OldestGame(rs.getLong("appid"), rs.getString("name"),
rs.getString("image_url"), rs.getTimestamp("first_seen_at").toInstant()))
.stream().findFirst();
@@ -74,10 +74,10 @@ public class UserStatsRepository {
public List<UserStats.GameMinutes> mostPlayed(long userId, int limit) {
return jdbc.query("""
select ug.appid, sg.name, sg.image_url, ug.playtime_minutes
from user_game ug join steam_game sg on sg.appid = ug.appid
from listed_user_game ug join steam_game sg on sg.appid = ug.appid
where %s and ug.playtime_minutes > 0
order by ug.playtime_minutes desc, ug.appid limit :limit
""".formatted(LISTED), user(userId).addValue("limit", limit),
""".formatted(OF_USER), user(userId).addValue("limit", limit),
(rs, i) -> new UserStats.GameMinutes(rs.getLong("appid"), rs.getString("name"),
rs.getString("image_url"), rs.getLong("playtime_minutes")));
}
@@ -90,7 +90,7 @@ public class UserStatsRepository {
select l.appid, sg.name, sg.image_url, l.months, l.minutes
from (select p.appid, count(distinct date_trunc('month', p.day)) as months, sum(p.minutes) as minutes
%s group by p.appid) l
join user_game ug on ug.user_id = :userId and ug.appid = l.appid and ug.removed_at is null
join listed_user_game ug on ug.user_id = :userId and ug.appid = l.appid
join steam_game sg on sg.appid = l.appid
order by l.months desc, l.minutes desc, l.appid limit 1
""".formatted(PLAYTIME_DAYS_NOT_HIDDEN), user(userId),
@@ -110,10 +110,10 @@ public class UserStatsRepository {
private List<UserStats.TagCount> tagCounts(long userId, String column, int limit) {
return jdbc.query("""
select tag.name, count(*) as games, coalesce(sum(ug.playtime_minutes), 0) as minutes
from user_game ug join game_mood m on m.appid = ug.appid cross join unnest(m.%s) as tag(name)
from listed_user_game ug join game_mood m on m.appid = ug.appid cross join unnest(m.%s) as tag(name)
where %s
group by tag.name order by minutes desc, games desc, tag.name limit :limit
""".formatted(column, LISTED), user(userId).addValue("limit", limit),
""".formatted(column, OF_USER), user(userId).addValue("limit", limit),
(rs, i) -> new UserStats.TagCount(rs.getString("name"), rs.getLong("games"), rs.getLong("minutes")));
}
@@ -128,8 +128,8 @@ public class UserStatsRepository {
return jdbc.queryForObject("""
select count(*) filter (where ug.playtime_minutes > 0) as started,
count(*) filter (where ug.playtime_minutes between 1 and 59) as under_one_hour
from user_game ug where %s
""".formatted(LISTED), user(userId),
from listed_user_game ug where %s
""".formatted(OF_USER), user(userId),
(rs, i) -> new UserStats.PlayedCounts(rs.getLong("started"), rs.getLong("under_one_hour")));
}
@@ -141,20 +141,20 @@ public class UserStatsRepository {
return jdbc.query("""
select cast(extract(year from coalesce(ug.finished_at_estimate, f.finished_at) at time zone :zone)
as integer) as year, count(*) as games
from user_game ug left join (%s) f on f.appid = ug.appid
from listed_user_game ug left join (%s) f on f.appid = ug.appid
where %s and ug.status in %s and coalesce(ug.finished_at_estimate, f.finished_at) is not null
group by year order by year
""".formatted(FIRST_FINISH, LISTED, FINISHED), user(userId).addValue("zone", zone.getId()),
""".formatted(FIRST_FINISH, OF_USER, FINISHED), user(userId).addValue("zone", zone.getId()),
(rs, i) -> new UserStats.YearCount(rs.getInt("year"), rs.getLong("games")));
}
public long finishedBeforeTracking(long userId) {
Long games = jdbc.queryForObject("""
select count(*) from user_game ug
select count(*) from listed_user_game ug
where %s and ug.status in %s and ug.finished_at_estimate is null
and not exists (select 1 from user_game_status_change sc
where sc.user_id = ug.user_id and sc.appid = ug.appid and %s)
""".formatted(LISTED, FINISHED, FINISH_CHANGE), user(userId), Long.class);
""".formatted(OF_USER, FINISHED, FINISH_CHANGE), user(userId), Long.class);
return games != null ? games : 0L;
}
@@ -170,10 +170,10 @@ public class UserStatsRepository {
or sc.from_status is not null or sc.changed_at > uss.history_since)) as added,
count(distinct sc.appid) filter (where sc.from_status = 'BACKLOG' and sc.to_status <> 'BACKLOG')
as done
from user_game_status_change sc join user_game ug on ug.user_id = sc.user_id and ug.appid = sc.appid
from user_game_status_change sc join listed_user_game ug on ug.user_id = sc.user_id and ug.appid = sc.appid
left join user_steam_sync uss on uss.user_id = sc.user_id
where %s and sc.changed_at >= :monthStart
""".formatted(LISTED), user(userId).addValue("monthStart", Timestamp.from(monthStart)),
""".formatted(OF_USER), user(userId).addValue("monthStart", Timestamp.from(monthStart)),
(rs, i) -> new UserStats.MonthMoves(rs.getLong("added"), rs.getLong("done")));
}
@@ -3,7 +3,6 @@ package core_service.modules.library.service;
import core_service.modules.library.domain.GameAchievementSchema;
import core_service.modules.library.domain.EndingAchievement;
import core_service.modules.library.domain.GameAchievementSchemaMeta;
import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameAchievement;
@@ -16,6 +15,7 @@ import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.background.BackgroundPools;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async;
@@ -94,7 +94,7 @@ public class AchievementSyncService {
}
}
@Async
@Async(BackgroundPools.SYNC_FOLLOW_UP)
@TransactionalEventListener
public void onLibrarySynced(LibrarySyncedEvent event) {
syncLibrary(event.userId(), event.steamId());
@@ -163,8 +163,7 @@ public class AchievementSyncService {
}
private void runLibrarySync(Long userId, String steamId) {
List<UserGame> games = userGameRepository.findByUserIdAndRemovedAtIsNull(userId).stream()
.filter(game -> game.getSource() == GameSource.STEAM && game.getHiddenAt() == null)
List<UserGame> games = userGameRepository.findListedSteamGames(userId).stream()
.sorted(Comparator.comparingInt(UserGame::getPlaytimeMinutes).reversed())
.toList();
Map<Long, GameAchievementSchemaMeta> schemas = schemaMetaRepository
@@ -1,6 +1,7 @@
package core_service.modules.library.service;
import core_service.modules.library.api.dto.BacklogEntryDto;
import core_service.modules.library.api.dto.BacklogPatch;
import core_service.modules.library.domain.FinishedHintType;
import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus;
@@ -40,6 +41,28 @@ public class BacklogService {
});
}
/**
* Changes only the fields the patch sends. Only a sent status counts as set by the user, so a save of the notes
* alone still lets the sync move the game from BACKLOG to PLAYING.
*
* @return empty when the game is not in the user's active library
*/
@Transactional
public Optional<BacklogEntryDto> patch(Long userId, Long appid, BacklogPatch patch) {
return userGameRepository.findById(new UserGameId(userId, appid))
.filter(game -> game.getRemovedAt() == null)
.map(game -> {
if (patch.getStatus() != null) {
game.setStatus(patch.getStatus().orElseThrow());
game.setStatusSetByUser(true);
}
if (patch.getRating() != null) game.setRating(patch.getRating().orElse(null));
if (patch.getNotes() != null) game.setNotes(patch.getNotes().filter(notes -> !notes.isBlank()).orElse(null));
if (patch.getPriority() != null) game.setPriority(patch.getPriority().orElse(null));
return toDto(game);
});
}
/**
* Changes only the status and keeps rating, notes and priority.
*
@@ -115,8 +115,7 @@ public class GameCorrectionService {
private void write(CorrectionField field, SteamGame game, Object value) {
switch (field) {
case NAME -> game.setName((String) value);
case IMAGE -> game.setImageUrl((String) value);
case NAME, IMAGE -> game.applyCorrection(field, (String) value);
// A new row has only the corrected link, so the next check would ask speedrun.com for nothing.
case SPEEDRUN -> gameSpeedrunRepository.writeWeblink(game.getAppid(), (String) value, Instant.now());
case MOODS -> gameMoodsRepository.writeMoods(game.getAppid(), strings(value), LONG_AGO);
@@ -77,8 +77,20 @@ public class GameDetailService {
if (userGameOpt.isEmpty() || userGameOpt.get().getRemovedAt() != null) {
return Optional.empty();
}
UserGame userGame = userGameOpt.get();
SteamGame meta = steamGameRepository.findById(appid).orElse(null);
Refreshed refreshed = refresh(userGameOpt.get(), steamId);
return Optional.of(assemble(refreshed.userGame(), meta, refreshed.hasSchema()));
}
/**
* @param userGame the game after the refresh, re-read when the finish rule changed its status
*/
private record Refreshed(UserGame userGame, boolean hasSchema) {}
// May call Steam, IGDB and speedrun.com and write.
private Refreshed refresh(UserGame userGame, String steamId) {
Long userId = userGame.getUserId();
Long appid = userGame.getAppid();
// A hidden game gets no calls to Steam, IGDB or speedrun.com; it shows what is stored.
boolean hidden = userGame.getHiddenAt() != null;
if (!hidden) {
@@ -89,7 +101,7 @@ public class GameDetailService {
// Users without a Steam ID never sync, so the time to beat of their games is asked again here.
timeToBeatService.refreshInBackground(List.of(appid));
moodService.refreshInBackground(List.of(appid));
return Optional.of(buildDto(userGame, meta, true, 0, 0, List.of()));
return new Refreshed(userGame, false);
}
if (!hidden) {
@@ -100,9 +112,8 @@ public class GameDetailService {
}
}
boolean hasSchema = schemaMetaRepository.findById(appid).map(GameAchievementSchemaMeta::isHasSchema).orElse(false);
if (!hasSchema) {
return Optional.of(buildDto(userGame, meta, true, 0, 0, List.of()));
return new Refreshed(userGame, false);
}
if (steamId != null && !hidden) {
@@ -117,7 +128,16 @@ public class GameDetailService {
if (!hidden && !userGame.isStatusSetByUser() && achievementSyncService.finishIfEndingUnlocked(userId, appid)) {
userGame = userGameRepository.findById(new UserGameId(userId, appid)).orElse(userGame);
}
return new Refreshed(userGame, true);
}
// Reads only.
private GameDetailDto assemble(UserGame userGame, SteamGame meta, boolean hasSchema) {
if (!hasSchema) {
return buildDto(userGame, meta, true, 0, 0, List.of(), List.of());
}
Long userId = userGame.getUserId();
Long appid = userGame.getAppid();
UserGameAchievementMeta userMeta = userAchievementMetaRepository.findByUserIdAndAppid(userId, appid).orElse(null);
boolean statsVisible = userMeta == null || userMeta.isStatsVisible();
int unlocked = userMeta != null ? userMeta.getAchievementsUnlocked() : 0;
@@ -145,13 +165,8 @@ public class GameDetailService {
})
.toList();
return Optional.of(buildDto(userGame, meta, statsVisible, schemaList.size(),
AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList));
}
private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible,
int total, int unlocked, List<AchievementDto> achievements) {
return buildDto(userGame, meta, statsVisible, total, unlocked, achievements, List.of());
return buildDto(userGame, meta, statsVisible, schemaList.size(),
AchievementProgress.capped(unlocked, schemaList.size()), achievementDtos, schemaList);
}
private GameDetailDto buildDto(UserGame userGame, SteamGame meta, boolean statsVisible, int total, int unlocked,
@@ -1,7 +1,7 @@
package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameImport;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.repo.GameImportRepository;
import jakarta.annotation.PreDestroy;
import lombok.extern.slf4j.Slf4j;
@@ -49,7 +49,6 @@ public class GameImportWorker {
private final GameImportRepository importRepository;
private final ManualGameService manualGameService;
private final IgdbApiService igdbApiService;
private final TimeToBeatService timeToBeatService;
private final MoodService moodService;
private final Duration stallTime;
@@ -59,11 +58,10 @@ public class GameImportWorker {
private final Lane saving;
public GameImportWorker(GameImportRepository importRepository, ManualGameService manualGameService,
IgdbApiService igdbApiService, TimeToBeatService timeToBeatService, MoodService moodService,
TimeToBeatService timeToBeatService, MoodService moodService,
@Value("${app.library.import-stall-minutes:5}") long stallMinutes) {
this.importRepository = importRepository;
this.manualGameService = manualGameService;
this.igdbApiService = igdbApiService;
this.timeToBeatService = timeToBeatService;
this.moodService = moodService;
this.stallTime = Duration.ofMinutes(stallMinutes);
@@ -140,11 +138,11 @@ public class GameImportWorker {
Map<Integer, GameImport.Row> rows = importRepository.findRows(importId).stream()
.collect(Collectors.toMap(row -> row.line().lineNo(), row -> row));
for (GameImport.Choice choice : choices) {
IgdbApiService.Game game = rows.get(choice.lineNo()).candidates().stream()
IgdbGame game = rows.get(choice.lineNo()).candidates().stream()
.filter(candidate -> candidate.igdbId() == choice.igdbId()).findFirst()
// A game from a new search in the review. IGDB knew it minutes ago, so no answer is an IGDB
// failure: the try stops and the line waits for the next one.
.or(() -> igdbApiService.getGame(choice.igdbId()))
.or(() -> manualGameService.findGame(choice.igdbId()))
.orElseThrow(() -> new IllegalStateException("IGDB did not answer for game " + choice.igdbId()));
if (Thread.currentThread().isInterrupted()) {
return;
@@ -170,11 +168,11 @@ public class GameImportWorker {
importRepository.markSaved(importId);
}
private record Match(boolean searchFailed, boolean exact, List<IgdbApiService.Game> candidates) {}
private record Match(boolean searchFailed, boolean exact, List<IgdbGame> candidates) {}
private Match match(GameImport.Line line) {
if (line.igdbId() != null) {
Optional<IgdbApiService.Game> game = igdbApiService.getGame(line.igdbId());
Optional<IgdbGame> game = manualGameService.findGame(line.igdbId());
return new Match(game.isEmpty(), game.isPresent(), game.stream().toList());
}
return manualGameService.findGames(line.title())
@@ -185,9 +183,9 @@ public class GameImportWorker {
}
// Same name first, among them one on the line's platform; else IGDB's order.
static List<IgdbApiService.Game> rank(GameImport.Line line, List<IgdbApiService.Game> found) {
Comparator<IgdbApiService.Game> order = Comparator
.comparing((IgdbApiService.Game game) -> !sameName(game.name(), line.title()))
static List<IgdbGame> rank(GameImport.Line line, List<IgdbGame> found) {
Comparator<IgdbGame> order = Comparator
.comparing((IgdbGame game) -> !sameName(game.name(), line.title()))
.thenComparing(game -> line.platform() == null
|| game.platforms().stream().noneMatch(line.platform()::equalsIgnoreCase));
return found.stream().sorted(order).limit(MAX_CANDIDATES).toList();
@@ -0,0 +1,54 @@
package core_service.modules.library.service;
import core_service.modules.library.domain.GameKeys;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.TreeSet;
import java.util.function.Function;
/**
* IGDB finds a Steam game by its appid and a game off Steam by its IGDB id: one call per key kind, answers by appid.
*/
final class IgdbLookup {
/**
* @param answered the appids whose call worked, sorted; an appid without a value in {@code found} has no data
*/
record Answer<T>(Set<Long> answered, Map<Long, T> found) {}
private IgdbLookup() {
}
static <T> Answer<T> byAppid(Collection<Long> appids,
Function<Set<Long>, Optional<Map<Long, T>>> bySteamAppid,
Function<Set<Long>, Optional<Map<Long, T>>> byIgdbId) {
Set<Long> steamAppids = new TreeSet<>();
Set<Long> igdbIds = new TreeSet<>();
for (Long appid : appids) {
if (GameKeys.isSteam(appid)) {
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, T> found = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
bySteamAppid.apply(steamAppids).ifPresent(values -> {
found.putAll(values);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
byIgdbId.apply(igdbIds).ifPresent(values -> {
values.forEach((igdbId, value) -> found.put(GameKeys.ofIgdb(igdbId), value));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
return new Answer<>(answered, found);
}
}
@@ -37,6 +37,7 @@ import java.util.Set;
import java.util.stream.Collectors;
import core_service.modules.library.domain.UserSteamSync.Status;
import core_service.shared.background.BackgroundPools;
/**
* Worker bean. Runs the actual Steam fetch + DB upserts. Separate bean so the Spring proxy applies @Async
@@ -68,7 +69,7 @@ public class LibrarySyncWorker {
@Value("${app.library.zone:Europe/Berlin}")
private ZoneId zone;
@Async
@Async(BackgroundPools.DEFAULT)
public void runSync(Long userId, String steamId) {
sync(userId, steamId);
}
@@ -4,6 +4,7 @@ import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.api.dto.GameSearchDto;
import core_service.modules.library.api.dto.ManualGameUpdate;
import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.SteamGame;
@@ -93,24 +94,22 @@ public class ManualGameService {
// Games already in ToBePlayed first, with their catalog name and image (an auditor's correction included); then
// games added by hand whose name matches (also a duplicate of a game IGDB now knows on Steam, so an auditor can
// merge it); then the rest in IGDB's order. One hit per key.
private List<GameSearchDto.Hit> hits(Long userId, String query, List<IgdbApiService.Game> games) {
private List<GameSearchDto.Hit> hits(Long userId, String query, List<IgdbGame> games) {
Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList());
Map<Long, SteamGame> catalog = steamGameRepository.findAllById(games.stream().map(game -> key(game, targets)).toList())
.stream().collect(Collectors.toMap(SteamGame::getAppid, game -> game));
List<SteamGame> addedByOthers = steamGameRepository.findManualByName(UserGameRepository.containsPattern(query),
PageRequest.of(0, MAX_CATALOG_HITS));
Set<Long> owned = new HashSet<>(userGameRepository.findActiveAppids(userId, Stream.concat(
games.stream().flatMap(game -> possibleKeys(game, targets).stream()),
addedByOthers.stream().map(SteamGame::getAppid)).toList()));
Set<Long> owned = ownedAppids(userId, games, targets, addedByOthers.stream().map(SteamGame::getAppid).toList());
List<GameSearchDto.Hit> inCatalog = new ArrayList<>();
List<GameSearchDto.Hit> rest = new ArrayList<>();
for (IgdbApiService.Game game : games) {
for (IgdbGame game : games) {
long appid = key(game, targets);
SteamGame entry = catalog.get(appid);
GameSearchDto.Hit hit = new GameSearchDto.Hit(appid, game.igdbId(), entry != null ? entry.getName() : game.name(),
game.year(), entry != null && entry.getImageUrl() != null ? entry.getImageUrl() : game.imageUrl(),
fittingPlatforms(game), possibleKeys(game, targets).stream().anyMatch(owned::contains), entry != null);
fittingPlatforms(game), isOwned(game, targets, owned), entry != null);
(entry != null ? inCatalog : rest).add(hit);
}
Stream<GameSearchDto.Hit> others = addedByOthers.stream().map(game -> new GameSearchDto.Hit(game.getAppid(),
@@ -125,7 +124,7 @@ public class ManualGameService {
// Few hits often mean a typo. IGDB's own hits keep their order; the variant hits follow, closest name first.
// The variants go through the same cache, and one that fails adds nothing.
private List<IgdbApiService.Game> withTypoHelp(String query, List<IgdbApiService.Game> found) {
private List<IgdbGame> withTypoHelp(String query, List<IgdbGame> found) {
if (found.size() >= MIN_HITS) {
return found;
}
@@ -137,12 +136,12 @@ public class ManualGameService {
Set<Long> seen = new HashSet<>();
found.forEach(game -> seen.add(game.igdbId()));
List<IgdbApiService.Game> extra = new ArrayList<>();
List<IgdbGame> extra = new ArrayList<>();
for (String variant : variants) {
if (found.size() + extra.size() >= MIN_HITS) {
break;
}
igdbApiService.searchGames(variant).orElse(List.of()).stream()
igdbApiService.searchGames(variant).orElse(List.of()).stream().map(ManualGameService::toIgdbGame)
.filter(game -> seen.add(game.igdbId()))
.forEach(extra::add);
}
@@ -162,7 +161,7 @@ public class ManualGameService {
return new AddResult.SearchUnavailable();
}
// Outside the transaction: no database connection waits for IGDB.
Optional<IgdbApiService.Game> found = igdbApiService.getGame(igdbId);
Optional<IgdbGame> found = findGame(igdbId);
if (found.isEmpty()) {
return new AddResult.UnknownGame();
}
@@ -181,7 +180,7 @@ public class ManualGameService {
* @param importId the import that adds the game, so its undo can remove it; null for a single add. A removed
* row that comes back keeps no import id.
*/
public AddResult addGame(Long userId, IgdbApiService.Game game, String platform, GameStatus status,
public AddResult addGame(Long userId, IgdbGame game, String platform, GameStatus status,
int playtimeMinutes, Long importId) {
AddResult result;
try {
@@ -201,35 +200,34 @@ public class ManualGameService {
/**
* IGDB's hits for a title, with the same typo help as {@link #search}; empty when IGDB failed or is off.
*/
public Optional<List<IgdbApiService.Game>> findGames(String query) {
public Optional<List<IgdbGame>> findGames(String query) {
if (!igdbApiService.isEnabled()) {
return Optional.empty();
}
return igdbApiService.searchGames(query).map(found -> withTypoHelp(query, found));
return igdbApiService.searchGames(query)
.map(found -> withTypoHelp(query, found.stream().map(ManualGameService::toIgdbGame).toList()));
}
/**
* The IGDB ids of the games the user already has, under any of their keys (a merge included).
*/
public Set<Long> ownedIgdbIds(Long userId, Collection<IgdbApiService.Game> games) {
public Set<Long> ownedIgdbIds(Long userId, Collection<IgdbGame> games) {
if (games.isEmpty()) {
return Set.of();
}
Map<Long, Long> targets = mergeRepository.findTargets(games.stream().flatMap(game -> ownKeys(game).stream()).toList());
Set<Long> owned = new HashSet<>(userGameRepository.findActiveAppids(userId,
games.stream().flatMap(game -> possibleKeys(game, targets).stream()).distinct().toList()));
return games.stream().filter(game -> possibleKeys(game, targets).stream().anyMatch(owned::contains))
.map(IgdbApiService.Game::igdbId).collect(Collectors.toSet());
Set<Long> owned = ownedAppids(userId, games, targets, List.of());
return games.stream().filter(game -> isOwned(game, targets, owned))
.map(IgdbGame::igdbId).collect(Collectors.toSet());
}
private AddResult addInTransaction(Long userId, IgdbApiService.Game game, String platform, GameStatus status,
private AddResult addInTransaction(Long userId, IgdbGame game, String platform, GameStatus status,
int playtimeMinutes, Long importId) {
return transaction.execute(tx -> {
Instant now = Instant.now();
Map<Long, Long> targets = mergeRepository.findTargets(ownKeys(game));
long appid = key(game, targets);
List<Long> possibleKeys = possibleKeys(game, targets);
if (!userGameRepository.findActiveAppids(userId, possibleKeys).isEmpty()) {
if (!ownedAppids(userId, List.of(game), targets, List.of()).isEmpty()) {
return new AddResult.AlreadyInLibrary();
}
Optional<UserGame> existing = userGameRepository.findById(new UserGameId(userId, appid));
@@ -287,14 +285,11 @@ public class ManualGameService {
// A Steam row belongs to the sync, which has the better name and art; only a game off Steam takes IGDB's, and
// only its own: a merged game must not rename the game it went into.
private void saveCatalogEntry(long appid, IgdbApiService.Game game, Instant now) {
private void saveCatalogEntry(long appid, IgdbGame game, Instant now) {
Optional<SteamGame> existing = steamGameRepository.findById(appid);
if (existing.isEmpty()) {
SteamGame created = new SteamGame(appid, game.name(), null, now);
if (!GameKeys.isSteam(appid)) {
created.setImageUrl(game.imageUrl());
}
steamGameRepository.save(created);
String imageUrl = GameKeys.isSteam(appid) ? null : game.imageUrl();
steamGameRepository.save(new SteamGame(appid, game.name(), null, imageUrl, now));
} else if (appid == GameKeys.ofIgdb(game.igdbId())) {
steamGameRepository.setSourceName(appid, game.name());
steamGameRepository.setImageUrl(appid, game.imageUrl());
@@ -303,25 +298,47 @@ public class ManualGameService {
}
// A longer name would not fit the column; with no platform left, the user types one.
static List<String> fittingPlatforms(IgdbApiService.Game game) {
static List<String> fittingPlatforms(IgdbGame game) {
return game.platforms().stream().filter(platform -> platform.length() <= ManualGameUpdate.MAX_PLATFORM_LENGTH).toList();
}
/**
* One game by its IGDB id; empty when IGDB does not know it or failed.
*/
public Optional<IgdbGame> findGame(long igdbId) {
return igdbApiService.getGame(igdbId).map(ManualGameService::toIgdbGame);
}
private static IgdbGame toIgdbGame(IgdbApiService.Game game) {
return new IgdbGame(game.igdbId(), game.name(), game.year(), game.imageUrl(), game.platforms(), game.steamAppid());
}
// IGDB can learn a game's Steam appid after the user added it under minus its IGDB id. Each key goes through its
// merge, if an auditor merged it.
private static List<Long> possibleKeys(IgdbApiService.Game game, Map<Long, Long> targets) {
private static List<Long> possibleKeys(IgdbGame game, Map<Long, Long> targets) {
return ownKeys(game).stream().map(own -> targets.getOrDefault(own, own)).distinct().toList();
}
private Set<Long> ownedAppids(Long userId, Collection<IgdbGame> games, Map<Long, Long> targets,
Collection<Long> extraAppids) {
return new HashSet<>(userGameRepository.findActiveAppids(userId, Stream.concat(
games.stream().flatMap(game -> possibleKeys(game, targets).stream()), extraAppids.stream())
.distinct().toList()));
}
private static boolean isOwned(IgdbGame game, Map<Long, Long> targets, Set<Long> ownedAppids) {
return possibleKeys(game, targets).stream().anyMatch(ownedAppids::contains);
}
/**
* @param targets {@link GameMergeRepository#findTargets} of the {@link #ownKeys}
*/
private static long key(IgdbApiService.Game game, Map<Long, Long> targets) {
private static long key(IgdbGame game, Map<Long, Long> targets) {
return possibleKeys(game, targets).getFirst();
}
// The key IGDB gives today first, then minus the IGDB id when the game is on Steam.
private static List<Long> ownKeys(IgdbApiService.Game game) {
private static List<Long> ownKeys(IgdbGame game) {
long igdbKey = GameKeys.ofIgdb(game.igdbId());
return game.steamAppid() != null ? List.of(game.steamAppid(), igdbKey) : List.of(igdbKey);
}
@@ -1,23 +1,19 @@
package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.domain.Mood;
import core_service.modules.library.repo.GameMoodsRepository;
import core_service.modules.library.repo.UserGameRepository;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async;
import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service;
import java.time.Duration;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
/**
* Stores the IGDB genres of games and the {@link Mood}s from IGDB's genres, themes and game modes, the same way
@@ -31,40 +27,20 @@ public class MoodService {
private final IgdbApiService igdbApiService;
private final GameMoodsRepository gameMoodsRepository;
private final UserGameRepository userGameRepository;
private final TaskExecutor applicationTaskExecutor;
@Value("${app.library.mood-ttl-days:30}")
private long ttlDays;
public void refresh(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return;
// Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock.
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(gameMoodsRepository.findFreshAppids(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
Set<Long> stale = stale(appids);
if (stale.isEmpty()) return;
Set<Long> steamAppids = new TreeSet<>();
Set<Long> igdbIds = new TreeSet<>();
for (Long appid : stale) {
if (GameKeys.isSteam(appid)) {
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, IgdbApiService.GameTags> tags = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
igdbApiService.getGameTags(steamAppids).ifPresent(found -> {
tags.putAll(found);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
igdbApiService.getGameTagsByGameIds(igdbIds).ifPresent(found -> {
found.forEach((igdbId, gameTags) -> tags.put(GameKeys.ofIgdb(igdbId), gameTags));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
IgdbLookup.Answer<IgdbApiService.GameTags> answer = IgdbLookup.byAppid(stale, igdbApiService::getGameTags,
igdbApiService::getGameTagsByGameIds);
Set<Long> answered = answer.answered();
Map<Long, IgdbApiService.GameTags> tags = answer.found();
if (answered.size() < stale.size()) {
log.warn("Moods of {} games not refreshed, IGDB did not answer", stale.size() - answered.size());
}
@@ -83,36 +59,31 @@ public class MoodService {
/**
* Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it.
*/
@Async
public void refreshInBackground(Collection<Long> appids) {
try {
refresh(appids);
} catch (Exception e) {
log.warn("Moods of {} not refreshed: {}", appids, e.getMessage());
}
if (!igdbApiService.isEnabled()) return;
Refreshes.staleInBackground(applicationTaskExecutor, "Moods of " + appids, () -> appids, this::stale,
this::refresh);
}
/**
* For the genres and moods of the stats page: a user without a Steam sync gets them only here.
*/
@Async
public void refreshListedInBackground(Long userId) {
try {
refresh(userGameRepository.findAllListedAppids(userId));
} catch (Exception e) {
log.warn("Moods of the library of user {} not refreshed: {}", userId, e.getMessage());
}
if (!igdbApiService.isEnabled()) return;
Refreshes.staleInBackground(applicationTaskExecutor, "Moods of the library of user " + userId,
() -> userGameRepository.findAllListedAppids(userId), this::stale, this::refresh);
}
/**
* For the picker: a user without a Steam sync gets moods only here, and a Steam user before the next sync.
*/
@Async
public void refreshBacklogInBackground(Long userId) {
try {
refresh(userGameRepository.findBacklogAppids(userId));
} catch (Exception e) {
log.warn("Backlog moods of user {} not refreshed: {}", userId, e.getMessage());
}
if (!igdbApiService.isEnabled()) return;
Refreshes.staleInBackground(applicationTaskExecutor, "Backlog moods of user " + userId,
() -> userGameRepository.findBacklogAppids(userId), this::stale, this::refresh);
}
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameMoodsRepository::findFreshAppids, ttlDays);
}
}
@@ -0,0 +1,64 @@
package core_service.modules.library.service;
import lombok.extern.slf4j.Slf4j;
import org.springframework.core.task.TaskExecutor;
import java.time.Duration;
import java.time.Instant;
import java.util.Collection;
import java.util.Set;
import java.util.TreeSet;
import java.util.function.BiFunction;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.function.Supplier;
/**
* The shared steps of the IGDB and speedrun.com refreshes: which games have no fresh row, and the background run.
*/
@Slf4j
final class Refreshes {
private Refreshes() {
}
/**
* Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock.
*
* @param findFreshAppids the appids among the given ones with a row newer than the instant
*/
static Set<Long> stale(Collection<Long> appids, BiFunction<Collection<Long>, Instant, Set<Long>> findFreshAppids,
long ttlDays) {
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(findFreshAppids.apply(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
return stale;
}
/**
* Checks on the calling thread which games are stale and refreshes only those in the background, so a fresh game
* takes no place in the queue. No error reaches the caller: the next view or sync asks again.
*
* @param appids read on the calling thread
*/
static void staleInBackground(TaskExecutor executor, String what, Supplier<Collection<Long>> appids,
Function<Collection<Long>, Set<Long>> stale, Consumer<Set<Long>> refresh) {
try {
Collection<Long> candidates = appids.get();
if (candidates.isEmpty()) return;
Set<Long> staleAppids = stale.apply(candidates);
if (staleAppids.isEmpty()) return;
// ponytail: views of one stale game in the same second ask once each; an in-flight set if that hurts
executor.execute(() -> quietly(what, () -> refresh.accept(staleAppids)));
} catch (Exception e) {
log.warn("{} not refreshed: {}", what, e.getMessage());
}
}
static void quietly(String what, Runnable refresh) {
try {
refresh.run();
} catch (Exception e) {
log.warn("{} not refreshed: {}", what, e.getMessage());
}
}
}
@@ -8,9 +8,11 @@ import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.shared.background.BackgroundPools;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import org.springframework.transaction.event.TransactionalEventListener;
@@ -46,6 +48,7 @@ public class SpeedrunService {
private final GameSpeedrunRepository gameSpeedrunRepository;
private final SteamGameRepository steamGameRepository;
private final UserGameRepository userGameRepository;
private final TaskExecutor syncFollowUpExecutor;
private final ConcurrentSkipListSet<Long> pending = new ConcurrentSkipListSet<>();
private final AtomicBoolean running = new AtomicBoolean();
@@ -71,28 +74,27 @@ public class SpeedrunService {
}
}
@Async
@Async(BackgroundPools.SYNC_FOLLOW_UP)
@TransactionalEventListener
public void onLibrarySynced(LibrarySyncedEvent event) {
refreshBacklogInBackground(event.userId());
Refreshes.quietly("Speedrun games of the backlog of user " + event.userId(),
() -> refresh(userGameRepository.findBacklogAppids(event.userId())));
}
@Async
public void refreshBacklogInBackground(Long userId) {
try {
refresh(userGameRepository.findBacklogAppids(userId));
} catch (Exception e) {
log.warn("Speedrun games of the backlog of user {} not refreshed: {}", userId, e.getMessage());
}
if (!speedrunApiService.isEnabled()) return;
Refreshes.staleInBackground(syncFollowUpExecutor, "Speedrun games of the backlog of user " + userId,
() -> userGameRepository.findBacklogAppids(userId), this::stale, this::refresh);
}
@Async
public void refreshInBackground(Collection<Long> appids) {
try {
refresh(appids);
} catch (Exception e) {
log.warn("Speedrun games of {} not refreshed: {}", appids, e.getMessage());
}
if (!speedrunApiService.isEnabled()) return;
Refreshes.staleInBackground(syncFollowUpExecutor, "Speedrun games of " + appids, () -> appids, this::stale,
this::refresh);
}
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameSpeedrunRepository::findFreshAppids, ttlDays);
}
private void drain() {
@@ -1,21 +1,17 @@
package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameKeys;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async;
import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service;
import java.time.Duration;
import java.time.Instant;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
@Service
@RequiredArgsConstructor
@@ -24,20 +20,19 @@ public class TimeToBeatService {
private final IgdbApiService igdbApiService;
private final GameTimeToBeatRepository gameTimeToBeatRepository;
private final TaskExecutor applicationTaskExecutor;
@Value("${app.library.time-to-beat-ttl-days:30}")
private long ttlDays;
/**
* Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it.
* Same as {@link #refresh}, but the request does not wait for IGDB, and an error does not fail it. Only stale apps
* take a place in the background queue.
*/
@Async
public void refreshInBackground(Collection<Long> appids) {
try {
refresh(appids);
} catch (Exception e) {
log.warn("Time to beat of {} not refreshed: {}", appids, e.getMessage());
}
if (!igdbApiService.isEnabled()) return;
Refreshes.staleInBackground(applicationTaskExecutor, "Time to beat of " + appids, () -> appids, this::stale,
this::refresh);
}
/**
@@ -46,34 +41,13 @@ public class TimeToBeatService {
*/
public void refresh(Collection<Long> appids) {
if (!igdbApiService.isEnabled() || appids.isEmpty()) return;
// Sorted: parallel syncs then lock the upserted rows in the same order and cannot deadlock.
Set<Long> stale = new TreeSet<>(appids);
stale.removeAll(gameTimeToBeatRepository.findFreshAppids(appids, Instant.now().minus(Duration.ofDays(ttlDays))));
Set<Long> stale = stale(appids);
if (stale.isEmpty()) return;
Set<Long> steamAppids = new TreeSet<>();
Set<Long> igdbIds = new TreeSet<>();
for (Long appid : stale) {
if (GameKeys.isSteam(appid)) {
steamAppids.add(appid);
} else {
igdbIds.add(GameKeys.igdbId(appid));
}
}
Map<Long, IgdbApiService.TimeToBeat> times = new HashMap<>();
Set<Long> answered = new TreeSet<>();
if (!steamAppids.isEmpty()) {
igdbApiService.getTimeToBeat(steamAppids).ifPresent(found -> {
times.putAll(found);
answered.addAll(steamAppids);
});
}
if (!igdbIds.isEmpty()) {
igdbApiService.getTimeToBeatByGameIds(igdbIds).ifPresent(found -> {
found.forEach((igdbId, time) -> times.put(GameKeys.ofIgdb(igdbId), time));
igdbIds.forEach(igdbId -> answered.add(GameKeys.ofIgdb(igdbId)));
});
}
IgdbLookup.Answer<IgdbApiService.TimeToBeat> answer = IgdbLookup.byAppid(stale, igdbApiService::getTimeToBeat,
igdbApiService::getTimeToBeatByGameIds);
Set<Long> answered = answer.answered();
Map<Long, IgdbApiService.TimeToBeat> times = answer.found();
if (answered.size() < stale.size()) {
log.warn("Time to beat for {} apps not refreshed, IGDB did not answer", stale.size() - answered.size());
}
@@ -87,4 +61,8 @@ public class TimeToBeatService {
}
}
}
private Set<Long> stale(Collection<Long> appids) {
return Refreshes.stale(appids, gameTimeToBeatRepository::findFreshAppids, ttlDays);
}
}
@@ -10,8 +10,7 @@ import java.util.Map;
@Repository
public class StatsRepository {
private static final String LISTED =
"removed_at IS NULL AND hidden_at IS NULL AND user_id NOT IN (SELECT user_id FROM deleted_user)";
private static final String NOT_DELETED = "user_id NOT IN (SELECT user_id FROM deleted_user)";
private final JdbcTemplate jdbc;
@@ -22,77 +21,75 @@ public class StatsRepository {
public long countDistinctGames() {
Long v = jdbc.queryForObject(
"SELECT COUNT(DISTINCT appid) FROM user_game WHERE " + LISTED, Long.class);
"SELECT COUNT(DISTINCT appid) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L;
}
public long countLibraryEntries() {
Long v = jdbc.queryForObject(
"SELECT COUNT(*) FROM user_game WHERE " + LISTED, Long.class);
"SELECT COUNT(*) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L;
}
public long countGamesPlayed() {
Long v = jdbc.queryForObject(
"SELECT COUNT(*) FROM user_game WHERE " + LISTED + " AND playtime_minutes > 0", Long.class);
"SELECT COUNT(*) FROM listed_user_game WHERE " + NOT_DELETED + " AND playtime_minutes > 0", Long.class);
return v != null ? v : 0L;
}
public long sumPlaytimeMinutes() {
Long v = jdbc.queryForObject(
"SELECT COALESCE(SUM(playtime_minutes), 0) FROM user_game WHERE " + LISTED, Long.class);
"SELECT COALESCE(SUM(playtime_minutes), 0) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L;
}
public long countUsersWithLibrary() {
Long v = jdbc.queryForObject(
"SELECT COUNT(DISTINCT user_id) FROM user_game WHERE " + LISTED, Long.class);
"SELECT COUNT(DISTINCT user_id) FROM listed_user_game WHERE " + NOT_DELETED, Long.class);
return v != null ? v : 0L;
}
public long biggestLibrary() {
Long v = jdbc.queryForObject("""
SELECT COALESCE(MAX(c), 0) FROM (
SELECT COUNT(*) AS c FROM user_game
SELECT COUNT(*) AS c FROM listed_user_game
WHERE %s GROUP BY user_id
) t
""".formatted(LISTED), Long.class);
""".formatted(NOT_DELETED), Long.class);
return v != null ? v : 0L;
}
public long biggestBacklog() {
Long v = jdbc.queryForObject("""
SELECT COALESCE(MAX(c), 0) FROM (
SELECT COUNT(*) AS c FROM user_game
SELECT COUNT(*) AS c FROM listed_user_game
WHERE %s AND playtime_minutes = 0 GROUP BY user_id
) t
""".formatted(LISTED), Long.class);
""".formatted(NOT_DELETED), Long.class);
return v != null ? v : 0L;
}
public List<Map<String, Object>> topOwnedGames(int limit) {
return jdbc.queryForList("""
SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, COUNT(*) AS value
FROM user_game ug
FROM listed_user_game ug
LEFT JOIN steam_game sg ON sg.appid = ug.appid
WHERE ug.removed_at IS NULL AND ug.hidden_at IS NULL
AND ug.user_id NOT IN (SELECT user_id FROM deleted_user)
WHERE %s
GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url
ORDER BY value DESC
LIMIT ?
""", limit);
""".formatted(NOT_DELETED), limit);
}
public List<Map<String, Object>> topPlayedGames(int limit) {
return jdbc.queryForList("""
SELECT ug.appid, sg.name, sg.img_icon_url, sg.image_url, SUM(ug.playtime_minutes) AS value
FROM user_game ug
FROM listed_user_game ug
LEFT JOIN steam_game sg ON sg.appid = ug.appid
WHERE ug.removed_at IS NULL AND ug.hidden_at IS NULL
AND ug.user_id NOT IN (SELECT user_id FROM deleted_user)
WHERE %s
GROUP BY ug.appid, sg.name, sg.img_icon_url, sg.image_url
ORDER BY value DESC
LIMIT ?
""", limit);
""".formatted(NOT_DELETED), limit);
}
}
@@ -0,0 +1,22 @@
package core_service.shared.background;
import org.springframework.boot.autoconfigure.task.TaskExecutionAutoConfiguration;
/**
* The bean names of the two background pools (configuration.AsyncConfig). Every {@code @Async} names one of them:
* with the scheduler there are two TaskExecutor beans, and a bare {@code @Async} would run unbounded.
*/
public final class BackgroundPools {
/** Syncs and the refreshes a page view starts: short tasks, a full queue drops the task. */
public static final String DEFAULT = TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME;
/**
* The achievement runs after a sync and every speedrun refresh: they hold a thread for minutes (Steam and
* speedrun.com pace), so page views keep the default pool. Small tasks, nothing is dropped.
*/
public static final String SYNC_FOLLOW_UP = "syncFollowUpExecutor";
private BackgroundPools() {
}
}
@@ -10,7 +10,7 @@ import java.lang.annotation.Target;
* The lowest active role that may call this service method, or every method of this class. A method marker
* replaces the class marker. Every service method a controller calls carries one of {@link RequiresRole},
* {@link Public} or {@link Internal} (ArchitectureTest). Checked only while an HTTP request is served: scheduled
* jobs and {@code @Async} work run without a user.
* jobs and background work (the pools in {@code AsyncConfig}) run without a user.
*/
@Documented
@Retention(RetentionPolicy.RUNTIME)
@@ -0,0 +1,6 @@
-- The games a user's lists and numbers show: still owned and not hidden. Native SQL reads this view; JPQL uses
-- UserGameRepository.LISTED. Postgres fixes the columns of "select *" at creation and blocks dropping or retyping a
-- column the view uses: a migration that adds, drops or retypes a column of user_game drops this view first and
-- creates it again.
CREATE VIEW listed_user_game AS
SELECT * FROM user_game WHERE removed_at IS NULL AND hidden_at IS NULL;
@@ -1,14 +1,24 @@
package core_service.architecture;
import core_service.shared.background.BackgroundPools;
import core_service.shared.test.AbstractArchitectureTest;
import com.tngtech.archunit.base.DescribedPredicate;
import com.tngtech.archunit.core.domain.JavaClass;
import com.tngtech.archunit.core.domain.JavaMethod;
import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.junit.AnalyzeClasses;
import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.lang.ArchCondition;
import com.tngtech.archunit.lang.ArchRule;
import com.tngtech.archunit.lang.ConditionEvents;
import com.tngtech.archunit.lang.SimpleConditionEvent;
import com.tngtech.archunit.library.dependencies.SlicesRuleDefinition;
import org.springframework.scheduling.annotation.Async;
import java.util.Set;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.classes;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.methods;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses;
/**
@@ -59,12 +69,57 @@ class ArchitectureTest extends AbstractArchitectureTest {
.should().dependOnClassesThat().resideInAPackage("..api..")
.because("Persistence layer must be independent of the API layer.");
@ArchTest
static final ArchRule domain_should_not_depend_on_other_layers = noClasses()
.that().resideInAPackage("core_service.modules..domain..")
.should().dependOnClassesThat().resideInAnyPackage("..api..", "..service..", "..repo..",
"core_service.modules.steam_adapter..", "core_service.modules.igdb_adapter..",
"core_service.modules.speedrun_adapter..")
.because("Entities and stored shapes must not change when a controller, service or external API changes.");
// The listed services build their module's response DTOs; the list only gets shorter.
@ArchTest
static final ArchRule services_should_not_depend_on_apis = noClasses()
.that().resideInAPackage("core_service.modules..service..")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.BacklogService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.GameDetailService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.GameImportService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.LibraryQueryService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.ManualGameService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.SteamProfileService")
.and().doNotHaveFullyQualifiedName("core_service.modules.library.service.UserStatsService")
.and().doNotHaveFullyQualifiedName("core_service.modules.stats.service.PublicStatsService")
.should().dependOnClassesThat(JavaClass.Predicates.resideInAPackage("..api..")
.and(DescribedPredicate.not(JavaClass.Predicates.resideInAnyPackage(
"core_service.modules.steam_adapter.api.dto.response..",
"core_service.modules.igdb_adapter.api.dto.response..",
"core_service.modules.speedrun_adapter.api.dto.response.."))))
.because("Services must not depend on the HTTP layer; the adapters' response DTOs are the external APIs' shapes.");
@ArchTest
static final ArchRule repos_should_be_interfaces_named_repository = classes()
.that().resideInAPackage("..repo..")
.should().haveSimpleNameEndingWith("Repository")
.because("Repository interfaces are the persistence boundary; naming makes them explicit.");
@ArchTest
static final ArchRule async_methods_should_name_a_pool = methods()
.that().areAnnotatedWith(Async.class)
.should(new ArchCondition<>("name a pool of BackgroundPools") {
@Override
public void check(JavaMethod method, ConditionEvents events) {
String pool = method.getAnnotationOfType(Async.class).value();
boolean named = Set.of(BackgroundPools.DEFAULT, BackgroundPools.SYNC_FOLLOW_UP).contains(pool);
events.add(new SimpleConditionEvent(method, named, method.getFullName() + " runs on '" + pool + "'"));
}
})
.because("A bare @Async runs on an unbounded executor (BackgroundPools).");
@ArchTest
static final ArchRule no_class_level_async = noClasses()
.should().beAnnotatedWith(Async.class)
.because("@Async on a class names no pool for its methods; put it on each method.");
// ===================================================================================
// 3. SHARED / CONFIGURATION RULES
// ===================================================================================
@@ -0,0 +1,72 @@
package core_service.configuration;
import core_service.shared.test.AbstractUnitTest;
import org.junit.jupiter.api.Test;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
class AsyncConfigTest extends AbstractUnitTest {
@Test
void fullQueue_DropsTheTaskWithoutFailingTheCaller() throws InterruptedException {
ThreadPoolTaskExecutor executor = new AsyncConfig().applicationTaskExecutor();
executor.initialize();
CountDownLatch release = new CountDownLatch(1);
CountDownLatch started = new CountDownLatch(AsyncConfig.THREADS);
AtomicBoolean droppedRan = new AtomicBoolean();
try {
for (int i = 0; i < AsyncConfig.THREADS; i++) {
executor.execute(() -> {
started.countDown();
awaitQuietly(release);
});
}
assertTrue(started.await(5, TimeUnit.SECONDS));
for (int i = 0; i < AsyncConfig.QUEUE; i++) {
executor.execute(() -> { });
}
assertDoesNotThrow(() -> executor.execute(() -> droppedRan.set(true)));
assertEquals(AsyncConfig.QUEUE, executor.getQueueSize());
} finally {
release.countDown();
executor.shutdown();
}
assertTrue(executor.getThreadPoolExecutor().awaitTermination(5, TimeUnit.SECONDS));
assertFalse(droppedRan.get());
}
@Test
void followUpPool_KeepsEveryTask() throws InterruptedException {
ThreadPoolTaskExecutor executor = new AsyncConfig().syncFollowUpExecutor();
executor.initialize();
int tasks = AsyncConfig.QUEUE * 3;
CountDownLatch done = new CountDownLatch(tasks);
try {
for (int i = 0; i < tasks; i++) {
executor.execute(done::countDown);
}
assertTrue(done.await(5, TimeUnit.SECONDS));
} finally {
executor.shutdown();
}
}
private static void awaitQuietly(CountDownLatch latch) {
try {
latch.await(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
@@ -1,6 +1,6 @@
package core_service.configuration.security;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
@@ -16,7 +16,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile logs every request in, so only the CSRF header and CORS decide here.
*/
@AutoConfigureMockMvc
class ClientHeaderFilterTest extends AbstractIntegrationTest {
class ClientHeaderFilterTest extends AbstractMockedAdaptersTest {
private static final String FRONTEND = "https://tbp.andreas-dahm.eu";
@@ -48,6 +48,13 @@ class ClientHeaderFilterTest extends AbstractIntegrationTest {
.andExpect(header().string("Access-Control-Allow-Origin", FRONTEND))
.andExpect(header().string("Access-Control-Allow-Credentials", "true"));
mockMvc.perform(options("/api/library/games/220/backlog")
.header("Origin", FRONTEND)
.header("Access-Control-Request-Method", "PATCH")
.header("Access-Control-Request-Headers", "x-tbp-client,content-type"))
.andExpect(status().isOk())
.andExpect(header().string("Access-Control-Allow-Origin", FRONTEND));
mockMvc.perform(options("/api/library/sync")
.header("Origin", "https://evil.andreas-dahm.eu")
.header("Access-Control-Request-Method", "POST"))
@@ -118,8 +118,11 @@ class AccountCleanupRepositoryTest extends AbstractPersistenceTest {
// A new table with user rows must be in the purge list, or "Delete my account" leaves its rows behind.
@Test
void tables_CoverEveryTableWithAUserId() {
List<String> withUserId = jdbc.queryForList("select table_name from information_schema.columns "
+ "where table_schema = 'public' and column_name = 'user_id' order by table_name", String.class);
// Base tables only: a view like listed_user_game has no rows of its own.
List<String> withUserId = jdbc.queryForList("select c.table_name from information_schema.columns c "
+ "join information_schema.tables t on t.table_schema = c.table_schema and t.table_name = c.table_name "
+ "where c.table_schema = 'public' and c.column_name = 'user_id' and t.table_type = 'BASE TABLE' "
+ "order by c.table_name", String.class);
assertThat(AccountCleanupRepository.TABLES).containsExactlyInAnyOrderElementsOf(withUserId);
}
@@ -8,7 +8,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.GameImportWorker;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import com.jayway.jsonpath.JsonPath;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -16,7 +16,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;
@@ -48,7 +47,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class GameImportControllerTest extends AbstractIntegrationTest {
class GameImportControllerTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023,
@@ -70,9 +69,6 @@ class GameImportControllerTest extends AbstractIntegrationTest {
@Autowired
private JdbcTemplate jdbc;
@MockitoBean
private IgdbApiService igdbApiService;
@BeforeEach
void seed() {
jdbc.update("delete from game_alias");
@@ -104,7 +100,7 @@ class GameImportControllerTest extends AbstractIntegrationTest {
awaitStatus(id, "READY");
}
// The matching and the save run in the background.
// The matching and the save run on the worker's own lane threads, not on the test executor, so the test polls.
private void awaitStatus(long id, String expected) throws InterruptedException {
for (int attempt = 0; attempt < 50; attempt++) {
if (expected.equals(jdbc.queryForObject("select status from game_import where id = ?", String.class, id))) {
@@ -146,6 +142,23 @@ class GameImportControllerTest extends AbstractIntegrationTest {
USER_ID, appid);
}
@Test
void get_ReadyImportStoredBeforeIgdbGame_ReadsItsCandidates() throws Exception {
long id = insertImport(USER_ID, "READY", "total = 1");
// The JSON the import wrote with IgdbApiService.Game, before the own record IgdbGame.
jdbc.update("insert into game_import_row (import_id, line_no, title, matched_at, exact, candidates) "
+ "values (?, 1, 'Zelda', now(), true, cast(? as jsonb))", id,
"[{\"igdbId\":119388,\"name\":\"Zelda: Tears of the Kingdom\",\"year\":2023,"
+ "\"imageUrl\":null,\"platforms\":[\"Switch\"],\"steamAppid\":null}]");
mockMvc.perform(get("/api/library/imports/" + id))
.andExpect(status().isOk())
.andExpect(jsonPath("$.rows[0].candidates[0].igdbId").value(119388))
.andExpect(jsonPath("$.rows[0].candidates[0].name").value("Zelda: Tears of the Kingdom"))
.andExpect(jsonPath("$.rows[0].candidates[0].year").value(2023))
.andExpect(jsonPath("$.rows[0].candidates[0].platforms[0]").value("Switch"));
}
@Test
void create_MatchesInTheBackground_SameNameFirstAndMarksGamesInTheLibrary() throws Exception {
jdbc.update("insert into steam_game (appid, name) values (1245620, 'ELDEN RING') on conflict do nothing");
@@ -10,13 +10,11 @@ import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncWorker;
import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant;
@@ -27,7 +25,6 @@ import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.nullValue;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.timeout;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
@@ -38,7 +35,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
class LibraryControllerAchievementsTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final String REPAIRED_COVER = "https://shared.akamai.steamstatic.com/store_item_assets/steam/apps/1/x/header.jpg";
@@ -46,9 +43,6 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
@Autowired
private MockMvc mockMvc;
@MockitoBean
private SteamApiService steamApiService;
@Autowired
private SteamGameRepository steamGameRepository;
@@ -107,9 +101,7 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
game(70L, 8, null, true); // unlocks never fetched
game(80L, 10, 12, true); // schema lost achievements, still perfect
game(90L, 6, 0, true); // not started
SteamGame repaired = steamGameRepository.findById(20L).orElseThrow();
repaired.setImageUrl(REPAIRED_COVER);
steamGameRepository.save(repaired);
steamGameRepository.setImageUrl(20L, REPAIRED_COVER);
steamGameRepository.save(new SteamGame(95L, "Game 95", null, Instant.now()));
UserGame removed = new UserGame(USER_ID, 95L, 60, Instant.now(), Instant.now());
removed.setRemovedAt(Instant.now());
@@ -163,6 +155,6 @@ class LibraryControllerAchievementsTest extends AbstractIntegrationTest {
librarySyncWorker.runSyncNow(USER_ID, "76561198000000001");
verify(steamApiService, timeout(5_000)).getSchemaForGame(620L);
verify(steamApiService).getSchemaForGame(620L);
}
}
@@ -53,8 +53,7 @@ class LibraryControllerActivityTest extends AbstractIntegrationTest {
void seed() {
userGameRepository.deleteAll();
jdbc.update("delete from user_game_playtime_day");
SteamGame repairedPortal = new SteamGame(620L, "Portal 2", null, Instant.now());
repairedPortal.setImageUrl(REPAIRED_COVER);
SteamGame repairedPortal = new SteamGame(620L, "Portal 2", null, REPAIRED_COVER, Instant.now());
steamGameRepository.saveAll(List.of(
new SteamGame(220L, "Half-Life 2", null, Instant.now()),
repairedPortal,
@@ -17,11 +17,15 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;
import java.time.Instant;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.patch;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.put;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
@@ -138,6 +142,87 @@ class LibraryControllerBacklogTest extends AbstractIntegrationTest {
.andExpect(status().isNotFound());
}
private ResultActions patchBacklog(long appid, String body) throws Exception {
return mockMvc.perform(patch("/api/library/games/" + appid + "/backlog").header(ClientHeaderFilter.CLIENT_HEADER, "test")
.contentType(MediaType.APPLICATION_JSON).content(body));
}
private void storeAllFields() {
UserGame game = stored();
game.setRating(4);
game.setNotes("old notes");
game.setPriority(GamePriority.HIGH);
userGameRepository.save(game);
}
@Test
void patchBacklog_OnlyNotes_KeepsTheOtherFieldsAndTheAutomaticStatus() throws Exception {
storeAllFields();
patchBacklog(APPID, "{\"notes\":\"new notes\"}")
.andExpect(status().isOk())
.andExpect(jsonPath("$.notes").value("new notes"))
.andExpect(jsonPath("$.rating").value(4));
UserGame game = stored();
assertEquals("new notes", game.getNotes());
assertEquals(4, game.getRating());
assertEquals(GamePriority.HIGH, game.getPriority());
assertEquals(GameStatus.BACKLOG, game.getStatus());
assertFalse(game.isStatusSetByUser());
}
@Test
void patchBacklog_Status_IsSetByTheUser() throws Exception {
patchBacklog(APPID, "{\"status\":\"PLAYING\"}").andExpect(status().isOk());
assertEquals(GameStatus.PLAYING, stored().getStatus());
assertTrue(stored().isStatusSetByUser());
}
@Test
void patchBacklog_NullClearsAnOptionalFieldAndBlankNotesAreNull() throws Exception {
storeAllFields();
patchBacklog(APPID, "{\"rating\":null,\"priority\":null,\"notes\":\" \"}").andExpect(status().isOk());
UserGame game = stored();
assertNull(game.getRating());
assertNull(game.getPriority());
assertNull(game.getNotes());
}
@ParameterizedTest
@ValueSource(strings = {
"{\"status\":null}",
"{\"status\":\"WISHLIST\"}",
"{\"rating\":0}",
"{\"rating\":6}",
"{\"rating\":4.7}",
"{\"priority\":\"URGENT\"}"})
void patchBacklog_InvalidBody_Is400AndStoresNothing(String body) throws Exception {
storeAllFields();
patchBacklog(APPID, body).andExpect(status().isBadRequest());
UserGame game = stored();
assertEquals(GameStatus.BACKLOG, game.getStatus());
assertEquals(4, game.getRating());
assertEquals(GamePriority.HIGH, game.getPriority());
}
@Test
void patchBacklog_NotesTooLong_Is400() throws Exception {
patchBacklog(APPID, "{\"notes\":\"" + "x".repeat(2001) + "\"}").andExpect(status().isBadRequest());
}
@Test
void patchBacklog_GameNotOwned_Is404() throws Exception {
patchBacklog(999L, "{\"notes\":\"x\"}")
.andExpect(status().isNotFound())
.andExpect(jsonPath("$.code").value("GAME_NOT_FOUND"));
}
@Test
void updateStatus_KeepsRatingNotesAndPriority() throws Exception {
UserGame game = stored();
@@ -7,7 +7,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
@@ -34,7 +34,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class LibraryControllerHiddenTest extends AbstractIntegrationTest {
class LibraryControllerHiddenTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
@@ -11,19 +11,14 @@ import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncService;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedLibraryServicesTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant;
import java.util.List;
import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.hamcrest.Matchers.hasSize;
@@ -37,7 +32,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class LibraryControllerPickerTest extends AbstractIntegrationTest {
class LibraryControllerPickerTest extends AbstractMockedLibraryServicesTest {
private static final long USER_ID = 1L;
@@ -56,15 +51,6 @@ class LibraryControllerPickerTest extends AbstractIntegrationTest {
@Autowired
private GameMoodsRepository gameMoodsRepository;
@MockitoBean
private LibrarySyncService syncService;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@Autowired
private GameSpeedrunRepository gameSpeedrunRepository;
@@ -7,10 +7,7 @@ import core_service.modules.library.domain.UserGame;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.library.service.LibrarySyncService;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedLibraryServicesTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
@@ -20,7 +17,6 @@ import org.springframework.boot.test.system.CapturedOutput;
import org.springframework.boot.test.system.OutputCaptureExtension;
import org.springframework.http.converter.HttpMessageNotWritableException;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import java.sql.SQLException;
@@ -43,7 +39,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class LibraryControllerStatsTest extends AbstractIntegrationTest {
class LibraryControllerStatsTest extends AbstractMockedLibraryServicesTest {
private static final long USER_ID = 1L;
@@ -59,16 +55,6 @@ class LibraryControllerStatsTest extends AbstractIntegrationTest {
@Autowired
private GameTimeToBeatRepository gameTimeToBeatRepository;
// The same mocks as LibraryControllerPickerTest, so both share one Spring context.
@MockitoBean
private LibrarySyncService syncService;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@BeforeEach
void seed() {
userGameRepository.deleteAll();
@@ -1,12 +1,10 @@
package core_service.modules.library.api;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerSummaryResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import java.util.Optional;
@@ -21,14 +19,11 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1 with Steam ID 76561198000000001.
*/
@AutoConfigureMockMvc
class LibraryControllerSteamProfileTest extends AbstractIntegrationTest {
class LibraryControllerSteamProfileTest extends AbstractMockedAdaptersTest {
@Autowired
private MockMvc mockMvc;
@MockitoBean
private SteamApiService steamApiService;
@Test
void steamProfile_SendsNameAndAvatar() throws Exception {
when(steamApiService.getPlayerSummary("76561198000000001")).thenReturn(Optional.of(new SteamPlayerSummaryResponse.Player(
@@ -4,14 +4,13 @@ import core_service.configuration.security.ClientHeaderFilter;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.GameSource;
import core_service.modules.library.domain.GameStatus;
import core_service.modules.library.domain.GameTimeToBeat;
import core_service.modules.library.domain.SteamGame;
import core_service.modules.library.domain.UserGame;
import core_service.modules.library.domain.UserGameId;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import core_service.shared.test.ContractTokens;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -19,7 +18,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc;
import org.springframework.http.MediaType;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import java.time.Instant;
@@ -48,7 +46,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
* The dev profile authenticates every request as user 1.
*/
@AutoConfigureMockMvc
class ManualGameControllerTest extends AbstractIntegrationTest {
class ManualGameControllerTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final IgdbApiService.Game ZELDA = new IgdbApiService.Game(119388, "Zelda: Tears of the Kingdom", 2023,
@@ -71,9 +69,6 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
@Autowired
private JdbcTemplate jdbc;
@MockitoBean
private IgdbApiService igdbApiService;
@BeforeEach
void seed() {
jdbc.update("delete from game_alias");
@@ -115,19 +110,7 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
SteamGame catalog = steamGameRepository.findById(-119388L).orElseThrow();
assertEquals("Zelda: Tears of the Kingdom", catalog.getName());
assertEquals(ZELDA.imageUrl(), catalog.getImageUrl());
assertEquals(3600, timeToBeatStored(-119388L).getNormallyMinutes());
}
// The add asks IGDB for the time to beat in the background.
private GameTimeToBeat timeToBeatStored(long appid) throws InterruptedException {
for (int attempt = 0; attempt < 50; attempt++) {
Optional<GameTimeToBeat> found = gameTimeToBeatRepository.findById(appid);
if (found.isPresent()) {
return found.get();
}
Thread.sleep(100);
}
throw new AssertionError("No time to beat stored for appid " + appid);
assertEquals(3600, gameTimeToBeatRepository.findById(-119388L).orElseThrow().getNormallyMinutes());
}
@Test
@@ -333,7 +316,7 @@ class ManualGameControllerTest extends AbstractIntegrationTest {
@Test
void gameAddedByHand_ShowsUpInTheLibraryAndItsDetail() throws Exception {
add(119388, "Switch");
timeToBeatStored(-119388L);
assertTrue(gameTimeToBeatRepository.findById(-119388L).isPresent());
mockMvc.perform(get("/api/library/games").param("q", "zelda"))
.andExpect(status().isOk())
@@ -0,0 +1,23 @@
package core_service.modules.library.repo;
import core_service.shared.test.AbstractPersistenceTest;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import static org.junit.jupiter.api.Assertions.assertEquals;
class ListedUserGameViewTest extends AbstractPersistenceTest {
private static final String COLUMNS = "select column_name from information_schema.columns "
+ "where table_schema = 'public' and table_name = ? order by ordinal_position";
@Autowired
private JdbcTemplate jdbc;
@Test
void view_HasEveryColumnOfUserGame() {
assertEquals(jdbc.queryForList(COLUMNS, String.class, "user_game"),
jdbc.queryForList(COLUMNS, String.class, "listed_user_game"));
}
}
@@ -124,6 +124,29 @@ class UserGameRepositoryTest extends AbstractPersistenceTest {
assertEquals(Set.of(220L, 380L), Set.copyOf(userGameRepository.findBacklogAppids(USER_ID)));
}
@Test
void findListedSteamGames_LeavesOutHiddenRemovedAndHandAddedGames() {
userGameRepository.findById(new UserGameId(USER_ID, 380L)).orElseThrow().setHiddenAt(Instant.now());
userGameRepository.findById(new UserGameId(USER_ID, 700L)).orElseThrow().setRemovedAt(Instant.now());
steamGameRepository.save(new SteamGame(999L, "Elden Ring", null, Instant.now()));
userGameRepository.save(UserGame.manual(USER_ID, 999L, "PS5", GameStatus.PLAYING, 60, Instant.now()));
userGameRepository.flush();
assertEquals(Set.of(220L, 620L),
Set.copyOf(userGameRepository.findListedSteamGames(USER_ID).stream().map(UserGame::getAppid).toList()));
}
@Test
void listedUserGameView_LeavesOutHiddenAndRemovedGames() {
userGameRepository.findById(new UserGameId(USER_ID, 380L)).orElseThrow().setHiddenAt(Instant.now());
userGameRepository.findById(new UserGameId(USER_ID, 700L)).orElseThrow().setRemovedAt(Instant.now());
userGameRepository.flush();
assertEquals(Set.of(220L, 620L), Set.copyOf(entityManager.createNativeQuery(
"select appid from listed_user_game where user_id = :userId", Long.class)
.setParameter("userId", USER_ID).getResultList()));
}
@Test
void findAchievementProgress_LeavesOutGamesAddedByHand() {
steamGameRepository.save(new SteamGame(999L, "Elden Ring", null, Instant.now()));
@@ -3,13 +3,11 @@ package core_service.modules.library.service;
import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameAchievementRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
@@ -28,7 +26,7 @@ import static org.mockito.Mockito.when;
* The game detail page refreshes inside its own @Transactional while the background sync refreshes the same
* (user, appid) in a transaction of its own, e.g. right after the user pressed "sync" and opened the most played game.
*/
class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
class AchievementSyncServiceConcurrencyTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001";
@@ -45,9 +43,6 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
@Autowired
private PlatformTransactionManager transactionManager;
@MockitoBean
private SteamApiService steamApiService;
@BeforeEach
@AfterEach
void clean() {
@@ -61,20 +56,20 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
new SteamPlayerAchievementsResponse.PlayerStats(STEAM_ID, "Portal 2",
List.of(new SteamPlayerAchievementsResponse.Achievement("A", 1, 1_700_000_000L)), true, null)));
CountDownLatch firstWrote = new CountDownLatch(1);
CountDownLatch secondStarted = new CountDownLatch(1);
CountDownLatch secondDone = new CountDownLatch(1);
CompletableFuture<Void> detailPage = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager)
.executeWithoutResult(tx -> {
achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1));
firstWrote.countDown();
await(secondStarted);
// Keep the page's transaction open while the background run writes the same rows.
sleep();
// The page's transaction stays open until the background run wrote the same rows; a run that
// waited for this transaction would time out the latch.
await(secondDone);
}));
CompletableFuture<Void> backgroundRun = CompletableFuture.runAsync(() -> {
await(firstWrote);
secondStarted.countDown();
achievementSyncService.refreshUserIfStale(USER_ID, STEAM_ID, 620L, 60, Duration.ofHours(1));
secondDone.countDown();
});
assertDoesNotThrow(() -> CompletableFuture.allOf(detailPage, backgroundRun).get(30, TimeUnit.SECONDS));
@@ -90,11 +85,4 @@ class AchievementSyncServiceConcurrencyTest extends AbstractIntegrationTest {
}
}
private static void sleep() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
@@ -96,7 +96,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
}
private void library(List<UserGame> games, List<GameAchievementSchemaMeta> schemas, List<UserGameAchievementMeta> metas) {
when(userGameRepository.findByUserIdAndRemovedAtIsNull(USER_ID)).thenReturn(games);
when(userGameRepository.findListedSteamGames(USER_ID)).thenReturn(games);
when(schemaMetaRepository.findAllById(any())).thenReturn(schemas);
when(userAchievementMetaRepository.findByUserId(USER_ID)).thenReturn(metas);
}
@@ -117,28 +117,6 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
verify(userAchievementMetaRepository, never()).save(any());
}
@Test
void syncLibrary_GameAddedByHand_AsksSteamNothing() {
UserGame manual = UserGame.manual(USER_ID, 620L, "PS5", core_service.modules.library.domain.GameStatus.PLAYING,
150, Instant.now());
library(List.of(manual), List.of(schema(620L, 10)), List.of());
service.syncLibrary(USER_ID, STEAM_ID);
verifyNoInteractions(steamApiService);
}
@Test
void syncLibrary_HiddenGame_AsksSteamNothing() {
UserGame hidden = game(620L, 120);
hidden.setHiddenAt(Instant.now());
library(List.of(hidden), List.of(), List.of());
service.syncLibrary(USER_ID, STEAM_ID);
verifyNoInteractions(steamApiService);
}
@Test
void syncLibrary_ChangedPlaytime_StoresUnlocksAndThePlaytime() {
library(List.of(game(620L, 150)), List.of(schema(620L, 10)), List.of(unlocks(620L, 3, 120)));
@@ -358,7 +336,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
@Test
void syncLibrary_CallWhileRunning_RunsOnceMoreAfterwards() throws Exception {
when(userGameRepository.findByUserIdAndRemovedAtIsNull(USER_ID)).thenReturn(List.of(game(620L, 60)));
when(userGameRepository.findListedSteamGames(USER_ID)).thenReturn(List.of(game(620L, 60)));
when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10)));
// The second run sees the row the first run stored.
when(userAchievementMetaRepository.findByUserId(USER_ID))
@@ -378,7 +356,7 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
first.join(5_000);
verify(steamApiService, times(1)).getPlayerAchievements(STEAM_ID, 620L);
verify(userGameRepository, times(2)).findByUserIdAndRemovedAtIsNull(USER_ID);
verify(userGameRepository, times(2)).findListedSteamGames(USER_ID);
}
@Test
@@ -399,8 +377,8 @@ class AchievementSyncServiceTest extends AbstractUnitTest {
AchievementSyncService paced = new AchievementSyncService(userGameRepository, schemaRepository, schemaMetaRepository,
userAchievementRepository, userAchievementMetaRepository, steamApiService,
mock(PlatformTransactionManager.class), 30, pauseMillis);
when(userGameRepository.findByUserIdAndRemovedAtIsNull(1L)).thenReturn(List.of(new UserGame(1L, 620L, 60, Instant.now(), Instant.now())));
when(userGameRepository.findByUserIdAndRemovedAtIsNull(2L)).thenReturn(List.of(new UserGame(2L, 620L, 60, Instant.now(), Instant.now())));
when(userGameRepository.findListedSteamGames(1L)).thenReturn(List.of(new UserGame(1L, 620L, 60, Instant.now(), Instant.now())));
when(userGameRepository.findListedSteamGames(2L)).thenReturn(List.of(new UserGame(2L, 620L, 60, Instant.now(), Instant.now())));
when(schemaMetaRepository.findAllById(any())).thenReturn(List.of(schema(620L, 10)));
when(userAchievementMetaRepository.findByUserId(anyLong())).thenReturn(List.of());
List<Long> steamCallsAtNanos = Collections.synchronizedList(new ArrayList<>());
@@ -55,12 +55,12 @@ class GameCorrectionServiceConcurrencyTest extends AbstractIntegrationTest {
void twoAuditorsCorrectTheSameFieldForTheFirstTime_BothSucceed() {
CountDownLatch firstCorrected = new CountDownLatch(1);
// The first commits a moment after the second has started, so both see no row before either commits.
// The first commits only when the second waits for its row, so both saw no row before either commits.
CompletableFuture<Void> first = CompletableFuture.runAsync(() -> new TransactionTemplate(transactionManager)
.executeWithoutResult(tx -> {
gameCorrectionService.correct(GAME, "name", "Portal Two", 1L);
firstCorrected.countDown();
pause();
awaitLockWaiter(jdbcTemplate);
}));
CompletableFuture<Void> second = CompletableFuture.runAsync(() -> {
await(firstCorrected);
@@ -86,14 +86,6 @@ class GameCorrectionServiceConcurrencyTest extends AbstractIntegrationTest {
assertThat(steamGameRepository.findById(GAME).orElseThrow().getImageUrl()).isEqualTo(IMAGE);
}
private static void pause() {
try {
Thread.sleep(300);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private static void await(CountDownLatch latch) {
try {
if (!latch.await(10, TimeUnit.SECONDS)) throw new IllegalStateException("latch timed out");
@@ -11,13 +11,11 @@ import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import java.time.Instant;
import java.util.List;
@@ -32,7 +30,7 @@ import static org.mockito.Mockito.when;
@TestPropertySource(properties = {
"spring.datasource.hikari.maximum-pool-size=1",
"spring.datasource.hikari.connection-timeout=1000"})
class GameDetailServicePoolTest extends AbstractIntegrationTest {
class GameDetailServicePoolTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001";
@@ -52,9 +50,6 @@ class GameDetailServicePoolTest extends AbstractIntegrationTest {
@Autowired
private UserGameAchievementMetaRepository userAchievementMetaRepository;
@MockitoBean
private SteamApiService steamApiService;
@Autowired
private GameAchievementSchemaRepository schemaRepository;
@@ -12,23 +12,25 @@ import core_service.modules.library.repo.GameAchievementSchemaMetaRepository;
import core_service.modules.library.repo.SteamGameRepository;
import core_service.modules.library.repo.GameAchievementSchemaRepository;
import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.library.repo.GameMoodsRepository;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.modules.library.repo.UserGameAchievementMetaRepository;
import core_service.modules.library.repo.UserGameAchievementRepository;
import core_service.modules.library.repo.UserGameRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamPlayerAchievementsResponse;
import core_service.modules.steam_adapter.api.dto.response.SteamSchemaForGameResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.web.client.ResourceAccessException;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -37,7 +39,7 @@ import static org.mockito.Mockito.when;
/**
* The detail page refreshes stale rows in a transaction of its own, then reads them in its own read transaction.
*/
class GameDetailServiceTest extends AbstractIntegrationTest {
class GameDetailServiceTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 1L;
private static final String STEAM_ID = "76561198000000001";
@@ -63,17 +65,11 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
@Autowired
private UserGameAchievementMetaRepository userAchievementMetaRepository;
@MockitoBean
private SteamApiService steamApiService;
@Autowired
private GameTimeToBeatRepository gameTimeToBeatRepository;
@MockitoBean
private TimeToBeatService timeToBeatService;
@MockitoBean
private MoodService moodService;
@MockitoBean
private SpeedrunService speedrunService;
@Autowired
private GameMoodsRepository gameMoodsRepository;
@Autowired
private GameSpeedrunRepository gameSpeedrunRepository;
@@ -86,6 +82,8 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
userAchievementRepository.deleteAll();
userAchievementMetaRepository.deleteAll();
gameSpeedrunRepository.deleteAll();
gameTimeToBeatRepository.deleteAll();
gameMoodsRepository.deleteAll();
steamGameRepository.save(new SteamGame(620L, "Portal 2", null, Instant.now()));
userGameRepository.save(new UserGame(USER_ID, 620L, 60, Instant.now(), Instant.now()));
}
@@ -115,7 +113,7 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, STEAM_ID, 620L).orElseThrow();
org.mockito.Mockito.verifyNoInteractions(steamApiService, speedrunService, timeToBeatService, moodService);
org.mockito.Mockito.verifyNoInteractions(steamApiService, igdbApiService, speedrunApiService);
assertEquals(true, detail.hidden());
assertEquals(1, detail.achievementsTotal());
}
@@ -124,20 +122,26 @@ class GameDetailServiceTest extends AbstractIntegrationTest {
void getGameDetail_GameAddedByHand_AsksForTheTimeToBeatAndMoodsAgain() {
userGameRepository.save(UserGame.manual(USER_ID, 620L, "PS5", GameStatus.PLAYING, 60, Instant.now()));
when(igdbApiService.isEnabled()).thenReturn(true);
when(igdbApiService.getTimeToBeat(Set.of(620L))).thenReturn(Optional.of(Map.of()));
when(igdbApiService.getGameTags(Set.of(620L))).thenReturn(Optional.of(Map.of()));
gameDetailService.getGameDetail(USER_ID, null, 620L);
org.mockito.Mockito.verify(timeToBeatService).refreshInBackground(List.of(620L));
org.mockito.Mockito.verify(moodService).refreshInBackground(List.of(620L));
assertTrue(gameTimeToBeatRepository.findById(620L).isPresent());
assertTrue(gameMoodsRepository.findById(620L).isPresent());
}
@Test
void getGameDetail_ShowsTheSpeedrunLinkAndAsksForIt() {
gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2", Instant.now());
gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2",
Instant.now().minus(Duration.ofDays(31)));
when(speedrunApiService.isEnabled()).thenReturn(true);
GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, null, 620L).orElseThrow();
assertEquals("https://www.speedrun.com/portal_2", detail.speedrunUrl());
org.mockito.Mockito.verify(speedrunService).refreshInBackground(List.of(620L));
org.mockito.Mockito.verify(speedrunApiService).findGames("Portal 2");
}
@Test
@@ -1,6 +1,6 @@
package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.domain.IgdbGame;
import core_service.modules.library.domain.GameImport;
import core_service.modules.library.repo.GameImportRepository;
import org.junit.jupiter.api.Test;
@@ -20,8 +20,8 @@ import static org.mockito.Mockito.when;
class GameImportWorkerTest {
private static IgdbApiService.Game game(long igdbId, String name, String... platforms) {
return new IgdbApiService.Game(igdbId, name, null, null, List.of(platforms), null);
private static IgdbGame game(long igdbId, String name, String... platforms) {
return new IgdbGame(igdbId, name, null, null, List.of(platforms), null);
}
private static GameImport.Line line(String title, String platform) {
@@ -37,12 +37,12 @@ class GameImportWorkerTest {
@Test
void rank_SameNameFirst_ThenTheLinesPlatform_ThenIgdbsOrder_AtMostFive() {
List<IgdbApiService.Game> found = List.of(game(1, "Doom Eternal", "PC"), game(2, "DOOM", "PC"),
List<IgdbGame> found = List.of(game(1, "Doom Eternal", "PC"), game(2, "DOOM", "PC"),
game(3, "Doom", "Switch"), game(4, "Doom 3"), game(5, "Doom 64"), game(6, "Doom II"));
assertThat(GameImportWorker.rank(line("Doom", "switch"), found)).extracting(IgdbApiService.Game::igdbId)
assertThat(GameImportWorker.rank(line("Doom", "switch"), found)).extracting(IgdbGame::igdbId)
.containsExactly(3L, 2L, 1L, 4L, 5L);
assertThat(GameImportWorker.rank(line("Doom", null), found)).extracting(IgdbApiService.Game::igdbId)
assertThat(GameImportWorker.rank(line("Doom", null), found)).extracting(IgdbGame::igdbId)
.startsWith(2L, 3L);
assertThat(GameImportWorker.rank(line("x", null), IntStream.range(0, 9)
.mapToObj(i -> game(i, "Game " + i)).toList())).hasSize(GameImportWorker.MAX_CANDIDATES);
@@ -56,7 +56,7 @@ class GameImportWorkerTest {
.thenReturn(Optional.of(7L), Optional.empty());
when(repository.findStalled(any())).thenReturn(Map.of());
GameImportWorker worker = new GameImportWorker(repository, mock(ManualGameService.class),
mock(IgdbApiService.class), mock(TimeToBeatService.class), mock(MoodService.class), 5);
mock(TimeToBeatService.class), mock(MoodService.class), 5);
try {
worker.wakeMatching();
verify(repository, timeout(2000)).findNext(GameImportWorker.MATCHING_QUEUE);
@@ -136,8 +136,7 @@ class ImageRepairServiceTest extends AbstractUnitTest {
@Test
void repairNightly_ChecksTheStoredRepairFirst() {
due(row(620L, ImageKind.COVER, Instant.now()));
SteamGame game = new SteamGame(620L, "Portal 2", null, Instant.now());
game.setImageUrl(HASHED_620);
SteamGame game = new SteamGame(620L, "Portal 2", null, HASHED_620, Instant.now());
when(steamGameRepository.findAllById(List.of(620L))).thenReturn(List.of(game));
when(steamCdnService.isAvailable(HASHED_620)).thenReturn(true);
@@ -1,16 +1,16 @@
package core_service.modules.library.service;
import core_service.modules.library.domain.UserSteamSync;
import core_service.modules.library.repo.GameSpeedrunRepository;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.modules.library.repo.UserSteamSyncRepository;
import core_service.modules.steam_adapter.api.dto.response.SteamOwnedGamesResponse;
import core_service.modules.steam_adapter.service.SteamApiService;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import java.util.List;
import java.util.Optional;
@@ -27,7 +27,7 @@ import static org.mockito.Mockito.when;
/**
* The claim and the finish of a sync are committed on their own, so other requests see RUNNING and FAILED.
*/
class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest {
class LibrarySyncWorkerConcurrencyTest extends AbstractMockedAdaptersTest {
private static final long USER_ID = 4711L;
private static final long APPID = 991220L;
@@ -42,8 +42,8 @@ class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest {
@Autowired
private JdbcTemplate jdbcTemplate;
@MockitoBean
private SteamApiService steamApiService;
@Autowired
private GameSpeedrunRepository gameSpeedrunRepository;
@BeforeEach
@AfterEach
@@ -53,6 +53,23 @@ class LibrarySyncWorkerConcurrencyTest extends AbstractIntegrationTest {
jdbcTemplate.update("DELETE FROM wishlist_game WHERE user_id = ?", USER_ID);
jdbcTemplate.update("DELETE FROM user_steam_sync WHERE user_id = ?", USER_ID);
jdbcTemplate.update("DELETE FROM steam_game WHERE appid = ?", APPID);
jdbcTemplate.update("DELETE FROM game_speedrun WHERE appid = ?", APPID);
}
@Test
void runSyncNow_TheFollowUpStoresTheSpeedrunRow() {
when(steamApiService.getWishlist(STEAM_ID)).thenReturn(Optional.empty());
when(steamApiService.getOwnedGames(STEAM_ID))
.thenReturn(List.of(new SteamOwnedGamesResponse.Game((int) APPID, "Half-Life 2", 0, "icon", true, 0)));
when(speedrunApiService.isEnabled()).thenReturn(true);
when(speedrunApiService.findGames("Half-Life 2")).thenReturn(Optional.of(List.of(
new SpeedrunApiService.Game("hl2", "Half-Life 2", 2004, "https://www.speedrun.com/hl2"))));
when(speedrunApiService.hasVerifiedRuns("hl2")).thenReturn(Optional.of(true));
worker.runSyncNow(USER_ID, STEAM_ID);
verify(speedrunApiService).hasVerifiedRuns("hl2");
assertThat(gameSpeedrunRepository.findById(APPID)).isPresent();
}
@Test
@@ -8,6 +8,9 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.test.util.ReflectionTestUtils;
import java.time.Instant;
@@ -38,6 +41,9 @@ class MoodServiceTest extends AbstractUnitTest {
@Mock
private UserGameRepository userGameRepository;
@Spy
private TaskExecutor applicationTaskExecutor = new SyncTaskExecutor();
@InjectMocks
private MoodService service;
@@ -72,6 +78,16 @@ class MoodServiceTest extends AbstractUnitTest {
verify(gameMoodsRepository).upsert(eq(427520L), any(), any(), eq(true), any(Instant.class));
}
@Test
void refreshInBackground_AllFresh_QueuesNothing() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameMoodsRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(620L));
service.refreshInBackground(List.of(620L));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test
void refreshBacklogInBackground_RefreshesTheBacklogOfTheUser() {
when(igdbApiService.isEnabled()).thenReturn(true);
@@ -11,6 +11,9 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.dao.DataAccessResourceFailureException;
import java.time.Instant;
@@ -44,6 +47,8 @@ class SpeedrunServiceTest extends AbstractUnitTest {
private SteamGameRepository steamGameRepository;
@Mock
private UserGameRepository userGameRepository;
@Spy
private TaskExecutor syncFollowUpExecutor = new SyncTaskExecutor();
@InjectMocks
private SpeedrunService service;
@@ -222,4 +227,14 @@ class SpeedrunServiceTest extends AbstractUnitTest {
verifyNoInteractions(gameSpeedrunRepository, steamGameRepository);
}
@Test
void onLibrarySynced_RefreshesTheBacklogOnTheListenerThread() {
when(userGameRepository.findBacklogAppids(7L)).thenReturn(List.of());
service.onLibrarySynced(new LibrarySyncedEvent(7L, "76561198000000000"));
verify(userGameRepository).findBacklogAppids(7L);
verify(syncFollowUpExecutor, never()).execute(any());
}
}
@@ -2,12 +2,12 @@ package core_service.modules.library.service;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.library.repo.GameTimeToBeatRepository;
import core_service.shared.test.AbstractIntegrationTest;
import core_service.shared.test.AbstractMockedAdaptersTest;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
@@ -26,7 +26,10 @@ import static org.mockito.Mockito.when;
/**
* Two users' syncs run in parallel and can miss the same app, e.g. every app right after the first deploy.
*/
class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
class TimeToBeatServiceConcurrencyTest extends AbstractMockedAdaptersTest {
@Autowired
private JdbcTemplate jdbc;
@Autowired
private TimeToBeatService timeToBeatService;
@@ -37,9 +40,6 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
@Autowired
private PlatformTransactionManager transactionManager;
@MockitoBean
private IgdbApiService igdbApiService;
// Before as well: other integration tests leave rows in this shared table, and a fresh row would hide the race.
@BeforeEach
@AfterEach
@@ -60,8 +60,8 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
timeToBeatService.refresh(List.of(620L));
firstWrote.countDown();
await(secondStarted);
// Keep the transaction open while the second sync writes the same row.
sleep();
// Keep the transaction open until the second sync waits for the same row.
awaitLockWaiter(jdbc);
}));
CompletableFuture<Void> second = CompletableFuture.runAsync(() -> {
await(firstWrote);
@@ -82,11 +82,4 @@ class TimeToBeatServiceConcurrencyTest extends AbstractIntegrationTest {
}
}
private static void sleep() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
@@ -7,6 +7,10 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Spy;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.dao.QueryTimeoutException;
import org.springframework.core.task.TaskExecutor;
import org.springframework.test.util.ReflectionTestUtils;
import java.time.Instant;
@@ -15,6 +19,7 @@ import java.util.Map;
import java.util.Optional;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyCollection;
import static org.mockito.ArgumentMatchers.eq;
@@ -31,6 +36,9 @@ class TimeToBeatServiceTest extends AbstractUnitTest {
@Mock
private GameTimeToBeatRepository gameTimeToBeatRepository;
@Spy
private TaskExecutor applicationTaskExecutor = new SyncTaskExecutor();
@InjectMocks
private TimeToBeatService service;
@@ -53,6 +61,37 @@ class TimeToBeatServiceTest extends AbstractUnitTest {
verify(gameTimeToBeatRepository, never()).upsert(eq(220L), any(), any(), any(), any());
}
@Test
void refreshInBackground_AllFresh_QueuesNothing() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(220L, 620L));
service.refreshInBackground(List.of(220L, 620L));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test
void refreshInBackground_StaleCheckFails_DoesNotFailTheCaller() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenThrow(new QueryTimeoutException("slow"));
assertDoesNotThrow(() -> service.refreshInBackground(List.of(620L)));
verify(applicationTaskExecutor, never()).execute(any());
}
@Test
void refreshInBackground_QueuesOnlyTheStaleApps() {
when(igdbApiService.isEnabled()).thenReturn(true);
when(gameTimeToBeatRepository.findFreshAppids(anyCollection(), any())).thenReturn(Set.of(220L));
when(igdbApiService.getTimeToBeat(Set.of(620L))).thenReturn(Optional.of(Map.of()));
service.refreshInBackground(List.of(220L, 620L));
verify(igdbApiService).getTimeToBeat(Set.of(620L));
verify(gameTimeToBeatRepository).upsert(eq(620L), isNull(), isNull(), isNull(), any(Instant.class));
}
@Test
void refresh_GamesOffSteam_AreAskedByIgdbIdAndStoredUnderTheirKey() {
when(igdbApiService.isEnabled()).thenReturn(true);
@@ -1,17 +1,24 @@
package core_service.shared.test;
import core_service.shared.background.BackgroundPools;
import org.junit.jupiter.api.Tag;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
import org.springframework.core.task.SyncTaskExecutor;
import org.springframework.core.task.TaskExecutor;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.bean.override.convention.TestBean;
import org.testcontainers.containers.PostgreSQLContainer;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.time.Duration;
import java.util.Base64;
import java.util.concurrent.atomic.AtomicReference;
/**
* Base class for integration tests.
@@ -37,6 +44,63 @@ public abstract class AbstractIntegrationTest {
postgres.start();
}
// Background work is done when the call returns, so a test reads the result at once. The default pool runs it on
// the calling thread. The follow-up pool runs it on a new thread and waits: an after-commit listener then has its
// own transactions, as in production, and does not join the finished one of the sync.
@TestBean(name = BackgroundPools.DEFAULT)
private TaskExecutor applicationTaskExecutor;
@TestBean(name = BackgroundPools.SYNC_FOLLOW_UP)
private TaskExecutor syncFollowUpExecutor;
static TaskExecutor applicationTaskExecutor() {
return new SyncTaskExecutor();
}
static TaskExecutor syncFollowUpExecutor() {
return task -> {
AtomicReference<Throwable> failure = new AtomicReference<>();
Thread thread = Thread.ofVirtual().start(() -> {
try {
task.run();
} catch (Throwable e) {
failure.set(e);
}
});
try {
if (!thread.join(Duration.ofSeconds(30))) {
throw new AssertionError("Follow-up task still runs after 30 s");
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IllegalStateException(e);
}
if (failure.get() != null) {
throw new AssertionError("Follow-up task failed", failure.get());
}
};
}
/**
* Waits until another connection waits for a row lock: the race a concurrency test sets up is then in place.
*/
protected static void awaitLockWaiter(JdbcTemplate jdbc) {
for (int attempt = 0; attempt < 250; attempt++) {
Integer waiting = jdbc.queryForObject(
"select count(*) from pg_stat_activity where wait_event_type = 'Lock'", Integer.class);
if (waiting != null && waiting > 0) {
return;
}
try {
Thread.sleep(20);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IllegalStateException(e);
}
}
throw new AssertionError("No connection waits for a lock");
}
// No key is committed: a fresh one per JVM. Tests that sign tokens use TEST_KEYS.getPrivate().
protected static final KeyPair TEST_KEYS = generateKeys();
@@ -0,0 +1,22 @@
package core_service.shared.test;
import core_service.modules.igdb_adapter.service.IgdbApiService;
import core_service.modules.speedrun_adapter.service.SpeedrunApiService;
import core_service.modules.steam_adapter.service.SteamApiService;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
/**
* Integration tests with the three external APIs mocked, in one shared context. A mock answers empty, so a sync a
* request starts finds no games and leaves the library as it is.
*/
public abstract class AbstractMockedAdaptersTest extends AbstractIntegrationTest {
@MockitoBean
protected SteamApiService steamApiService;
@MockitoBean
protected IgdbApiService igdbApiService;
@MockitoBean
protected SpeedrunApiService speedrunApiService;
}
@@ -0,0 +1,21 @@
package core_service.shared.test;
import core_service.modules.library.service.LibrarySyncService;
import core_service.modules.library.service.MoodService;
import core_service.modules.library.service.SpeedrunService;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
/**
* Integration tests of the picker and stats pages: the sync and the background refreshes they start are mocked.
*/
public abstract class AbstractMockedLibraryServicesTest extends AbstractIntegrationTest {
@MockitoBean
protected LibrarySyncService syncService;
@MockitoBean
protected MoodService moodService;
@MockitoBean
protected SpeedrunService speedrunService;
}