diff --git a/readme.md b/readme.md index 5e1cd20..f47fed9 100644 --- a/readme.md +++ b/readme.md @@ -17,8 +17,13 @@ To Be Played is split into three repositories: sync. Resilience4j retries and rate-limits every Steam call; Caffeine caches game counts and player summaries. - **Backlog**: status (`BACKLOG`, `PLAYING`, `FINISHED`, `COMPLETED` = 100 %, `DROPPED`), rating 1–5, notes, priority, filters and sorting. "Pile of Shame" summary with a random suggestion of what to play. -- **Picker**: up to 10 random backlog games for a time budget (IGDB normal time to beat), HIGH priority first, then - MEDIUM, then the rest. Games already shown can be excluded for a reroll. +- **Picker**: up to 10 random backlog games for a time budget (IGDB normal time to beat) and a mood, HIGH priority + first, then MEDIUM, then the rest. Games already shown can be excluded for a reroll. Moods `STORY`, `COOP`, + `CHILL`, `ACTION`, `SCARY` come from IGDB genres, themes and game modes; `SPEEDRUN` from speedrun.com. +- **Speedrun games**: a backlog game counts as a speedrun game when speedrun.com has a game with the same name (and + a release year at most one year off, when IGDB knows it) with at least one verified run. The game detail returns + the speedrun.com link as `speedrunUrl`. speedrun.com has no Steam or IGDB link and no key, so one queue works for + all users, with up to 3 requests per game and 1 request per second; a result is kept 30 days. - **Achievements**: after every library sync a background job fetches the achievement schema of each owned game and the user's unlocks of each game whose playtime changed. One Steam call per second, so user requests keep half of the rate limit; it stops after 3 Steam errors in a row or when Steam hides the achievements. A Steam error never @@ -52,7 +57,7 @@ accepted only by `GET /api/public/profiles/library` in the `X-Profile-Ticket` he | `PUT` | `/api/library/games/{appid}/backlog` | JWT | | `PUT` | `/api/library/games/{appid}/status` (status only, keeps rating, notes, priority) | JWT | | `POST` | `/api/library/sync` | JWT | -| `GET` | `/api/library/picker?maxMinutes=&count=&exclude=` | JWT | +| `GET` | `/api/library/picker?maxMinutes=&mood=&count=&exclude=` | JWT | | `GET` | `/api/library/achievements/summary` | JWT | | `GET` | `/api/library/sync-status`, `/summary`, `/activity`, `/wishlist`, `/steam-profile`, `/games-count` | JWT | | `GET` | `/api/public/profiles/library` | profile ticket | @@ -62,7 +67,7 @@ accepted only by `GET /api/public/profiles/library` in the `X-Profile-Ticket` he - Java 25, Spring Boot 4.1 (Web MVC, Security, Data JPA, Validation, Cache with Caffeine), virtual threads - PostgreSQL 17, Flyway migrations -- Resilience4j: retry and rate limiter for Steam, rate limiter for IGDB +- Resilience4j: retry and rate limiter for Steam, rate limiters for IGDB and speedrun.com - JJWT with Ed25519 public key - Spring Boot Actuator + Micrometer (Prometheus) - Tests: JUnit 5, Testcontainers (PostgreSQL), ArchUnit @@ -84,14 +89,17 @@ flowchart LR Stats["stats"] Steam["steam_adapter"] Igdb["igdb_adapter"] + Speedrun["speedrun_adapter"] LibService --> Steam LibService --> Igdb + LibService --> Speedrun end Browser -->|"tbp_access cookie"| Security Security -->|"JWKS by kid"| Auth["ToBePlayed-Auth"] Steam --> SteamApi["Steam Web API"] Igdb --> IgdbApi["IGDB API"] Igdb -->|"OAuth token"| Twitch["Twitch"] + Speedrun --> SpeedrunApi["speedrun.com API"] LibRepo --> Db[("PostgreSQL tbp_core")] Stats --> Db ``` @@ -101,12 +109,14 @@ flowchart LR | `library` | Sync, backlog, manual games, playtime history, wishlist, achievements, public profile data | | `stats` | Public numbers for the landing page | | `steam_adapter` | Steam Web API client | -| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, game search | +| `igdb_adapter` | IGDB client (Twitch OAuth): time to beat, game tags for moods, release years, game search | +| `speedrun_adapter` | speedrun.com client: game search by name, verified runs | Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` (JPA entities) and `repo` (repositories). `src/test/java/core_service/architecture/ArchitectureTest.java` checks these rules on every build: -- Modules do not depend on each other. Only `library` may use `steam_adapter` and `igdb_adapter`. +- Modules do not depend on each other. Only `library` may use `steam_adapter`, `igdb_adapter` and + `speedrun_adapter`. - No cycles between `modules`, `shared` and `configuration`. - `api` never uses `repo`, and `repo` never uses `api`. - Every class in `repo` ends with `Repository`. @@ -126,7 +136,8 @@ Inside a module: `api` (controllers, DTOs), `service` (business logic), `domain` owns. When Steam returns no games, the library stays as it is. 5. `TimeToBeatService` and `MoodService` ask IGDB for missing or old time-to-beat and mood data, for Steam games and manual games. -6. After the commit, `LibrarySyncedEvent` starts `AchievementSyncService` in the background. +6. After the commit, `LibrarySyncedEvent` starts `AchievementSyncService` and `SpeedrunService` (backlog games only) + in the background. The frontend polls `GET /api/library/sync-status` until the run ends. @@ -180,8 +191,6 @@ Runs unit, WebMvc, persistence and integration tests plus the ArchUnit rules. Ne ## Roadmap -- Add a game by name from IGDB for platforms without Steam (needs an internal game id next to the Steam `appid`). -- Mood filter for the picker (needs genres or themes from IGDB). - Sale alerts for wishlist games. - Move `steam_adapter` DTOs out of `api`, then turn on strict module independence in ArchUnit. diff --git a/src/main/java/core_service/modules/igdb_adapter/service/IgdbApiService.java b/src/main/java/core_service/modules/igdb_adapter/service/IgdbApiService.java index f2d36c8..c178818 100644 --- a/src/main/java/core_service/modules/igdb_adapter/service/IgdbApiService.java +++ b/src/main/java/core_service/modules/igdb_adapter/service/IgdbApiService.java @@ -24,6 +24,8 @@ import java.util.LinkedHashSet; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; +import java.util.TreeSet; import java.util.stream.Collectors; import java.util.stream.Stream; @@ -173,6 +175,51 @@ public class IgdbApiService { } } + /** + * @return the first release years of the Steam apps IGDB links to a game; an app linked to several games (say a + * remaster) gets all their years, an app without a known year is missing. Empty when the adapter is off or a + * request failed. + */ + public Optional>> getReleaseYears(Collection steamAppids) { + if (!isEnabled() || steamAppids.isEmpty()) { + return Optional.empty(); + } + try { + List appids = List.copyOf(steamAppids); + Map> result = new HashMap<>(); + for (int from = 0; from < appids.size(); from += BATCH) { + Map> steamByIgdb = steamAppidsByIgdbGame(appids.subList(from, Math.min(from + BATCH, appids.size()))); + yearsOf(steamByIgdb.keySet()).forEach((igdbId, year) -> steamByIgdb.get(igdbId) + .forEach(appid -> result.computeIfAbsent(appid, key -> new TreeSet<>()).add(year))); + } + return Optional.of(result); + } catch (Exception e) { + log.error("Error communicating with IGDB for release years", e); + return Optional.empty(); + } + } + + /** + * Like {@link #getReleaseYears}, but for IGDB game ids. + */ + public Optional>> getReleaseYearsByGameIds(Collection igdbIds) { + if (!isEnabled() || igdbIds.isEmpty()) { + return Optional.empty(); + } + try { + List ids = List.copyOf(igdbIds); + Map> result = new HashMap<>(); + for (int from = 0; from < ids.size(); from += BATCH) { + yearsOf(ids.subList(from, Math.min(from + BATCH, ids.size()))) + .forEach((igdbId, year) -> result.put(igdbId, Set.of(year))); + } + return Optional.of(result); + } catch (Exception e) { + log.error("Error communicating with IGDB for release years", e); + return Optional.empty(); + } + } + /** * Games whose name matches, without editions and other versions. Empty when the adapter is off or IGDB failed. */ @@ -260,8 +307,7 @@ public class IgdbApiService { if (game.id() == null || game.name() == null || game.name().isBlank()) { return Optional.empty(); } - Integer year = game.firstReleaseDate() == null ? null - : Instant.ofEpochSecond(game.firstReleaseDate()).atZone(ZoneOffset.UTC).getYear(); + Integer year = game.firstReleaseDate() == null ? null : year(game.firstReleaseDate()); List platforms = game.platforms() == null ? List.of() : game.platforms().stream() .map(p -> p.abbreviation() != null && !p.abbreviation().isBlank() ? p.abbreviation() : p.name()) .filter(name -> name != null && !name.isBlank()) @@ -315,6 +361,27 @@ public class IgdbApiService { return result; } + private Map yearsOf(Collection igdbIds) { + if (igdbIds.isEmpty()) { + return Map.of(); + } + String gameIds = igdbIds.stream().map(String::valueOf).collect(Collectors.joining(",")); + IgdbResponses.Game[] games = query("/games", + "fields first_release_date; where id = (" + gameIds + "); limit " + LIMIT + ";", + IgdbResponses.Game[].class); + Map result = new HashMap<>(); + for (IgdbResponses.Game game : games) { + if (game.id() != null && game.firstReleaseDate() != null) { + result.put(game.id(), year(game.firstReleaseDate())); + } + } + return result; + } + + private static int year(long unixSeconds) { + return Instant.ofEpochSecond(unixSeconds).atZone(ZoneOffset.UTC).getYear(); + } + private static List names(List named) { return named == null ? List.of() : named.stream() .map(IgdbResponses.Named::name) diff --git a/src/main/java/core_service/modules/library/api/LibraryController.java b/src/main/java/core_service/modules/library/api/LibraryController.java index e9599c2..8088fa3 100644 --- a/src/main/java/core_service/modules/library/api/LibraryController.java +++ b/src/main/java/core_service/modules/library/api/LibraryController.java @@ -18,6 +18,7 @@ import core_service.modules.library.service.GameDetailService; import core_service.modules.library.service.LibraryQueryService; import core_service.modules.library.service.LibrarySyncService; import core_service.modules.library.service.MoodService; +import core_service.modules.library.service.SpeedrunService; import core_service.modules.library.service.SteamProfileService; import core_service.modules.library.service.SyncCooldownException; import core_service.shared.security.UserPrincipal; @@ -62,6 +63,7 @@ public class LibraryController { private final BacklogService backlogService; private final SteamProfileService steamProfileService; private final MoodService moodService; + private final SpeedrunService speedrunService; @GetMapping("/games-count") public ResponseEntity getGamesCount(@AuthenticationPrincipal UserPrincipal principal) { @@ -192,7 +194,9 @@ public class LibraryController { if (principal.steamId() != null) { syncService.syncIfStale(userId, String.valueOf(principal.steamId())); } - if (mood != null) { + if (mood == Mood.SPEEDRUN) { + speedrunService.refreshBacklogInBackground(userId); + } else if (mood != null) { moodService.refreshBacklogInBackground(userId); } return ResponseEntity.ok(queryService.pickBacklogGames(userId, budget, mood, excluded, safeCount)); diff --git a/src/main/java/core_service/modules/library/api/dto/GameDetailDto.java b/src/main/java/core_service/modules/library/api/dto/GameDetailDto.java index 09286f8..5f37579 100644 --- a/src/main/java/core_service/modules/library/api/dto/GameDetailDto.java +++ b/src/main/java/core_service/modules/library/api/dto/GameDetailDto.java @@ -29,7 +29,9 @@ public record GameDetailDto( TimeToBeatDto timeToBeat, GameSource source, String platform, - String imageUrl + String imageUrl, + // null until a check finds a speedrun.com game with verified runs + String speedrunUrl ) { /** diff --git a/src/main/java/core_service/modules/library/domain/GameSpeedrun.java b/src/main/java/core_service/modules/library/domain/GameSpeedrun.java new file mode 100644 index 0000000..d3c15ba --- /dev/null +++ b/src/main/java/core_service/modules/library/domain/GameSpeedrun.java @@ -0,0 +1,37 @@ +package core_service.modules.library.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AccessLevel; +import lombok.Getter; +import lombok.NoArgsConstructor; + +import java.time.Instant; + +/** + * The speedrun.com game of one library game, see {@link Mood#SPEEDRUN}. + */ +@Entity +@Table(name = "game_speedrun") +@Getter +@NoArgsConstructor(access = AccessLevel.PROTECTED) +public class GameSpeedrun { + + @Id + @Column(name = "appid") + private Long appid; + + /** + * Null when speedrun.com has no game with verified runs for it. + */ + @Column(name = "speedrun_game_id", length = 16) + private String speedrunGameId; + + @Column(name = "weblink", length = 512) + private String weblink; + + @Column(name = "checked_at", nullable = false) + private Instant checkedAt; +} diff --git a/src/main/java/core_service/modules/library/domain/Mood.java b/src/main/java/core_service/modules/library/domain/Mood.java index 7c13727..3922761 100644 --- a/src/main/java/core_service/modules/library/domain/Mood.java +++ b/src/main/java/core_service/modules/library/domain/Mood.java @@ -11,7 +11,11 @@ import java.util.stream.Collectors; * have several moods or none. */ public enum Mood { - STORY, COOP, CHILL, ACTION, SCARY; + STORY, COOP, CHILL, ACTION, SCARY, + /** + * Has verified runs on speedrun.com. Stored in game_speedrun, not game_mood, so {@link #of} never returns it. + */ + SPEEDRUN; private static final Set STORY_GENRES = Set.of("role-playing (rpg)", "visual novel"); private static final Set STORY_THEMES = Set.of("drama"); diff --git a/src/main/java/core_service/modules/library/domain/SteamGame.java b/src/main/java/core_service/modules/library/domain/SteamGame.java index f538fb8..9b225f1 100644 --- a/src/main/java/core_service/modules/library/domain/SteamGame.java +++ b/src/main/java/core_service/modules/library/domain/SteamGame.java @@ -34,6 +34,13 @@ public class SteamGame { @Column(name = "image_url", length = 512) private String imageUrl; + /** + * The name of an app Steam sends without one, e.g. an unvetted app. + */ + public static String placeholderName(long appid) { + return "App " + appid; + } + public SteamGame(Long appid, String name, String imgIconUrl, Instant lastSeenAt) { this.appid = appid; this.name = name; diff --git a/src/main/java/core_service/modules/library/repo/GameSpeedrunRepository.java b/src/main/java/core_service/modules/library/repo/GameSpeedrunRepository.java new file mode 100644 index 0000000..0c92a8a --- /dev/null +++ b/src/main/java/core_service/modules/library/repo/GameSpeedrunRepository.java @@ -0,0 +1,28 @@ +package core_service.modules.library.repo; + +import core_service.modules.library.domain.GameSpeedrun; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Modifying; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; +import org.springframework.transaction.annotation.Transactional; + +import java.time.Instant; +import java.util.Collection; +import java.util.Set; + +public interface GameSpeedrunRepository extends JpaRepository { + + @Query("select s.appid from GameSpeedrun s where s.appid in :appids and s.checkedAt > :since") + Set findFreshAppids(@Param("appids") Collection appids, @Param("since") Instant since); + + @Modifying + @Transactional + @Query(value = "insert into game_speedrun (appid, speedrun_game_id, weblink, checked_at) " + + "values (:appid, :speedrunGameId, :weblink, :checkedAt) " + + "on conflict (appid) do update set speedrun_game_id = excluded.speedrun_game_id, " + + "weblink = excluded.weblink, checked_at = excluded.checked_at", + nativeQuery = true) + void upsert(@Param("appid") Long appid, @Param("speedrunGameId") String speedrunGameId, + @Param("weblink") String weblink, @Param("checkedAt") Instant checkedAt); +} diff --git a/src/main/java/core_service/modules/library/repo/UserGameRepository.java b/src/main/java/core_service/modules/library/repo/UserGameRepository.java index b8fa363..8803fd7 100644 --- a/src/main/java/core_service/modules/library/repo/UserGameRepository.java +++ b/src/main/java/core_service/modules/library/repo/UserGameRepository.java @@ -93,7 +93,9 @@ public interface UserGameRepository extends JpaRepository List excluded = new ArrayList<>(excludeAppids); // "not in ()" is invalid SQL excluded.add(NO_GAME); - return queryPickerGames(userId, maxMinutes, mood == null ? null : mood.name(), excluded, count); + boolean speedrun = mood == Mood.SPEEDRUN; + String storedMood = mood == null || speedrun ? null : mood.name(); + return queryPickerGames(userId, maxMinutes, storedMood, speedrun, excluded, count); } @Query(value = "select ug.* from user_game ug where ug.user_id = :userId and ug.removed_at is null " + @@ -102,11 +104,14 @@ public interface UserGameRepository extends JpaRepository " where t.appid = ug.appid and t.normally_minutes <= :maxMinutes)) " + "and (cast(:mood as varchar) is null or exists (select 1 from game_mood m " + " where m.appid = ug.appid and cast(:mood as varchar) = any(m.moods))) " + + "and (:speedrun = false or exists (select 1 from game_speedrun s " + + " where s.appid = ug.appid and s.weblink is not null)) " + "order by case ug.priority when 'HIGH' then 0 when 'MEDIUM' then 1 else 2 end, random() " + "limit :count", nativeQuery = true) List queryPickerGames(@Param("userId") Long userId, @Param("maxMinutes") Integer maxMinutes, @Param("mood") String mood, + @Param("speedrun") boolean speedrun, @Param("excludeAppids") Collection excludeAppids, @Param("count") int count); diff --git a/src/main/java/core_service/modules/library/service/GameDetailService.java b/src/main/java/core_service/modules/library/service/GameDetailService.java index d8ae2ed..8bcf8e9 100644 --- a/src/main/java/core_service/modules/library/service/GameDetailService.java +++ b/src/main/java/core_service/modules/library/service/GameDetailService.java @@ -6,6 +6,7 @@ import core_service.modules.library.domain.AchievementProgress; import core_service.modules.library.domain.GameAchievementSchema; import core_service.modules.library.domain.GameAchievementSchemaMeta; import core_service.modules.library.domain.GameSource; +import core_service.modules.library.domain.GameSpeedrun; import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGameAchievement; @@ -14,6 +15,7 @@ import core_service.modules.library.domain.UserGameId; import core_service.modules.library.repo.GameAchievementSchemaMetaRepository; import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.GameAchievementSchemaRepository; +import core_service.modules.library.repo.GameSpeedrunRepository; import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.UserGameAchievementMetaRepository; import core_service.modules.library.repo.UserGameAchievementRepository; @@ -44,6 +46,8 @@ public class GameDetailService { private final GameTimeToBeatRepository gameTimeToBeatRepository; private final TimeToBeatService timeToBeatService; private final MoodService moodService; + private final GameSpeedrunRepository gameSpeedrunRepository; + private final SpeedrunService speedrunService; @Value("${app.library.achievement-ttl-hours:1}") private long achievementTtlHours; @@ -61,6 +65,8 @@ public class GameDetailService { } UserGame userGame = userGameOpt.get(); SteamGame meta = steamGameRepository.findById(appid).orElse(null); + // Only backlog games are checked in bulk; any other game gets its link on a later visit. + speedrunService.refreshInBackground(List.of(appid)); if (userGame.getSource() == GameSource.MANUAL) { // Users without a Steam ID never sync, so the time to beat of their games is asked again here. timeToBeatService.refreshInBackground(List.of(appid)); @@ -148,7 +154,8 @@ public class GameDetailService { .orElse(null), userGame.getSource(), userGame.getPlatform(), - meta != null ? meta.getImageUrl() : null + meta != null ? meta.getImageUrl() : null, + gameSpeedrunRepository.findById(userGame.getAppid()).map(GameSpeedrun::getWeblink).orElse(null) ); } } diff --git a/src/main/java/core_service/modules/library/service/LibrarySyncWorker.java b/src/main/java/core_service/modules/library/service/LibrarySyncWorker.java index b0a1d02..308745d 100644 --- a/src/main/java/core_service/modules/library/service/LibrarySyncWorker.java +++ b/src/main/java/core_service/modules/library/service/LibrarySyncWorker.java @@ -173,7 +173,7 @@ public class LibrarySyncWorker { // An unvetted app can come without a name, and the name column is NOT NULL. boolean hasName = g.name() != null && !g.name().isBlank(); if (existing == null) { - steamGameRepository.save(new SteamGame(appid, hasName ? g.name() : "App " + appid, g.imgIconUrl(), now)); + steamGameRepository.save(new SteamGame(appid, hasName ? g.name() : SteamGame.placeholderName(appid), g.imgIconUrl(), now)); } else { if (hasName) existing.setName(g.name()); existing.setImgIconUrl(g.imgIconUrl()); diff --git a/src/main/java/core_service/modules/library/service/SpeedrunMatch.java b/src/main/java/core_service/modules/library/service/SpeedrunMatch.java new file mode 100644 index 0000000..41073c5 --- /dev/null +++ b/src/main/java/core_service/modules/library/service/SpeedrunMatch.java @@ -0,0 +1,82 @@ +package core_service.modules.library.service; + +import core_service.modules.speedrun_adapter.service.SpeedrunApiService; + +import java.text.Normalizer; +import java.util.Collection; +import java.util.HashSet; +import java.util.List; +import java.util.Locale; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +/** + * Finds a library game among speedrun.com's search hits. speedrun.com has no Steam or IGDB link, so the name must be + * the same and, when both years are known, the release year may differ by at most one. + */ +final class SpeedrunMatch { + + static final int MAX_CANDIDATES = 2; + static final String WEBLINK_PREFIX = "https://www.speedrun.com/"; + + // speedrun.com tells remakes apart by the year: "Prey (2017)" next to "Prey". + private static final String TRADEMARK_SIGNS = "[™®©]"; + private static final Pattern YEAR_SUFFIX = Pattern.compile("^(.*?)\\s*\\((\\d{4})\\)$"); + // The column sizes of game_speedrun. + private static final int MAX_GAME_ID_LENGTH = 16; + private static final int MAX_WEBLINK_LENGTH = 512; + + private SpeedrunMatch() { + } + + /** + * @param years our release years of the game; empty when unknown + * @param hits in speedrun.com's order + * @return the hits worth asking for runs, best first, at most {@link #MAX_CANDIDATES} + */ + static List candidates(String name, Collection years, List hits) { + NameAndYear ours = split(name); + Set ourYears = new HashSet<>(years); + if (ours.year() != null) ourYears.add(ours.year()); + return hits.stream() + .filter(hit -> hit.id() != null && hit.id().length() <= MAX_GAME_ID_LENGTH) + // The weblink goes to the browser as a link; only speedrun.com's own pages may get there. + .filter(hit -> hit.weblink() != null && hit.weblink().startsWith(WEBLINK_PREFIX) + && hit.weblink().length() <= MAX_WEBLINK_LENGTH) + .filter(hit -> { + NameAndYear theirs = split(hit.name()); + Integer year = theirs.year() != null ? theirs.year() : hit.year(); + return theirs.name().equals(ours.name()) && (year == null || ourYears.isEmpty() + || ourYears.stream().anyMatch(ourYear -> Math.abs(ourYear - year) <= 1)); + }) + .limit(MAX_CANDIDATES) + .toList(); + } + + /** + * The name for speedrun.com's search, which does not find "DOOM Eternal" for "DOOM Eternal™". + */ + static String searchText(String name) { + return name.replaceAll(TRADEMARK_SIGNS, "").trim(); + } + + /** + * Lower case, without accents, ™ ® © and punctuation, words split by one space. + */ + static String normalize(String name) { + // Before NFKD, which turns ™ into "TM". + String withoutMarks = name.replaceAll(TRADEMARK_SIGNS, ""); + String plain = Normalizer.normalize(withoutMarks, Normalizer.Form.NFKD).replaceAll("\\p{M}", ""); + return plain.toLowerCase(Locale.ROOT).replaceAll("[^\\p{L}\\p{N}]+", " ").trim(); + } + + private static NameAndYear split(String name) { + Matcher matcher = YEAR_SUFFIX.matcher(name.trim()); + return matcher.matches() + ? new NameAndYear(normalize(matcher.group(1)), Integer.parseInt(matcher.group(2))) + : new NameAndYear(normalize(name), null); + } + + private record NameAndYear(String name, Integer year) {} +} diff --git a/src/main/java/core_service/modules/library/service/SpeedrunService.java b/src/main/java/core_service/modules/library/service/SpeedrunService.java new file mode 100644 index 0000000..03e0369 --- /dev/null +++ b/src/main/java/core_service/modules/library/service/SpeedrunService.java @@ -0,0 +1,189 @@ +package core_service.modules.library.service; + +import core_service.modules.igdb_adapter.service.IgdbApiService; +import core_service.modules.library.domain.GameKeys; +import core_service.modules.library.domain.Mood; +import core_service.modules.library.domain.SteamGame; +import core_service.modules.library.repo.GameSpeedrunRepository; +import core_service.modules.library.repo.SteamGameRepository; +import core_service.modules.library.repo.UserGameRepository; +import core_service.modules.speedrun_adapter.service.SpeedrunApiService; +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.scheduling.annotation.Async; +import org.springframework.stereotype.Service; +import org.springframework.transaction.event.TransactionalEventListener; + +import java.time.Duration; +import java.time.Instant; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.concurrent.ConcurrentSkipListSet; +import java.util.concurrent.atomic.AtomicBoolean; + +/** + * Stores which games have a speedrun.com page with verified runs, for {@link Mood#SPEEDRUN} and the link on the + * detail page. speedrun.com is searched by name, one game at a time, so all callers share one queue and one thread + * works it off: a game in several backlogs is asked once, and the rate limit has one waiter. + */ +@Service +@RequiredArgsConstructor +@Slf4j +public class SpeedrunService { + + static final int BATCH = 50; + // speedrun.com is down or blocks us; the rest of the queue is asked on the next trigger. + static final int MAX_ERRORS_IN_A_ROW = 3; + + private final SpeedrunApiService speedrunApiService; + private final IgdbApiService igdbApiService; + private final GameSpeedrunRepository gameSpeedrunRepository; + private final SteamGameRepository steamGameRepository; + private final UserGameRepository userGameRepository; + + private final ConcurrentSkipListSet pending = new ConcurrentSkipListSet<>(); + private final AtomicBoolean running = new AtomicBoolean(); + + @Value("${app.library.speedrun-ttl-days:30}") + private long ttlDays; + + /** + * Queues the games without a fresh row and, unless another thread already does it, works the queue off on this + * thread. A game takes up to 3 requests, and there is 1 request per second. + */ + public void refresh(Collection appids) { + if (!speedrunApiService.isEnabled() || appids.isEmpty()) return; + pending.addAll(appids); + while (running.compareAndSet(false, true)) { + try { + drain(); + } finally { + running.set(false); + } + // Games can be queued after the last poll and before the flag was cleared. + if (pending.isEmpty()) return; + } + } + + @Async + @TransactionalEventListener + public void onLibrarySynced(LibrarySyncedEvent event) { + refreshBacklogInBackground(event.userId()); + } + + @Async + public void refreshBacklogInBackground(Long userId) { + try { + refresh(userGameRepository.findBacklogAppids(userId)); + } catch (Exception e) { + log.warn("Speedrun games of the backlog of user {} not refreshed: {}", userId, e.getMessage()); + } + } + + @Async + public void refreshInBackground(Collection appids) { + try { + refresh(appids); + } catch (Exception e) { + log.warn("Speedrun games of {} not refreshed: {}", appids, e.getMessage()); + } + } + + private void drain() { + int errorsInARow = 0; + while (!pending.isEmpty()) { + List batch = new ArrayList<>(); + Long next; + while (batch.size() < BATCH && (next = pending.pollFirst()) != null) { + batch.add(next); + } + batch.removeAll(gameSpeedrunRepository.findFreshAppids(batch, Instant.now().minus(Duration.ofDays(ttlDays)))); + if (batch.isEmpty()) continue; + + Map names = new HashMap<>(); + for (SteamGame game : steamGameRepository.findAllById(batch)) { + names.put(game.getAppid(), game.getName()); + } + Optional>> years = releaseYears(batch); + if (years.isEmpty()) { + // Matching by name only would store a wrong remake for 30 days. + log.warn("Speedrun games of {} not refreshed, IGDB did not answer", batch.size()); + continue; + } + for (Long appid : batch) { + String name = names.get(appid); + if (name == null || name.equals(SteamGame.placeholderName(appid))) continue; + boolean stored; + try { + stored = checkAndStore(appid, name, years.get().getOrDefault(appid, Set.of())); + } catch (Exception e) { + log.warn("Speedrun game of appid={} not stored: {}", appid, e.getMessage()); + stored = false; + } + if (stored) { + errorsInARow = 0; + } else if (++errorsInARow >= MAX_ERRORS_IN_A_ROW) { + log.warn("Speedrun check failed {} times in a row, {} queued games dropped", errorsInARow, pending.size()); + pending.clear(); + return; + } + } + } + } + + /** + * @return false when a speedrun.com call failed; then nothing is stored and the game is asked again later + */ + private boolean checkAndStore(Long appid, String name, Set years) { + Optional> hits = speedrunApiService.findGames(SpeedrunMatch.searchText(name)); + if (hits.isEmpty()) return false; + SpeedrunApiService.Game match = null; + for (SpeedrunApiService.Game candidate : SpeedrunMatch.candidates(name, years, hits.get())) { + Optional hasRuns = speedrunApiService.hasVerifiedRuns(candidate.id()); + if (hasRuns.isEmpty()) return false; + if (hasRuns.get()) { + match = candidate; + break; + } + } + gameSpeedrunRepository.upsert(appid, match == null ? null : match.id(), match == null ? null : match.weblink(), Instant.now()); + return true; + } + + /** + * @return empty when IGDB failed. A game IGDB does not know, and every game when IGDB is off, has no years and is + * matched by name only. + */ + private Optional>> releaseYears(List appids) { + List steamAppids = new ArrayList<>(); + List igdbIds = new ArrayList<>(); + for (Long appid : appids) { + if (GameKeys.isSteam(appid)) { + steamAppids.add(appid); + } else { + igdbIds.add(GameKeys.igdbId(appid)); + } + } + Map> years = new HashMap<>(); + if (!igdbApiService.isEnabled()) { + return Optional.of(years); + } + if (!steamAppids.isEmpty()) { + Optional>> found = igdbApiService.getReleaseYears(steamAppids); + if (found.isEmpty()) return Optional.empty(); + years.putAll(found.get()); + } + if (!igdbIds.isEmpty()) { + Optional>> found = igdbApiService.getReleaseYearsByGameIds(igdbIds); + if (found.isEmpty()) return Optional.empty(); + found.get().forEach((igdbId, gameYears) -> years.put(GameKeys.ofIgdb(igdbId), gameYears)); + } + return Optional.of(years); + } +} diff --git a/src/main/java/core_service/modules/speedrun_adapter/api/dto/response/SpeedrunResponses.java b/src/main/java/core_service/modules/speedrun_adapter/api/dto/response/SpeedrunResponses.java new file mode 100644 index 0000000..d94f353 --- /dev/null +++ b/src/main/java/core_service/modules/speedrun_adapter/api/dto/response/SpeedrunResponses.java @@ -0,0 +1,43 @@ +package core_service.modules.speedrun_adapter.api.dto.response; + +import com.fasterxml.jackson.annotation.JsonIgnoreProperties; +import com.fasterxml.jackson.annotation.JsonProperty; + +import java.util.List; + +/** + * The parts of the speedrun.com API v1 answers the adapter reads. + */ +public final class SpeedrunResponses { + + private SpeedrunResponses() { + } + + @JsonIgnoreProperties(ignoreUnknown = true) + public record Games(@JsonProperty("data") List data) {} + + /** + * @param released the year of the first release + * @param links a link with rel "base-game" marks a mod or category extension of another game + */ + @JsonIgnoreProperties(ignoreUnknown = true) + public record Game( + @JsonProperty("id") String id, + @JsonProperty("names") Names names, + @JsonProperty("released") Integer released, + @JsonProperty("weblink") String weblink, + @JsonProperty("links") List links + ) {} + + @JsonIgnoreProperties(ignoreUnknown = true) + public record Names(@JsonProperty("international") String international) {} + + @JsonIgnoreProperties(ignoreUnknown = true) + public record Link(@JsonProperty("rel") String rel) {} + + @JsonIgnoreProperties(ignoreUnknown = true) + public record Runs(@JsonProperty("data") List data) {} + + @JsonIgnoreProperties(ignoreUnknown = true) + public record Run(@JsonProperty("id") String id) {} +} diff --git a/src/main/java/core_service/modules/speedrun_adapter/config/SpeedrunClientConfig.java b/src/main/java/core_service/modules/speedrun_adapter/config/SpeedrunClientConfig.java new file mode 100644 index 0000000..ed18c22 --- /dev/null +++ b/src/main/java/core_service/modules/speedrun_adapter/config/SpeedrunClientConfig.java @@ -0,0 +1,27 @@ +package core_service.modules.speedrun_adapter.config; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.http.HttpHeaders; +import org.springframework.http.client.SimpleClientHttpRequestFactory; +import org.springframework.web.client.RestClient; + +import java.time.Duration; + +@Configuration +public class SpeedrunClientConfig { + + @Bean + public RestClient speedrunRestClient(@Value("${app.speedrun.base-url}") String baseUrl) { + SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory(); + requestFactory.setConnectTimeout(Duration.ofSeconds(3)); + requestFactory.setReadTimeout(Duration.ofSeconds(10)); + return RestClient.builder() + .baseUrl(baseUrl) + .requestFactory(requestFactory) + // speedrun.com asks API users to name themselves. + .defaultHeader(HttpHeaders.USER_AGENT, "ToBePlayed (https://tbp.andreas-dahm.eu)") + .build(); + } +} diff --git a/src/main/java/core_service/modules/speedrun_adapter/service/SpeedrunApiService.java b/src/main/java/core_service/modules/speedrun_adapter/service/SpeedrunApiService.java new file mode 100644 index 0000000..86eaebb --- /dev/null +++ b/src/main/java/core_service/modules/speedrun_adapter/service/SpeedrunApiService.java @@ -0,0 +1,97 @@ +package core_service.modules.speedrun_adapter.service; + +import core_service.modules.speedrun_adapter.api.dto.response.SpeedrunResponses; +import io.github.resilience4j.ratelimiter.RateLimiter; +import io.github.resilience4j.ratelimiter.RateLimiterRegistry; +import lombok.Getter; +import lombok.extern.slf4j.Slf4j; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Service; +import org.springframework.web.client.RestClient; + +import java.util.List; +import java.util.Optional; + +/** + * Games and runs from speedrun.com (API v1, no key). speedrun.com knows no Steam appid or IGDB id, so games are + * found by name. + */ +@Service +@Slf4j +public class SpeedrunApiService { + + /** + * @param year of the first release, null when unknown + */ + public record Game(String id, String name, Integer year, String weblink) {} + + private static final int SEARCH_LIMIT = 20; + private static final String BASE_GAME = "base-game"; + + private final RestClient speedrunRestClient; + @Getter + private final boolean enabled; + private final RateLimiter rateLimiter; + + public SpeedrunApiService(@Qualifier("speedrunRestClient") RestClient speedrunRestClient, + @Value("${app.speedrun.enabled:true}") boolean enabled, + RateLimiterRegistry rateLimiterRegistry) { + this.speedrunRestClient = speedrunRestClient; + this.enabled = enabled; + this.rateLimiter = rateLimiterRegistry.rateLimiter("speedrunApi"); + } + + /** + * Games whose name looks like the given one, in speedrun.com's order, without mods and category extensions. + * Empty when the adapter is off or the request failed. + */ + public Optional> findGames(String name) { + if (!enabled || name == null || name.isBlank()) { + return Optional.empty(); + } + try { + RateLimiter.waitForPermission(rateLimiter); + // A URI variable, not queryParam(name): only variables get "&" and "+" encoded. + SpeedrunResponses.Games games = speedrunRestClient.get() + .uri("/games?name={name}&max={max}", name.trim(), SEARCH_LIMIT) + .retrieve() + .body(SpeedrunResponses.Games.class); + if (games == null || games.data() == null) { + throw new IllegalStateException("speedrun.com answered without games"); + } + return Optional.of(games.data().stream() + .filter(game -> game.id() != null && game.names() != null && game.names().international() != null + && !game.names().international().isBlank()) + .filter(game -> game.links() == null || game.links().stream().noneMatch(link -> BASE_GAME.equals(link.rel()))) + .map(game -> new Game(game.id(), game.names().international().trim(), game.released(), game.weblink())) + .toList()); + } catch (Exception e) { + log.warn("speedrun.com search for \"{}\" failed: {}", name, e.getMessage()); + return Optional.empty(); + } + } + + /** + * Empty when the adapter is off or the request failed. + */ + public Optional hasVerifiedRuns(String gameId) { + if (!enabled) { + return Optional.empty(); + } + try { + RateLimiter.waitForPermission(rateLimiter); + SpeedrunResponses.Runs runs = speedrunRestClient.get() + .uri("/runs?game={game}&status=verified&max=1", gameId) + .retrieve() + .body(SpeedrunResponses.Runs.class); + if (runs == null || runs.data() == null) { + throw new IllegalStateException("speedrun.com answered without runs"); + } + return Optional.of(!runs.data().isEmpty()); + } catch (Exception e) { + log.warn("speedrun.com runs of game {} failed: {}", gameId, e.getMessage()); + return Optional.empty(); + } + } +} \ No newline at end of file diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml index 741648d..aed3d2f 100644 --- a/src/main/resources/application.yml +++ b/src/main/resources/application.yml @@ -50,12 +50,17 @@ app: client-secret: ${IGDB_CLIENT_SECRET:} base-url: https://api.igdb.com/v4 token-base-url: https://id.twitch.tv + speedrun: + # No key needed. false stops every call to speedrun.com. + enabled: true + base-url: https://www.speedrun.com/api/v1 library: sync-ttl-hours: 24 force-sync-cooldown-seconds: 60 running-stale-minutes: 5 achievement-ttl-hours: 1 schema-ttl-days: 30 + speedrun-ttl-days: 30 resilience4j: retry: @@ -75,6 +80,11 @@ resilience4j: limit-for-period: 2 limit-refresh-period: 1s timeout-duration: 5s + speedrunApi: + # speedrun.com allows 100 requests/min per IP. + limit-for-period: 1 + limit-refresh-period: 1s + timeout-duration: 30s igdbApi: # IGDB allows 4 requests/s. Taken per HTTP request. limit-for-period: 4 diff --git a/src/main/resources/db/migration/V10__game_speedrun.sql b/src/main/resources/db/migration/V10__game_speedrun.sql new file mode 100644 index 0000000..12b7ab3 --- /dev/null +++ b/src/main/resources/db/migration/V10__game_speedrun.sql @@ -0,0 +1,9 @@ +-- The speedrun.com game per library key (Steam appid, or minus the IGDB id), shared by all users. Only a game with at +-- least one verified run gets an id and weblink. A row with both NULL means no such game was found; it is kept so the +-- game is not asked again before it is stale. +CREATE TABLE game_speedrun ( + appid BIGINT PRIMARY KEY, + speedrun_game_id VARCHAR(16), + weblink VARCHAR(512), + checked_at TIMESTAMPTZ NOT NULL +); diff --git a/src/test/java/core_service/architecture/ArchitectureTest.java b/src/test/java/core_service/architecture/ArchitectureTest.java index e52a464..4e4d1ef 100644 --- a/src/test/java/core_service/architecture/ArchitectureTest.java +++ b/src/test/java/core_service/architecture/ArchitectureTest.java @@ -17,7 +17,7 @@ import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses; * Notes on Core-specific structure (differs from auth-service): * - JPA entities live in `..domain..` (not `..persistence..`). * - Repository interfaces live in `..repo..`. - * - `steam_adapter` is an outbound port consumed by feature modules + * - `steam_adapter`, `igdb_adapter` and `speedrun_adapter` are outbound ports consumed by feature modules * (e.g. `library`), so strict slice independence is relaxed for it. */ @AnalyzeClasses(packages = "core_service", importOptions = ImportOption.DoNotIncludeTests.class) @@ -33,8 +33,9 @@ class ArchitectureTest extends AbstractArchitectureTest { .should().notDependOnEachOther() .ignoreDependency( JavaClass.Predicates.resideInAPackage("core_service.modules.library.."), - JavaClass.Predicates.resideInAnyPackage("core_service.modules.steam_adapter..", "core_service.modules.igdb_adapter..")) - .because("Feature modules should be isolated; steam_adapter and igdb_adapter are explicit shared ports."); + JavaClass.Predicates.resideInAnyPackage("core_service.modules.steam_adapter..", "core_service.modules.igdb_adapter..", + "core_service.modules.speedrun_adapter..")) + .because("Feature modules should be isolated; steam_adapter, igdb_adapter and speedrun_adapter are explicit shared ports."); @ArchTest static final ArchRule no_cyclic_dependencies = SlicesRuleDefinition.slices() diff --git a/src/test/java/core_service/modules/igdb_adapter/service/IgdbApiServiceTest.java b/src/test/java/core_service/modules/igdb_adapter/service/IgdbApiServiceTest.java index fb7218f..5f88adc 100644 --- a/src/test/java/core_service/modules/igdb_adapter/service/IgdbApiServiceTest.java +++ b/src/test/java/core_service/modules/igdb_adapter/service/IgdbApiServiceTest.java @@ -12,6 +12,7 @@ import org.springframework.web.client.RestClient; import java.util.List; import java.util.Map; import java.util.Optional; +import java.util.Set; import java.util.stream.LongStream; import static org.assertj.core.api.Assertions.assertThat; @@ -126,6 +127,39 @@ class IgdbApiServiceTest extends AbstractUnitTest { assertThat(tags.get(620L).themes()).contains("Horror"); } + @Test + void getReleaseYears_SteamAppLinkedToTwoIgdbGames_GetsBothYears() { + expectToken(); + igdb.expect(requestTo(containsString("/external_games"))) + .andRespond(withSuccess("[{\"game\": 72, \"uid\": \"620\"}, {\"game\": 233, \"uid\": \"620\"}, {\"game\": 9, \"uid\": \"220\"}]", + MediaType.APPLICATION_JSON)); + igdb.expect(requestTo(containsString("/games"))) + .andExpect(content().string(allOf(containsString("fields first_release_date;"), containsString("id = (")))) + // 2006-06-01 and 2017-05-05 UTC; game 9 has no date + .andRespond(withSuccess("[{\"id\": 72, \"first_release_date\": 1149120000}, {\"id\": 233, \"first_release_date\": 1493942400}, {\"id\": 9}]", + MediaType.APPLICATION_JSON)); + + assertThat(service.getReleaseYears(List.of(620L, 220L))).contains(Map.of(620L, Set.of(2006, 2017))); + } + + @Test + void getReleaseYearsByGameIds_MapsTheIgdbIdToItsYear() { + expectToken(); + igdb.expect(requestTo(containsString("/games"))) + .andExpect(content().string(containsString("id = (1025)"))) + .andRespond(withSuccess("[{\"id\": 1025, \"first_release_date\": 1493942400}]", MediaType.APPLICATION_JSON)); + + assertThat(service.getReleaseYearsByGameIds(List.of(1025L))).contains(Map.of(1025L, Set.of(2017))); + } + + @Test + void getReleaseYears_IgdbError_IsEmpty() { + expectToken(); + igdb.expect(requestTo(containsString("/external_games"))).andRespond(withServerError()); + + assertThat(service.getReleaseYears(List.of(620L))).isEmpty(); + } + @Test void getGameTags_NoSteamAppKnown_AsksNoGames() { expectToken(); diff --git a/src/test/java/core_service/modules/library/api/LibraryControllerPickerTest.java b/src/test/java/core_service/modules/library/api/LibraryControllerPickerTest.java index 3368ff1..9773402 100644 --- a/src/test/java/core_service/modules/library/api/LibraryControllerPickerTest.java +++ b/src/test/java/core_service/modules/library/api/LibraryControllerPickerTest.java @@ -7,11 +7,13 @@ import core_service.modules.library.domain.SteamGame; import core_service.modules.library.domain.UserGame; import core_service.modules.library.domain.UserGameId; import core_service.modules.library.repo.GameMoodsRepository; +import core_service.modules.library.repo.GameSpeedrunRepository; import core_service.modules.library.repo.GameTimeToBeatRepository; import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.UserGameRepository; import core_service.modules.library.service.LibrarySyncService; import core_service.modules.library.service.MoodService; +import core_service.modules.library.service.SpeedrunService; import core_service.shared.test.AbstractIntegrationTest; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -26,6 +28,7 @@ import java.util.List; import static org.hamcrest.Matchers.containsInAnyOrder; import static org.hamcrest.Matchers.hasSize; import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; @@ -59,11 +62,18 @@ class LibraryControllerPickerTest extends AbstractIntegrationTest { @MockitoBean private MoodService moodService; + @MockitoBean + private SpeedrunService speedrunService; + + @Autowired + private GameSpeedrunRepository gameSpeedrunRepository; + @BeforeEach void clean() { userGameRepository.deleteAll(); gameTimeToBeatRepository.deleteAll(); gameMoodsRepository.deleteAll(); + gameSpeedrunRepository.deleteAll(); } private void game(long appid, GameStatus status, GamePriority priority, Integer normallyMinutes) { @@ -95,6 +105,21 @@ class LibraryControllerPickerTest extends AbstractIntegrationTest { verify(moodService).refreshBacklogInBackground(USER_ID); } + @Test + void picker_Speedrun_PicksOnlyGamesWithASpeedrunLink() throws Exception { + game(10L, GameStatus.BACKLOG, null, null); + game(20L, GameStatus.BACKLOG, null, null); + game(30L, GameStatus.BACKLOG, null, null); + gameSpeedrunRepository.upsert(10L, "o1y9j9v6", "https://www.speedrun.com/celeste", Instant.now()); + gameSpeedrunRepository.upsert(20L, null, null, Instant.now()); + + mockMvc.perform(get("/api/library/picker").param("mood", "SPEEDRUN")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.games[*].appid", containsInAnyOrder(10))); + verify(speedrunService).refreshBacklogInBackground(USER_ID); + verifyNoInteractions(moodService); + } + @Test void picker_MoodAndBudget_BothMustFit() throws Exception { game(10L, GameStatus.BACKLOG, null, 300); diff --git a/src/test/java/core_service/modules/library/repo/GameSpeedrunRepositoryTest.java b/src/test/java/core_service/modules/library/repo/GameSpeedrunRepositoryTest.java new file mode 100644 index 0000000..18411d8 --- /dev/null +++ b/src/test/java/core_service/modules/library/repo/GameSpeedrunRepositoryTest.java @@ -0,0 +1,51 @@ +package core_service.modules.library.repo; + +import core_service.modules.library.domain.GameSpeedrun; +import core_service.shared.test.AbstractPersistenceTest; +import jakarta.persistence.EntityManager; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; + +import java.time.Duration; +import java.time.Instant; +import java.util.List; +import java.util.Set; + +import static org.assertj.core.api.Assertions.assertThat; + +class GameSpeedrunRepositoryTest extends AbstractPersistenceTest { + + @Autowired + private GameSpeedrunRepository gameSpeedrunRepository; + + @Autowired + private EntityManager entityManager; + + @BeforeEach + void clean() { + gameSpeedrunRepository.deleteAll(); + } + + @Test + void upsert_StoresNoMatchAndReplacesItWithTheLinkLater() { + gameSpeedrunRepository.upsert(620L, null, null, Instant.now()); + assertThat(gameSpeedrunRepository.findById(620L).orElseThrow().getWeblink()).isNull(); + + gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2", Instant.now()); + // The native upsert passes the persistence context; read the row again. + entityManager.clear(); + GameSpeedrun row = gameSpeedrunRepository.findById(620L).orElseThrow(); + assertThat(row.getSpeedrunGameId()).isEqualTo("om1mw4d2"); + assertThat(row.getWeblink()).isEqualTo("https://www.speedrun.com/portal_2"); + } + + @Test + void findFreshAppids_LeavesOutOldAndMissingRows() { + gameSpeedrunRepository.upsert(620L, null, null, Instant.now()); + gameSpeedrunRepository.upsert(220L, null, null, Instant.now().minus(Duration.ofDays(40))); + + assertThat(gameSpeedrunRepository.findFreshAppids(List.of(620L, 220L, 700L), Instant.now().minus(Duration.ofDays(30)))) + .isEqualTo(Set.of(620L)); + } +} diff --git a/src/test/java/core_service/modules/library/service/GameDetailServiceTest.java b/src/test/java/core_service/modules/library/service/GameDetailServiceTest.java index a5a2652..cb3c6b4 100644 --- a/src/test/java/core_service/modules/library/service/GameDetailServiceTest.java +++ b/src/test/java/core_service/modules/library/service/GameDetailServiceTest.java @@ -10,6 +10,7 @@ import core_service.modules.library.domain.UserGameAchievementMeta; import core_service.modules.library.repo.GameAchievementSchemaMetaRepository; import core_service.modules.library.repo.SteamGameRepository; import core_service.modules.library.repo.GameAchievementSchemaRepository; +import core_service.modules.library.repo.GameSpeedrunRepository; import core_service.modules.library.repo.UserGameAchievementMetaRepository; import core_service.modules.library.repo.UserGameAchievementRepository; import core_service.modules.library.repo.UserGameRepository; @@ -70,6 +71,12 @@ class GameDetailServiceTest extends AbstractIntegrationTest { @MockitoBean private MoodService moodService; + @MockitoBean + private SpeedrunService speedrunService; + + @Autowired + private GameSpeedrunRepository gameSpeedrunRepository; + @BeforeEach void seed() { userGameRepository.deleteAll(); @@ -77,6 +84,7 @@ class GameDetailServiceTest extends AbstractIntegrationTest { schemaMetaRepository.deleteAll(); userAchievementRepository.deleteAll(); userAchievementMetaRepository.deleteAll(); + gameSpeedrunRepository.deleteAll(); steamGameRepository.save(new SteamGame(620L, "Portal 2", null, Instant.now())); userGameRepository.save(new UserGame(USER_ID, 620L, 60, Instant.now(), Instant.now())); } @@ -106,6 +114,23 @@ class GameDetailServiceTest extends AbstractIntegrationTest { org.mockito.Mockito.verify(moodService).refreshInBackground(List.of(620L)); } + @Test + void getGameDetail_ShowsTheSpeedrunLinkAndAsksForIt() { + gameSpeedrunRepository.upsert(620L, "om1mw4d2", "https://www.speedrun.com/portal_2", Instant.now()); + + GameDetailDto detail = gameDetailService.getGameDetail(USER_ID, null, 620L).orElseThrow(); + + assertEquals("https://www.speedrun.com/portal_2", detail.speedrunUrl()); + org.mockito.Mockito.verify(speedrunService).refreshInBackground(List.of(620L)); + } + + @Test + void getGameDetail_NoSpeedrunGame_HasNoLink() { + gameSpeedrunRepository.upsert(620L, null, null, Instant.now()); + + assertEquals(null, gameDetailService.getGameDetail(USER_ID, null, 620L).orElseThrow().speedrunUrl()); + } + @Test void getGameDetail_StaleUserMeta_ShowsTheRefreshedCount() { schemaMetaRepository.save(new GameAchievementSchemaMeta(620L, 5, true, Instant.now())); diff --git a/src/test/java/core_service/modules/library/service/SpeedrunMatchTest.java b/src/test/java/core_service/modules/library/service/SpeedrunMatchTest.java new file mode 100644 index 0000000..c085067 --- /dev/null +++ b/src/test/java/core_service/modules/library/service/SpeedrunMatchTest.java @@ -0,0 +1,71 @@ +package core_service.modules.library.service; + +import core_service.modules.speedrun_adapter.service.SpeedrunApiService; +import core_service.shared.test.AbstractUnitTest; +import org.junit.jupiter.api.Test; + +import java.util.List; +import java.util.Set; + +import static org.assertj.core.api.Assertions.assertThat; + +class SpeedrunMatchTest extends AbstractUnitTest { + + private static SpeedrunApiService.Game hit(String id, String name, Integer year) { + return new SpeedrunApiService.Game(id, name, year, "https://www.speedrun.com/" + id); + } + + private static final SpeedrunApiService.Game PREY_2006 = hit("prey", "Prey", 2006); + private static final SpeedrunApiService.Game PREY_2017 = hit("prey2017", "Prey (2017)", 2017); + + @Test + void normalize_DropsCaseAccentsTrademarksAndPunctuation() { + assertThat(SpeedrunMatch.normalize(" Pokémon™: Let's Go, Pikachu!® ")).isEqualTo("pokemon let s go pikachu"); + assertThat(SpeedrunMatch.normalize("The Witcher 3: Wild Hunt")).isEqualTo(SpeedrunMatch.normalize("The Witcher® 3 - Wild Hunt")); + } + + @Test + void candidates_YearSuffixTellsTheRemakeApart() { + assertThat(SpeedrunMatch.candidates("Prey", Set.of(2017), List.of(PREY_2006, PREY_2017))).containsExactly(PREY_2017); + assertThat(SpeedrunMatch.candidates("Prey", Set.of(2006), List.of(PREY_2006, PREY_2017))).containsExactly(PREY_2006); + } + + @Test + void candidates_UnknownYear_KeepsSpeedrunOrder() { + assertThat(SpeedrunMatch.candidates("Prey", Set.of(), List.of(PREY_2006, PREY_2017))).containsExactly(PREY_2006, PREY_2017); + } + + @Test + void candidates_OneYearOffIsStillTheSameGame() { + SpeedrunApiService.Game celeste = hit("celeste", "Celeste", 2018); + + assertThat(SpeedrunMatch.candidates("Celeste", Set.of(2017), List.of(celeste))).containsExactly(celeste); + assertThat(SpeedrunMatch.candidates("Celeste", Set.of(2015), List.of(celeste))).isEmpty(); + } + + @Test + void candidates_OnlyTheSameName() { + List hits = List.of(hit("snes", "Doom (SNES)", 1995), hit("doom3", "Doom 3", 2004), + hit("d2016", "DOOM", 2016)); + + assertThat(SpeedrunMatch.candidates("DOOM", Set.of(), hits)).extracting(SpeedrunApiService.Game::id).containsExactly("d2016"); + } + + @Test + void candidates_AtMostTwo() { + List hits = List.of(hit("a", "Tetris", null), hit("b", "Tetris", null), hit("c", "Tetris", null)); + + assertThat(SpeedrunMatch.candidates("Tetris", Set.of(), hits)).extracting(SpeedrunApiService.Game::id).containsExactly("a", "b"); + } + + @Test + void candidates_RefusesLinksOutsideSpeedrunCom() { + List hits = List.of( + new SpeedrunApiService.Game("a", "Celeste", 2018, "javascript:alert(1)"), + new SpeedrunApiService.Game("b", "Celeste", 2018, "https://www.speedrun.com.evil.example/x"), + new SpeedrunApiService.Game("c", "Celeste", 2018, null), + new SpeedrunApiService.Game("d", "Celeste", 2018, SpeedrunMatch.WEBLINK_PREFIX + "x".repeat(500))); + + assertThat(SpeedrunMatch.candidates("Celeste", Set.of(), hits)).isEmpty(); + } +} diff --git a/src/test/java/core_service/modules/library/service/SpeedrunServiceTest.java b/src/test/java/core_service/modules/library/service/SpeedrunServiceTest.java new file mode 100644 index 0000000..f448fb5 --- /dev/null +++ b/src/test/java/core_service/modules/library/service/SpeedrunServiceTest.java @@ -0,0 +1,225 @@ +package core_service.modules.library.service; + +import core_service.modules.igdb_adapter.service.IgdbApiService; +import core_service.modules.library.domain.SteamGame; +import core_service.modules.library.repo.GameSpeedrunRepository; +import core_service.modules.library.repo.SteamGameRepository; +import core_service.modules.library.repo.UserGameRepository; +import core_service.modules.speedrun_adapter.service.SpeedrunApiService; +import core_service.shared.test.AbstractUnitTest; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockito.InjectMocks; +import org.mockito.Mock; +import org.springframework.dao.DataAccessResourceFailureException; + +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.stream.LongStream; + +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isNull; +import static org.mockito.Mockito.doThrow; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; +import static org.mockito.Mockito.when; + +class SpeedrunServiceTest extends AbstractUnitTest { + + @Mock + private SpeedrunApiService speedrunApiService; + @Mock + private IgdbApiService igdbApiService; + @Mock + private GameSpeedrunRepository gameSpeedrunRepository; + @Mock + private SteamGameRepository steamGameRepository; + @Mock + private UserGameRepository userGameRepository; + @InjectMocks + private SpeedrunService service; + + private static final String LINK = "https://www.speedrun.com/prey2017"; + + @BeforeEach + void setUp() { + when(speedrunApiService.isEnabled()).thenReturn(true); + } + + private void names(long... appids) { + when(steamGameRepository.findAllById(any())).thenAnswer(call -> { + Iterable asked = call.getArgument(0); + List games = new ArrayList<>(); + for (Long appid : asked) { + if (LongStream.of(appids).anyMatch(known -> known == appid)) { + games.add(new SteamGame(appid, "Prey", null, Instant.EPOCH)); + } + } + return games; + }); + } + + private static SpeedrunApiService.Game hit(String id, String name, int year) { + return new SpeedrunApiService.Game(id, name, year, "https://www.speedrun.com/" + id); + } + + @Test + void refresh_GameWithRuns_StoresTheLink() { + names(620); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.of(List.of(hit("prey2017", "Prey (2017)", 2017)))); + when(speedrunApiService.hasVerifiedRuns("prey2017")).thenReturn(Optional.of(true)); + + service.refresh(List.of(620L)); + + verify(gameSpeedrunRepository).upsert(eq(620L), eq("prey2017"), eq(LINK), any()); + } + + @Test + void refresh_UsesTheIgdbYearToPickTheRemake() { + names(620); + when(igdbApiService.isEnabled()).thenReturn(true); + when(igdbApiService.getReleaseYears(List.of(620L))).thenReturn(Optional.of(Map.of(620L, Set.of(2017)))); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.of(List.of( + hit("prey", "Prey", 2006), hit("prey2017", "Prey (2017)", 2017)))); + when(speedrunApiService.hasVerifiedRuns("prey2017")).thenReturn(Optional.of(true)); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, never()).hasVerifiedRuns("prey"); + verify(gameSpeedrunRepository).upsert(eq(620L), eq("prey2017"), eq(LINK), any()); + } + + @Test + void refresh_NoGameWithRuns_StoresAnEmptyRowAfterTwoCandidates() { + names(620); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.of(List.of( + hit("a", "Prey", 2006), hit("b", "Prey", 2006), hit("c", "Prey", 2006)))); + when(speedrunApiService.hasVerifiedRuns(anyString())).thenReturn(Optional.of(false)); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, times(2)).hasVerifiedRuns(anyString()); + verify(gameSpeedrunRepository).upsert(eq(620L), isNull(), isNull(), any()); + } + + @Test + void refresh_FailedCall_StoresNothing() { + names(620); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.of(List.of(hit("prey", "Prey", 2006)))); + when(speedrunApiService.hasVerifiedRuns("prey")).thenReturn(Optional.empty()); + + service.refresh(List.of(620L)); + + verify(gameSpeedrunRepository, never()).upsert(any(), any(), any(), any()); + } + + @Test + void refresh_FreshGame_IsNotAskedAgain() { + when(gameSpeedrunRepository.findFreshAppids(any(), any())).thenReturn(Set.of(620L)); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, never()).findGames(anyString()); + } + + @Test + void refresh_StopsAfterThreeErrorsInARow() { + names(1, 2, 3, 4, 5); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.empty()); + + service.refresh(List.of(1L, 2L, 3L, 4L, 5L)); + + verify(speedrunApiService, times(SpeedrunService.MAX_ERRORS_IN_A_ROW)).findGames("Prey"); + } + + @Test + void refresh_CalledDuringARun_QueuesTheGameForTheRunningThread() { + names(620, 220); + when(speedrunApiService.findGames("Prey")).thenAnswer(call -> { + // A second caller while the first one drains: it must not start a drain of its own. + service.refresh(List.of(220L)); + return Optional.of(List.of()); + }).thenReturn(Optional.of(List.of())); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, times(2)).findGames("Prey"); + verify(gameSpeedrunRepository).upsert(eq(620L), isNull(), isNull(), any()); + verify(gameSpeedrunRepository).upsert(eq(220L), isNull(), isNull(), any()); + } + + @Test + void refresh_DbErrorOnOneGame_TheOtherGamesOfTheBatchAreStillChecked() { + names(1, 2); + when(speedrunApiService.findGames("Prey")).thenReturn(Optional.of(List.of())); + doThrow(new DataAccessResourceFailureException("connection reset")) + .when(gameSpeedrunRepository).upsert(eq(1L), any(), any(), any()); + + service.refresh(List.of(1L, 2L)); + + verify(gameSpeedrunRepository).upsert(eq(2L), isNull(), isNull(), any()); + } + + @Test + void refresh_DbErrorOnOneGame_GamesQueuedByOtherCallersAreStillChecked() { + names(1, 3); + when(speedrunApiService.findGames("Prey")).thenAnswer(call -> { + service.refresh(List.of(3L)); + return Optional.of(List.of()); + }).thenReturn(Optional.of(List.of())); + doThrow(new DataAccessResourceFailureException("connection reset")) + .when(gameSpeedrunRepository).upsert(eq(1L), any(), any(), any()); + + service.refresh(List.of(1L)); + + verify(gameSpeedrunRepository).upsert(eq(3L), isNull(), isNull(), any()); + } + + @Test + void refresh_IgdbFailed_StoresNothingInsteadOfMatchingByNameOnly() { + names(620); + when(igdbApiService.isEnabled()).thenReturn(true); + when(igdbApiService.getReleaseYears(List.of(620L))).thenReturn(Optional.empty()); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, never()).findGames(anyString()); + verify(gameSpeedrunRepository, never()).upsert(any(), any(), any(), any()); + } + + @Test + void refresh_SearchesWithoutTrademarkSigns() { + when(steamGameRepository.findAllById(any())).thenReturn(List.of(new SteamGame(620L, "DOOM Eternal™", null, Instant.EPOCH))); + when(speedrunApiService.findGames("DOOM Eternal")).thenReturn(Optional.of(List.of())); + + service.refresh(List.of(620L)); + + verify(gameSpeedrunRepository).upsert(eq(620L), isNull(), isNull(), any()); + } + + @Test + void refresh_GameWithoutASteamName_IsNotSearched() { + when(steamGameRepository.findAllById(any())).thenReturn(List.of(new SteamGame(620L, SteamGame.placeholderName(620L), null, Instant.EPOCH))); + + service.refresh(List.of(620L)); + + verify(speedrunApiService, never()).findGames(anyString()); + } + + @Test + void refresh_Disabled_DoesNothing() { + when(speedrunApiService.isEnabled()).thenReturn(false); + + service.refresh(List.of(620L)); + + verifyNoInteractions(gameSpeedrunRepository, steamGameRepository); + } +} diff --git a/src/test/java/core_service/modules/speedrun_adapter/service/SpeedrunApiServiceTest.java b/src/test/java/core_service/modules/speedrun_adapter/service/SpeedrunApiServiceTest.java new file mode 100644 index 0000000..dc7f73b --- /dev/null +++ b/src/test/java/core_service/modules/speedrun_adapter/service/SpeedrunApiServiceTest.java @@ -0,0 +1,97 @@ +package core_service.modules.speedrun_adapter.service; + +import core_service.shared.test.AbstractUnitTest; +import io.github.resilience4j.ratelimiter.RateLimiterRegistry; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.http.MediaType; +import org.springframework.test.web.client.MockRestServiceServer; +import org.springframework.web.client.RestClient; + +import java.util.List; +import java.util.Optional; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.hamcrest.CoreMatchers.containsString; +import static org.springframework.test.web.client.match.MockRestRequestMatchers.queryParam; +import static org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo; +import static org.springframework.test.web.client.response.MockRestResponseCreators.withServerError; +import static org.springframework.test.web.client.response.MockRestResponseCreators.withSuccess; + +class SpeedrunApiServiceTest extends AbstractUnitTest { + + private MockRestServiceServer server; + private SpeedrunApiService service; + + private SpeedrunApiService create(boolean enabled) { + RestClient.Builder builder = RestClient.builder(); + server = MockRestServiceServer.bindTo(builder).build(); + return new SpeedrunApiService(builder.build(), enabled, RateLimiterRegistry.ofDefaults()); + } + + @BeforeEach + void setUp() { + service = create(true); + } + + @Test + void findGames_ReadsGamesAndDropsModsAndCategoryExtensions() { + server.expect(requestTo(containsString("/games?name=Portal%202&max=20"))) + .andRespond(withSuccess(""" + {"data": [ + {"id": "om1mw4d2", "names": {"international": "Portal 2"}, "released": 2011, + "weblink": "https://www.speedrun.com/portal_2", "links": [{"rel": "self"}, {"rel": "leaderboard"}]}, + {"id": "v1pxk8p6", "names": {"international": "Portal 2 Bhop"}, "released": 2012, + "weblink": "https://www.speedrun.com/p2bh", "links": [{"rel": "base-game"}]}, + {"id": "x", "names": {"international": " "}} + ]} + """, MediaType.APPLICATION_JSON)); + + assertThat(service.findGames("Portal 2")).contains(List.of( + new SpeedrunApiService.Game("om1mw4d2", "Portal 2", 2011, "https://www.speedrun.com/portal_2"))); + } + + @Test + void findGames_EncodesTheWholeName() { + server.expect(requestTo(containsString("name=Tom%20%26%20Jerry%E2%84%A2%2B&max"))) + .andRespond(withSuccess("{\"data\": []}", MediaType.APPLICATION_JSON)); + + assertThat(service.findGames("Tom & Jerry™+")).contains(List.of()); + } + + @Test + void findGames_ServerError_IsEmpty() { + server.expect(requestTo(containsString("/games"))).andRespond(withServerError()); + + assertThat(service.findGames("Celeste")).isEmpty(); + } + + @Test + void findGames_Disabled_CallsNothing() { + service = create(false); + + assertThat(service.findGames("Celeste")).isEmpty(); + server.verify(); + } + + @Test + void hasVerifiedRuns_TrueWithARunFalseWithout() { + server.expect(requestTo(containsString("/runs"))) + .andExpect(queryParam("game", "o1y9j9v6")) + .andExpect(queryParam("status", "verified")) + .andExpect(queryParam("max", "1")) + .andRespond(withSuccess("{\"data\": [{\"id\": \"run1\"}]}", MediaType.APPLICATION_JSON)); + server.expect(requestTo(containsString("/runs"))) + .andRespond(withSuccess("{\"data\": []}", MediaType.APPLICATION_JSON)); + + assertThat(service.hasVerifiedRuns("o1y9j9v6")).isEqualTo(Optional.of(true)); + assertThat(service.hasVerifiedRuns("o1y9j9v6")).isEqualTo(Optional.of(false)); + } + + @Test + void hasVerifiedRuns_ServerError_IsEmpty() { + server.expect(requestTo(containsString("/runs"))).andRespond(withServerError()); + + assertThat(service.hasVerifiedRuns("o1y9j9v6")).isEmpty(); + } +} diff --git a/src/test/java/core_service/shared/test/AbstractIntegrationTest.java b/src/test/java/core_service/shared/test/AbstractIntegrationTest.java index c072b4e..56121a5 100644 --- a/src/test/java/core_service/shared/test/AbstractIntegrationTest.java +++ b/src/test/java/core_service/shared/test/AbstractIntegrationTest.java @@ -18,8 +18,9 @@ import java.util.Base64; * Full Spring context with Testcontainers. */ @Tag("IntegrationTest") -// The nightly sync would call Steam if a test ran at 23:55. -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = "app.library.nightly-sync-cron=-") +// The nightly sync would call Steam if a test ran at 23:55; speedrun.com needs no key, so it is switched off. +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, + properties = {"app.library.nightly-sync-cron=-", "app.speedrun.enabled=false"}) @ActiveProfiles("dev") public abstract class AbstractIntegrationTest {