Grant the next unowned chapter when a collectible is picked up

A pickup does not name what it granted: its only per-object content is a
position, so reproducing Bungie's choice of chapter would mean mapping every
object in the world to a reward from activity and spawn data. The chapter is
chosen here instead. A pickup grants the next chapter of its book the account
does not hold, which is what a player observes anyway.

The bubble the incident carries says which activity the pickup happened in, and
an activity's pickups feed one book, so the association needed is one entry per
activity rather than one per object. Only the Menagerie is mapped so far.

Chapters are the node's children that name a lore row. The child naming none is
the book's parent triumph and is never granted, which would otherwise mark a
book complete without giving any of its contents.

Verified in game: three pickups granted records 1708 to 1710 and Confessions
reports three triumphs complete.

Sunrise/Sunrise.vcxproj

@@ -830,6 +830,7 @@
     <ClCompile Include="src\state\build_data\nodes\node_build_data_runtime.cpp" />
     <ClCompile Include="src\state\build_data\nodes\node_catalog.cpp" />
     <ClCompile Include="src\state\record_claims\record_claims.cpp" />
+    <ClCompile Include="src\state\lore\lore_grant.cpp" />
   </ItemGroup>
   <ItemGroup Condition="'$(SunriseRunClangTidy)'=='true'">
     <ClCompile Remove="vendor\detours\detours.cpp" />
@@ -1441,6 +1442,7 @@
     <ClInclude Include="src\state\build_data\nodes\definition.h" />
     <ClInclude Include="src\state\build_data\nodes\node_catalog.h" />
     <ClInclude Include="src\state\record_claims\record_claims.h" />
+    <ClInclude Include="src\state\lore\lore_grant.h" />
   </ItemGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
 </Project>

Sunrise/src/client/content/items/packages/package_record_build.cpp

@@ -165,6 +165,22 @@ bool build_records(const reader::Source& source,
         }
         ++count;
     }
+    // TEMPORARY: what score a lore record actually carries in this build. The published manifest
+    // shows ScoreValue 0 for Confessions Entry I, which does not match the game, so read the rows
+    // rather than trust either. 1707 is the book's parent triumph and 1708 to 1716 its chapters.
+    for (std::uint64_t row = 1707; row <= 1716 && row < count; ++row) {
+        std::array<char, 180> line{};
+        const int told = std::snprintf(
+            line.data(), line.size(), "ev=lorescore record=%llu score=%u lore_row=%u flag=%u",
+            static_cast<unsigned long long>(row), static_cast<unsigned>(output[row].scoreValue),
+            static_cast<unsigned>(output[row].loreRow),
+            static_cast<unsigned>(output[row].completionFlagIndex));
+        if (told > 0) {
+            core::log::write(core::log::Channel::client, core::log::Level::info,
+                             {line.data(), static_cast<std::size_t>(told)});
+        }
+    }
+
     report("ok", static_cast<unsigned long long>(count));
     return count != 0;
 }

Sunrise/src/server/bap/encrypted/activity_message/receipts/activity_message_receipts.cpp

@@ -4,6 +4,7 @@
  * was read so the caller can record one arrival receipt. None of them acts on what it read.
  */
 
+#include "../../../../../state/lore/lore_grant.h"
 #include <vector>
 #include <algorithm>
 #include "../../../../../state/build_data/collectibles/collectible_catalog.h"
@@ -342,42 +343,41 @@ Framed frame_incident(const message::Request& request) noexcept {
             const std::uint64_t sequence = take(32);
             const std::uint64_t kindRaw = take(3);
             const std::uint64_t identity = take(64);
-            // The nested block follows the header. Its first field is a type-25 tagged union
-            // occupying bits 99 to 122, and the collectible sits in its low fifteen bits -- the same
-            // width Collections uses for a native row index. The bound was not guessed: the bubble
-            // hash appears at bits 126, 158 and 190, three consecutive u32 fields of the nested
-            // block, which fixes where the union has to end, and the character SOID lands at bit 35
-            // exactly where the header schema puts it.
-            // Only type 2 has this layout. Every other type code selects a different schema, and
-            // decoding one of those against this one produces a plausible looking index that means
-            // nothing: target 1121 is type 13 and yielded 21830 that way.
+            // Which book. The payload does not name the chapter it granted -- its only
+            // per-object content is a position -- so the reward is chosen here instead. The bubble
+            // does say which activity the pickup happened in, and an activity's pickups feed one
+            // book, which is a far smaller association than one entry per object in the world.
+            //
+            // The bubble sits at bit 126, the first of three consecutive u32 fields of the nested
+            // block. That position is measured: the same hash appears at 126, 158 and 190, and the
+            // character SOID lands at bit 35 exactly where the header schema puts it.
+            //
+            // Only type 2 carries this layout. Another type read against it yields a plausible
+            // number that means nothing, which is how a counter was once mistaken for an identity.
             constexpr std::int32_t kPickupTypeCode = 2;
+            constexpr std::size_t kBubbleBit = 126;
             state::build_data::sobjects::Definition row{};
             const bool pickupType =
                 state::build_data::sobjects::find(static_cast<std::uint16_t>(parsed.primaryTarget),
                                                   row)
                 && row.typeCode == kPickupTypeCode;
-
-            constexpr std::size_t kUnionBits = 24;
-            constexpr std::size_t kIndexBits = 15;
-            std::uint32_t collectibleIndex = 0;
-            if (pickupType && body.size() * 8 >= 99 + kUnionBits) {
-                const std::uint64_t unionField = take(kUnionBits);
-                collectibleIndex =
-                    static_cast<std::uint32_t>(unionField & ((1U << kIndexBits) - 1U));
-                state::build_data::collectibles::Definition collectible{};
-                if (state::build_data::collectibles::find(
-                        static_cast<std::uint16_t>(collectibleIndex), collectible)) {
-                    report(core::log::Level::info,
-                           "ev=activity stage=collectible index=%u hash=0x%08X item=%u",
-                           collectibleIndex,
-                           collectible.collectibleHash,
-                           static_cast<unsigned>(collectible.itemDefinitionIndex));
-                } else {
-                    report(core::log::Level::warn,
-                           "ev=activity stage=collectible index=%u result=unknown rows=%zu",
-                           collectibleIndex, state::build_data::collectibles::count());
+            if (pickupType && body.size() * 8 >= kBubbleBit + 32) {
+                std::uint32_t bubble = 0;
+                for (std::size_t step = 0; step < 32; ++step) {
+                    const std::size_t at = kBubbleBit + step;
+                    const auto byte = static_cast<std::uint8_t>(body[at / 8]);
+                    bubble = (bubble << 1U) | ((byte >> (7 - (at % 8))) & 1U);
                 }
+                const std::uint16_t node = state::lore::book_for_bubble(bubble);
+                const state::lore::GrantOutcome outcome = state::lore::grant_next_chapter(node);
+                report(outcome == state::lore::GrantOutcome::granted ? core::log::Level::info
+                                                                     : core::log::Level::warn,
+                       "ev=activity stage=lore bubble=0x%08X node=%u result=%s record=%u",
+                       bubble, static_cast<unsigned>(node),
+                       state::lore::grant_outcome_name(outcome),
+                       static_cast<unsigned>(outcome == state::lore::GrantOutcome::granted
+                                                 ? state::lore::last_granted_record()
+                                                 : 0));
             }
 
             report(core::log::Level::info,

Sunrise/src/state/lore/lore_grant.cpp

@@ -0,0 +1,124 @@
+#include "lore_grant.h"
+
+#include <array>
+#include <atomic>
+#include <cstddef>
+#include <span>
+#include <vector>
+
+#include "../build_data/nodes/definition.h"
+#include "../build_data/nodes/node_catalog.h"
+#include "../build_data/records/definition.h"
+#include "../build_data/runtime.h"
+#include "../record_claims/record_claims.h"
+
+namespace sunrise::state::lore {
+namespace {
+
+/**
+ * Which book an activity's pickups feed.
+ *
+ * One entry per activity, not one per object. This is the whole of the authored data the chosen
+ * design needs, and it is the part that would be replaced if Bungie's own object-to-reward mapping
+ * were ever reproduced from activity and spawn data.
+ *
+ * Only the Menagerie is known so far, measured from its pickups: the bubble its incidents carry
+ * against the book those pickups fill.
+ */
+struct BubbleBook {
+    std::uint32_t bubble;
+    std::uint16_t node;
+};
+
+constexpr std::array<BubbleBook, 1> kBubbleBooks{{
+    // caluseum_experience, whose vases fill Confessions.
+    {0x811C9DC5U, 838U},
+}};
+
+std::atomic<std::uint16_t> g_lastGranted{0};
+
+} // namespace
+
+/** @return A short name for the outcome, for logs. */
+const char* grant_outcome_name(GrantOutcome outcome) noexcept {
+    switch (outcome) {
+    case GrantOutcome::granted:
+        return "granted";
+    case GrantOutcome::unknownBook:
+        return "unknown_book";
+    case GrantOutcome::emptyBook:
+        return "empty_book";
+    case GrantOutcome::bookComplete:
+        return "book_complete";
+    case GrantOutcome::refused:
+        return "refused";
+    }
+    return "unknown";
+}
+
+/** @return The presentation node of the book that activity's pickups feed. */
+std::uint16_t book_for_bubble(std::uint32_t bubble) noexcept {
+    for (const BubbleBook& entry : kBubbleBooks) {
+        if (entry.bubble == bubble) {
+            return entry.node;
+        }
+    }
+    return kNoBook;
+}
+
+/** Grants the next chapter of one book that the account does not already hold. */
+GrantOutcome grant_next_chapter(std::uint16_t node) noexcept {
+    if (node == kNoBook) {
+        return GrantOutcome::unknownBook;
+    }
+
+    namespace nodes = build_data::nodes;
+    namespace records = build_data::records;
+    std::vector<nodes::Definition> rows(nodes::kDefinitionCapacity);
+    std::size_t count = 0;
+    if (!nodes::snapshot(std::span<nodes::Definition>{rows}, count)) {
+        return GrantOutcome::emptyBook;
+    }
+
+    const nodes::Definition* book = nullptr;
+    for (std::size_t row = 0; row < count; ++row) {
+        if (rows[row].definitionIndex == node) {
+            book = &rows[row];
+            break;
+        }
+    }
+    if (book == nullptr || book->childCount == 0) {
+        return GrantOutcome::emptyBook;
+    }
+
+    bool sawChapter = false;
+    for (std::size_t child = 0; child < book->childCount; ++child) {
+        records::Definition record{};
+        if (!build_data::find_record_definition(book->children[child], record)
+            || record.completionFlagIndex == records::kUnavailableFlagIndex) {
+            continue;
+        }
+        // A child naming no lore row is the book's parent triumph, not a chapter. Granting it would
+        // mark the book complete without giving any of its contents.
+        if (record.loreRow == records::kUnavailableLoreRow) {
+            continue;
+        }
+        sawChapter = true;
+        if (record_claims::claimed(record.completionFlagIndex)) {
+            continue;
+        }
+        if (!record_claims::claim(record.completionFlagIndex, record.scoreValue)) {
+            return GrantOutcome::refused;
+        }
+        g_lastGranted.store(record.definitionIndex, std::memory_order_relaxed);
+        return GrantOutcome::granted;
+    }
+    return sawChapter ? GrantOutcome::bookComplete : GrantOutcome::emptyBook;
+}
+
+/** @return The record row the last successful grant claimed. */
+std::uint16_t last_granted_record() noexcept {
+    return g_lastGranted.load(std::memory_order_relaxed);
+}
+
+} // namespace sunrise::state::lore

Sunrise/src/state/lore/lore_grant.h

@@ -0,0 +1,57 @@
+#pragma once
+
+#include <cstdint>
+
+namespace sunrise::state::lore {
+
+/**
+ * Granting a lore chapter when a collectible is picked up.
+ *
+ * The incident a pickup emits does not name what it granted. Its only per-object content is a
+ * position: the same object picked twice produces a payload differing by one bit, the sequence,
+ * while a different object differs only across the position vector. Reproducing Bungie's choice of
+ * chapter would mean mapping every object in the world to a reward, from activity and spawn data.
+ *
+ * So the chapter is chosen here instead. A pickup grants the next chapter of its book that the
+ * account does not already hold, which is what a player observes anyway: collect, and the book
+ * fills in. The association needed is one book per activity rather than one reward per object.
+ */
+
+/** A node index that names no book. */
+inline constexpr std::uint16_t kNoBook = 0xFFFFU;
+
+/** Why one pickup granted nothing. */
+enum class GrantOutcome : std::uint8_t {
+    granted,
+    /** The bubble is not associated with a book. */
+    unknownBook,
+    /** The book has no chapters this build can resolve. */
+    emptyBook,
+    /** Every chapter of the book is already held. */
+    bookComplete,
+    /** The claim store refused the write. */
+    refused,
+};
+
+/** @return A short name for the outcome, for logs. */
+[[nodiscard]] const char* grant_outcome_name(GrantOutcome outcome) noexcept;
+
+/**
+ * @param bubble Bubble hash the incident carried.
+ * @return The presentation node of the book that activity's pickups feed, or kNoBook.
+ */
+[[nodiscard]] std::uint16_t book_for_bubble(std::uint32_t bubble) noexcept;
+
+/**
+ * Grants the next chapter of one book that the account does not already hold.
+ * Chapters are the node's children that name a lore row; the child naming none is the book's parent
+ * triumph and is never granted.
+ * @param node Presentation node of the book.
+ * @return What happened, so a refusal reads differently from a completed book.
+ */
+[[nodiscard]] GrantOutcome grant_next_chapter(std::uint16_t node) noexcept;
+
+/** @return The record row the last successful grant claimed. Only meaningful after `granted`. */
+[[nodiscard]] std::uint16_t last_granted_record() noexcept;
+
+} // namespace sunrise::state::lore

Sunrise/src/state/record_claims/record_claims.cpp

@@ -370,6 +370,12 @@ std::size_t apply_character_node_progress(std::span<std::int32_t> characterValue
     return progress.written;
 }
 
+/** @return True when this index is already held. */
+bool claimed(std::uint16_t flagIndex) noexcept {
+    const std::lock_guard<std::mutex> guard(g_lock);
+    return claimed_locked(flagIndex);
+}
+
 /** @return Total score of every held claim. */
 std::uint32_t total_score() noexcept {
     const std::lock_guard<std::mutex> guard(g_lock);