# Public analytics with local and Arweave caches This is the implementation guide for this repository's browser-based analytics system. It explains the design, wire formats, loading algorithm, integration points, and acceptance checks needed to reproduce it in another site. It is a source implementation, not an npm package, agent skill, or automatic installer. The instructions below work for a developer or coding agent without this project's chat history. Reference source: [Jonny-Ringo/Jon_williams on GitHub](https://github.com/Jonny-Ringo/Jon_williams). This guide describes code revision `418a57ec1b4dea260495017bc95fbaae90566f9a`. Source links below are pinned to that revision so they also work when this document is shared separately from the repository. The system collects small public events, reads them through Arweave GraphQL, and accelerates the dashboard with local and publicly shared weekly snapshots. It runs on a static site without a private analytics server, API key, login, connected visitor wallet, or funded signing key. Public caches are readable by anyone. Discovery is by application and date tags, never the publisher's address. This site's dashboard is at `analytics/`. Its fixed event namespace is `Jon-Williams-Personal-Site-Analytics`, shared across its GitHub Pages and Arweave deployments. An unrelated site must choose its own namespace. ## Start here | Goal | Read or use | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | Understand the design | [Two cache layers](#two-cache-layers) and [loading flow](#load-refresh-and-correction-flow) | | Run this repository | [Run the reference implementation](#run-the-reference-implementation) | | Implement it in another site | [Implementation workflow](#implementation-workflow) and [integration example](#minimal-read-only-integration) | | Write a compatible reader or publisher | [Event protocol](#event-protocol-version-1), [cache protocol](#public-cache-protocol-version-1), and [local state](#local-state-and-update-contract) | | Verify correctness | [Acceptance criteria](#acceptance-criteria) and [diagnostics](#diagnosing-slow-or-missing-data) | ## Run the reference implementation Use Node.js 22.12 or later, as required by `package.json`, and the committed `package-lock.json`. For a fresh checkout of the reference revision: ```sh git clone https://github.com/Jonny-Ringo/Jon_williams.git cd Jon_williams git checkout 418a57ec1b4dea260495017bc95fbaae90566f9a npm ci npm run dev ``` Open the local URL printed by Vite and append `/analytics/`. Local previews can read real public data and write browser-local caches, but cannot publish events or public snapshots. Start with a fresh browser profile to observe a cold load; reload the same profile to observe the local-cache path. To run the reference checks: ```sh npm run test:analytics npm run build npx playwright install chromium npm test npm run test:deploy ``` Build before browser or deployment tests, because they inspect `dist/`. Automated tests intercept analytics requests and uploads. A manual dashboard preview, in contrast, queries the real public services. `npm run deploy:check` prepares an Arweave site preview without uploading; `npm run deploy` permanently publishes the site. Site deployment and publishing analytics snapshots are separate operations. Merely deploying the site does not seed a shared cache. ## Implementation workflow Use these steps as the task checklist for a port. Keep the data protocol and failure behavior intact even if the UI or framework changes. 1. **Choose the data boundary.** Decide which deployments should share activity. Give them one stable `APP_NAME`. Use a different name for an unrelated site or test environment. Keep a maintained deployment's existing name to retain its history. Choose the allowed pages, event types, and target values before collecting events; never copy arbitrary URLs or user input into tags. 2. **Bring over the data modules together.** Copy `shared.js`, `data-item.js`, `cache-format.js`, `cache-network.js`, and `cache.js` from `src/analytics/`, preserving their relative imports. These modules use native browser APIs; they do not need a runtime Arweave SDK. The Node tests use `arbundles` to independently verify the signer. For a reimplementation in another language, reproduce the contracts below and the same regression cases. 3. **Configure identity and schema.** Change `APP_NAME`, `EVENTS`, `TARGETS`, and the allowed pages in `parseEvent` in `shared.js`. `CACHE_APP` derives from `APP_NAME`. Give local cache, opt-out, session, and Web Lock keys a consistent site-specific prefix. These keys currently live in `cache-format.js`, `shared.js`, `tracker.js`, and `cache.js`. Update fixtures to match. 4. **Connect the optional tracker.** Copy `tracker.js`, import `startAnalytics`, and call it once per document from your site entry point. Adapt its page detection and delegated click selectors to the host site. This implementation recognizes `career-page` and `analytics-page` body classes, `data-project`, `data-resume-download`, and `data-copy-email`. For a single-page app, add explicit navigation handling and avoid installing duplicate listeners. 5. **Connect the loader to the UI.** Call `loadAnalytics` as shown below. Render every useful update; do not wait for the returned promise to resolve before showing cached data. Abort the previous controller on refresh, navigation, or date-range change. Keep incomplete and failed states distinguishable from a successfully checked empty period. 6. **Add publication deliberately.** The loader defaults to read-only. Pass the production/privacy gate from `dashboard.js` only when publishing is intended. Use `navigator.locks` when available. Keep signing lazy and recheck privacy immediately before each upload. Do not substitute a site's deployment wallet. 7. **Integrate the host build.** To reuse this UI, bring `analytics/index.html`, `dashboard.js`, and `dashboard.css` together, then adapt their DOM selectors, page labels, and shared CSS/font imports. Register the dashboard HTML as a build entry. This repository does that in `vite.config.js`, uses `base: './'`, and includes `analytics/` in its Arweave paths manifest. Preserve route-relative asset URLs when deploying below a manifest transaction ID. 8. **Validate before publishing.** Run the acceptance cases below using mocked network requests, then perform a read-only smoke test against the intended public namespace. Verify a fresh profile, a warm reload, privacy exclusions, and failed gateways. Publish the application only when the task includes deployment, using the host project's deployment process. Keep protocol version `1` only while the stored representation remains compatible. For an incompatible schema or canonicalization change, introduce a new event or cache version as appropriate and an explicit migration/read policy. Changing the local layout requires a versioned storage key or a migration. A new cache version does not itself migrate old events. Do not silently mix incompatible snapshots. ## Minimal read-only integration After copying the data modules under `src/analytics/`, this example can live in `src/analytics-example.js`. `onUpdate` is where the host application renders. The call intentionally omits `canPublish`, so it never creates public uploads. ```js import { dateRange } from './analytics/shared.js'; import { loadAnalytics } from './analytics/cache.js'; let active; export async function refreshAnalytics(onUpdate, days = 30) { if (![7, 30, 90].includes(days)) throw new Error('Unsupported date range'); active?.abort(); const controller = new AbortController(); active = controller; let storage; try { storage = window.localStorage; } catch { /* Read without local storage. */ } try { await loadAnalytics({ dates: dateRange(days), signal: controller.signal, storage, locks: navigator.locks, onUpdate(state) { if (!controller.signal.aborted) onUpdate(state); }, }); } catch (error) { if (!controller.signal.aborted) throw error; } } export function stopAnalytics() { active?.abort(); } ``` The reference targets browsers with ES modules, Fetch and streamed responses, Web Crypto, `AbortController`, `AbortSignal.any`, and `AbortSignal.timeout`. Provide equivalent cancellation support if targeting older browsers. Storage and Web Locks are optional optimizations. Web Crypto is needed to validate snapshot hashes and to sign uploads. If a content security policy is present, its network rules must allow the GraphQL/data gateway and transaction-specific subdomains under `*.arweave.net`; publishing also needs `https://up.arweave.net`. ## Two cache layers **Local:** Validated event records are kept in `localStorage` under `jw-analytics-cache-v1`. Repeat visits can render before network requests finish. **Public:** Live dashboards publish snapshots of completed seven-day UTC windows to Arweave. Other visitors discover them by public GraphQL tags and fetch their data directly from the gateway. Both layers copy already-public event fields. They accelerate reporting without becoming an authoritative source of visits. ```mermaid flowchart TD Visit[Eligible page views and clicks] --> Sign[Temporary browser signer] Sign --> Upload[Public Arweave event upload] Upload --> Index[Gateway GraphQL index] Open[Open analytics dashboard] --> Local[Read validated local snapshots] Local --> UI[Progressive dashboard] Open --> Discover[Discover public weekly manifests] Discover --> Parts[Download and validate snapshot parts] Parts --> UI Open --> Current[Index current week immediately] Open --> Grace[At most 750 ms cache head start] Grace --> History[Index history newest missing weeks first] Index --> Current Index --> History Current --> UI History --> UI History --> Save[Save completed scans locally] Save --> Publish[If permitted, publish missing or changed closed weeks] Publish --> Discover ``` Local storage and public snapshots accelerate different paths. Local storage is per origin and browser; public discovery is shared across all visitors and deployments using the same namespace. Public upload acceptance is followed by gateway indexing, so a newly uploaded cache may not be discoverable immediately. ## Load, refresh, and correction flow 1. Abort any previous load when the date range changes or Refresh is clicked. 2. Validate local data and immediately display the selected cached dates. 3. In parallel, discover public snapshots and index the current unfinished week. 4. Request completed weeks from newest to oldest, with up to three public cache downloads running at once. Try the newest usable snapshot for each week. Preserve a more recently checked local snapshot instead of replacing it. 5. Give public discovery and downloads up to 750 ms to render first. If they finish sooner, start raw-history queries immediately. Otherwise, start those queries alongside the cache downloads without waiting for gateway timeouts or the current week's query. 6. Scan missing weeks first, then cached weeks, newest first within each group and two raw-history queries at a time. Render each result as it arrives. Every selected week is checked once, including old weeks, to catch late indexing and corrections. A complete raw scan cancels that week's pending cache download; late snapshots cannot overwrite its verified results. 7. Replace each window after its complete scan, update local storage, and render corrected counts. Deduplicate by event ID and count sessions across the whole range, rather than adding weekly session totals. 8. After rendering, publish missing or changed completed snapshots if permitted. A publishing failure does not discard the loaded results. For example, if a public cache covers through Sunday the 28th and a visitor arrives on Wednesday the 31st, cached history can appear while the dashboard queries the 29th through the 31st. It then rechecks older days. A newly indexed event for an older week produces a new snapshot version for that week. The first visitor before any shared cache exists still needs to index records. Once a live visitor publishes and the gateway indexes the manifest, other visitors benefit. No background scheduler runs while the page is closed. This prevents a slow older cache from holding up recent activity or the raw-data fallback. Local data still renders immediately, and a fast public cache can render before raw history finishes. The 750 ms head start is not a promise that the full dashboard loads within that time; network speed and pagination still affect completion. If a raw scan is partial or fails, public downloads remain available as a fallback within their timeout. A late cache merges with partial raw results for display, while only its complete snapshot is saved locally. ## Seven-day windows and versions Windows are Monday through Sunday in UTC, anchored at 1970-01-05. All visitors therefore use the same keys regardless of local timezone, requested range, or deployment URL. A week becomes publishable only after all seven calendar days have ended. The current week remains local and is always re-indexed. The first/last requested dates can lie inside a window. Indexing fetches the whole available window to create a reusable snapshot, but only selected dates contribute to displayed counts. New empty weeks before tracking began are not published. An existing week's revision can be empty after a successful scan. Arweave snapshots are immutable. Changed canonical records produce a new hash and a new version; old versions remain available. Readers choose the newest usable candidate in gateway `HEIGHT_DESC` order, not by a publisher-controlled timestamp. Older records are always rechecked on a later dashboard load. Unchanged snapshots are not republished. Before uploading, the client queries the public index again and compares validated payload hashes. A Web Lock prevents concurrent same-origin tab publishing where supported. A locally remembered accepted hash suppresses identical uploads for 24 hours while indexing catches up; changed hashes remain eligible immediately. Separate visitors can still race and publish identical snapshots. These duplicates never double-count events. Implement the publication decision in this order: 1. Require a closed UTC week, a complete raw scan from this load, and permission to publish. Neither cached data alone nor a truncated scan is sufficient. 2. Canonicalize the verified events and compute the full snapshot hash. 3. Acquire `jw-analytics-cache:` with `ifAvailable: true` when Web Locks exists. If another same-origin tab holds it, skip this attempt. 4. Skip an identical locally recorded `publishedHash` accepted within 24 hours. 5. Query public manifests again. If discovery fails or is truncated, do not assume the snapshot is missing. Among up to three candidates, read the first usable one. Skip if its validated hash matches; otherwise publish a revision. 6. Skip a week with no events and no discovered candidates. This avoids creating empty snapshots for dates before tracking started. 7. Upload bounded parts, then the manifest. Save `publishedHash` and `publishedAt` only after the manifest receipt is accepted. Stop further publishing for this load after an upload failure; keep the rendered analytics available. This is best-effort duplicate suppression, not a distributed lock. A cache whose payload is unreadable may cause a redundant upload even when its tags are present. Different browsers can also publish simultaneously. Event-ID deduplication keeps these duplicates from becoming extra visits. ## Event protocol, version 1 The tracker uploads one signed JSON event per data item. The raw reader uses transaction **tags**, not the JSON body, so the writer must produce consistent values in both. Here is a synthetic event for November 1, 2026 at 12:00 UTC: ```json { "event": "page_view", "eventId": "0123456789abcdef0123456789abcdef", "sessionId": "abcdef0123456789abcdef0123456789", "timestamp": 1793534400000, "metadata": { "page": "home", "target": "" } } ``` | Event tag | Value or rule | | -------------- | -------------------------------------------------------------- | | `App-Name` | Stable event namespace | | `App-Version` | `1` | | `Content-Type` | `application/json` | | `Event-Type` | Allowed event name below | | `Event-ID` | 16 random bytes encoded as 32 lowercase hexadecimal characters | | `Session-ID` | Same encoding; session identity, not a wallet or person | | `Timestamp` | UTC date `YYYY-MM-DD`, derived from the event time | | `Event-Time` | Unix milliseconds encoded as a decimal string | | `Page` | `home` or `career` in this implementation | | `Target` | An allowed target, including the empty string where specified | | Event | Allowed targets | Current meaning | | ----------------- | --------------------------------- | ---------------------------------------------------------------------- | | `page_view` | `""` | Eligible home/career document load | | `project_open` | `auction`, `metrics`, `community` | Project-detail interaction | | `resume_download` | `pdf` | Resume link clicked, not verified file delivery | | `contact_click` | `email`, `gmail`, `copy` | Mail link or copy interaction; historical Gmail events remain readable | | `github_click` | `github` | Link to GitHub clicked | Current limitation: Calendly links are not instrumented by `tracker.js`. Do not interpret the contact metric as a count of scheduled calls. To add a new target, update both the tracking hook and the shared validator, with compatibility tests. Query raw transactions with `App-Name`, `App-Version`, and `Timestamp` tag filters. Use every available date of the requested UTC week, even when only part of that week lies in the displayed range. Request `id`, tags, cursor, and `hasNextPage`, using the same GraphQL connection shape as cache discovery below. Continue with the last edge's cursor until `hasNextPage` is false. Reject malformed responses, GraphQL errors, and repeated cursors. Hitting the page cap means **partial**, not complete, even if the events collected so far look valid. Validate a transaction ID against `^[A-Za-z0-9_-]{43}$`, enforce the namespace and event version, and reject unknown event/page/target values. Timestamps must be safe nonnegative integers no more than 60 seconds in the future. Require the UTC date derived from `Event-Time` to equal `Timestamp` and belong to the requested dates. Validate both random IDs against `^[a-f0-9]{32}$`. For an independent signer, reproduce ANS-104 type 1 or use a compatible encoder. The reference uses a 4096-bit RSA-PSS key, SHA-256, 32-byte salt, Avro-encoded tags, and the ANS-104 SHA-384 deep-hash signing input. The data-item ID is unpadded base64url of SHA-256 over its signature. Do not sign the JSON alone or infer the data-item ID from the payload hash. Prefer reusing `data-item.js` and its `arbundles` compatibility test over rewriting this binary format from prose. ## Public cache protocol, version 1 Public caches use a separate namespace so they cannot count as analytics events. | Tag | Value | | ---------------- | -------------------------------------------- | | `App-Name` | `Jon-Williams-Personal-Site-Analytics-Cache` | | `App-Version` | `1` | | `Source-App` | `Jon-Williams-Personal-Site-Analytics` | | `Source-Version` | `1` | | `Cache-Kind` | `manifest` or `part` | | `Cache-Start` | Monday in `YYYY-MM-DD` format | | `Cache-End` | Sunday in `YYYY-MM-DD` format | | `Cache-Hash` | Lowercase SHA-256 hex of canonical JSON | | `Content-Type` | `application/json` | Use the Mondays covering the requested period in this discovery query: ```graphql query SharedAnalytics($starts: [String!]!, $after: String) { transactions( first: 100 after: $after sort: HEIGHT_DESC tags: [ { name: "App-Name", values: ["Jon-Williams-Personal-Site-Analytics-Cache"] } { name: "App-Version", values: ["1"] } { name: "Cache-Kind", values: ["manifest"] } { name: "Cache-Start", values: $starts } ] ) { pageInfo { hasNextPage } edges { cursor node { id tags { name value } } } } } ``` Fetch manifests and parts at `https://arweave.net/`. A manifest contains `version`, `source`, `eventVersion`, `start`, `end`, `createdAt` (Unix milliseconds), `hash`, `count`, and `parts`, an array of `{ id, hash }` references. Each part is a JSON array of event tuples: ```text [transactionId, eventId, sessionId, eventType, page, target, timestampMilliseconds] ``` Dates are derived from timestamps in UTC. Records are schema-validated, deduplicated, and sorted by event ID. For conflicting duplicate event IDs, the lexicographically smaller transaction ID wins consistently. A part hash covers `JSON.stringify(part)`. The full hash covers `JSON.stringify(allCanonicalTuples)` and must equal the manifest's `Cache-Hash` tag. Counts must match. The manifest is uploaded last, after all referenced parts are accepted. The exact logical manifest shape is below. This is a type description, not a payload to upload with literal placeholder values: ```typescript type CacheManifest = { version: '1'; source: string; // Event APP_NAME, not CACHE_APP eventVersion: '1'; start: string; // Monday, YYYY-MM-DD end: string; // Sunday, YYYY-MM-DD createdAt: number; // Unix milliseconds after this week has ended hash: string; // 64-character lowercase SHA-256 hex for all tuples count: number; // Number of canonical events, not number of page views parts: Array<{ id: string; // Accepted 43-character data-item ID hash: string; // SHA-256 of this part's JSON bytes }>; ``` Canonicalization is part of the protocol: 1. Validate every event against the event schema and the complete week's dates. 2. Deduplicate by `eventId`; use the smaller transaction ID for conflicting IDs. 3. Sort by `eventId` ascending. The IDs are fixed-width lowercase hex strings. 4. Encode each event using the seven-element tuple order above. Timestamps are JSON numbers, not strings. Hash UTF-8 bytes of compact `JSON.stringify(rows)` with SHA-256 and encode the digest as lowercase hex. Do not hash pretty JSON, event objects with a different key order, locale-formatted numbers, or dates. 5. Split this ordered array into parts whose encoded JSON is at most 90 KiB. Each part is itself a JSON array, with its own hash. Preserve part order in the manifest. The full hash covers all tuples together, not the list of part hashes. `Cache-Hash` on a part is that part's hash; on a manifest it is the full snapshot hash. For an empty revision, `count` is zero, `parts` is empty, and `hash` is SHA-256 of the UTF-8 string `[]`. Do not emit an empty part. The no-empty-new-week rule still applies. Readers must validate manifest namespace, versions, matching start/end dates, tag hash, creation time, counts, and part references before accepting the cache. `createdAt` must be at or after the Monday following `end`, and no more than 60 seconds ahead of the reader's clock. Allow at most 20 parts and 5,000 total events. Fetch and hash each part, concatenate its rows in manifest order, then validate and canonicalize the events. Reject duplicate rows, mismatched counts, and a mismatched full hash. A matching hash is an integrity check, not evidence that the publisher reported honest activity. Cache reads follow redirects only to HTTPS `arweave.net` or its subdomains and enforce the byte cap on the streamed body, not just `Content-Length`. Uploads use `application/octet-stream`, omit credentials/referrers, reject redirects, and check the returned `id` against the locally calculated data-item ID. A nonzero `winc` charge or non-success HTTP response is a publication failure. Never add payment fallback to the reference's free-only path. ## Local state and update contract `loadAnalytics` accepts `dates`, `signal`, and `onUpdate`. Optional parameters are `storage`, `fetcher`, `canPublish`, `locks`, `now`, and an injected `publish` function. `dates` is an ascending list of UTC `YYYY-MM-DD` strings. `now` is one fixed millisecond timestamp for the load. Production callers should keep the range within the UI's 7/30/90-day choices; tests inject a fixed time and network. `storage` implements `getItem` and `setItem`. `fetcher` follows Fetch semantics, including cancellation; `canPublish` defaults to false. The `onUpdate(state)` contract is: | Field | Meaning | | ---------------------------- | ------------------------------------------------------------------------------------------------- | | `events` | Deduplicated normalized event objects within the selected display dates | | `covered`, `total` | Selected dates with a loaded window versus requested dates; coverage alone does not mean verified | | `stage` | `local`, `public`, `indexing`, `ready`, or `done`; phases can interleave before `ready` | | `cacheUsed` | A local or public snapshot contributed to this load | | `partial`, `failed` | At least one raw window scan was truncated or failed | | `verified` | Every requested window completed a raw scan in this load | | `published`, `publishFailed` | Accepted manifest count and publication-failure count for this load | `ready` means reading has finished, including any failed/partial reads; it does not mean all results are verified. `done` follows publication attempts. Both may show useful cached results with warnings. The returned promise resolves after those attempts, so awaiting it before rendering defeats progressive loading. Normalized events have `id`, `eventId`, `sessionId`, `event`, `page`, `target`, `timestamp`, and derived UTC `date`. Count page views using only `page_view`. Count sessions with one set of session IDs across all displayed events, not a sum of per-week session totals. A session crossing a week boundary counts once within the selected range. The persisted value under `jw-analytics-cache-v1` has this shape: ```typescript type LocalCache = { version: '1'; windows: Array<{ start: string; checkedAt: number; events: Array<[string, string, string, string, string, string, number]>; publishedHash: string; // Empty if no locally remembered accepted upload publishedAt: number; // Zero if absent }>; ``` Persist only complete snapshots. In-memory entries also track `complete`, `source`, and optional `previousComplete`. These fields are not written in the local JSON. If a raw scan is truncated, merge its events with available cached events for display but persist the last complete snapshot, not that mixed set. If a public cache arrives after a partial scan, retain the partial scan's new events in the UI while saving the complete public snapshot as the fallback. Within a load, a completed raw scan always wins, even if a late cache advertises a newer timestamp. Abort its pending cache read and check the verified flag again before accepting any response already in flight. On range changes, abort all old requests and timers and prevent old callbacks from modifying the new view. Failed requests do not delete prior data or justify publishing an empty revision. These rules are required for correctness, not optional speed tweaks. ## Bounds, failures, and trust - Raw scans paginate 100 records at a time, up to 50 pages per week. Completed weeks are queried separately so each can render independently. Truncated scans are labeled partial and never published. Prior complete local snapshots survive partial scans. - Discovery reads at most 500 manifests and tries up to three candidates per window. Corrupt or unavailable snapshots fall back to indexed events. Failed or truncated pre-upload discovery never counts as proof that a cache is absent. - Payloads stay under 90 KiB and signed ANS-104 items under 100 KiB. Larger windows use multiple parts. Readers cap streamed bytes, part counts, and total records at 5,000 per window before accepting a snapshot. - Initial discovery has a five-second deadline. Each week's initial cache reads share an eight-second budget across candidates. Neither deadline delays raw history beyond the 750 ms head start. Individual requests have a twenty-second timeout. Once all completed weeks are verified, redundant discovery is canceled. - Local storage retains up to 120 days and drops oldest windows to fit a 3 MiB serialized budget. Denied storage, corrupt entries, and quota failures fall back to the network. Browser caches are separate for each origin. - Failed scans retain cached results where possible and show a warning, never a false zero. A successful empty scan means no currently indexed matching events. - Gateway indexing is asynchronous. Scans reflect the records exposed at that moment. This is not a realtime monitor or a transactionally consistent snapshot. - Anyone can submit matching tags. Hashes verify content consistency, not publisher honesty. Cached counts are provisional until raw checks finish, and raw events are also unauthenticated. Do not use this system for billing or audited counts. Queries use `https://arweave.net/graphql`. Uploads use only `https://up.arweave.net/tx`, with a temporary non-extractable browser RSA-PSS key. The deployment wallet is never used. Receipts must match the signed item's calculated ID and must not report a nonzero charge. An uploader failure stops publishing for that load. No payment credentials, paid retries, or alternative paid endpoints are used. Service allowances can change or run out; publication is best effort. External references for maintainers are the [Arweave GraphQL guide](https://gql-guide.arweave.net/) and [Turbo upload documentation](https://docs.ar.io/build/upload/turbo-credits). The limits above describe this implementation's safeguards, not a guarantee of future service pricing or availability. Verify service policy before changing the upload path or limits. ## Tracking and privacy The tracker records home/career views, project opens, resume link clicks, supported contact interactions, and GitHub link clicks. See the event table for the exact hooks and the current Calendly limitation. Historical Gmail clicks remain readable. Clicks do not confirm downloads or sent messages. Random session IDs expire after 30 minutes of inactivity or when a tab closes. Separate tabs can count separately; sessions are not unique people. Events contain only fixed event/page/target values, timestamps, random event IDs, and random session IDs. Caches repeat those fields plus public transaction IDs. No names, emails, IPs, referrers, full URLs, queries, or user-agent strings are included. Network providers still receive connection information. The dashboard never records a visit. Cache publishing requires HTTPS, a non-local origin, Web Crypto, available preference storage, no opt-out, and no Do Not Track or Global Privacy Control signal. Preferences are rechecked before every upload. Local previews can read but never publish. Local caching remains an optional dashboard optimization even when publishing is disabled. Clearing `jw-analytics-cache-v1` removes local copies, not Arweave records. Opt-out uses the separate `jw-analytics-excluded` key. The tracker keeps its 8 KiB event payload cap, 30-event document limit, and one-second repeated-click guard. Event delivery remains best effort with no persistent queue. ## Chart tooltips Every bar reveals its date and exact page-view count on hover, keyboard focus, or tap, including zero-count days. Escape dismisses the tooltip. Buttons have accessible labels, and the daily totals table remains available. Status-only updates do not recreate bars or steal focus. ## Implementation and checks | Source | Responsibility | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | [shared.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/shared.js) | Namespace, event schema, endpoints, UTC ranges, privacy helpers | | [tracker.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/tracker.js) | Page/click hooks, tab session lifecycle, event delivery | | [data-item.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/data-item.js) | Temporary key generation and ANS-104 encoding | | [cache-format.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/cache-format.js) | Weekly windows, canonical tuples, hashes, local storage | | [cache-network.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/cache-network.js) | Queries, public validation, bounded signed uploads | | [cache.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/cache.js) | Progressive loading, concurrent refresh, publication decisions | | [dashboard.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/dashboard.js) and [dashboard.css](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/src/analytics/dashboard.css) | Rendering, privacy controls, statuses, chart tooltips | | [analytics/index.html](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/analytics/index.html) | Dashboard DOM contract and module entry point | | [vite.config.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/vite.config.js) | Static build entries and route-relative assets | The protocol and loader tests are in [analytics-cache.test.mjs](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/scripts/analytics-cache.test.mjs) and [analytics.test.mjs](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/scripts/analytics.test.mjs). Browser cases are in [analytics.spec.js](https://github.com/Jonny-Ringo/Jon_williams/blob/418a57ec1b4dea260495017bc95fbaae90566f9a/tests/analytics.spec.js). Use the source and tests to resolve implementation details; if behavior changes, update this guide in the same change. ## Acceptance criteria A port is ready when these observable behaviors pass. Use deterministic fixtures and mocked uploads for automation; do not generate permanent test records in the production namespace. | Scenario | Required result | | -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | Valid local history, unavailable network | Cached counts render immediately, followed by a refresh warning; no data is erased | | No local cache, usable public snapshot | The public snapshot renders before a deliberately delayed raw-history response | | Stalled public discovery or payload | Historical raw queries start after at most the 750 ms head start; current-week latency does not block them | | Multiple historical windows | Recent caches are requested first; missing raw windows are prioritized; concurrency remains bounded | | Cache arrives after a complete raw scan | It cannot restore old or deleted indexed events or overwrite corrected counts | | Cache arrives after a partial scan | Display retains both cached history and newly found events; local persistence keeps only a complete snapshot | | Invalid tags, dates, hashes, counts, sizes, or redirects | Reject the snapshot and use available raw/local data; never trust malformed content | | Pagination limit, bad cursor, or query error | Mark incomplete/failed results honestly; never publish that scan as a complete snapshot | | Closed week with new or corrected events | Publish parts then manifest after verification; current week never publishes | | Unchanged snapshot or recent identical accepted upload | Skip publication; multiple public copies do not double-count events | | Date-range change or cancellation | Old tasks cannot render or upload after cancellation; abort pending reads and clear the head-start timer | | Privacy exclusion or local preview | No signing-key generation or uploads; read-only dashboard use remains available | | Rejected upload, mismatched receipt, or nonzero charge | Stop publication for the load, retain usable results, and make no paid fallback request | | Cross-week session and duplicate event fixtures | Deduplicate event IDs globally and count each session once in the selected range | | Narrow viewport, keyboard, and no JavaScript | No horizontal overflow; exact bar counts accessible by focus/tap/hover; explanatory fallback with JavaScript disabled | | Manifest-path deployment | Home, career, analytics, assets, and downloads resolve beneath the deployment ID | For signer compatibility, create a signed item with the browser encoder and verify it with an independent ANS-104 implementation as the existing test does. For a read-only port, keep the default publication gate closed; implement publisher acceptance cases only if that port will publish. ## Diagnosing slow or missing data Separate cache **discovery**, cache **payload downloads**, raw **indexing**, and publication. One can fail or stall while the others work. 1. Use a fresh browser profile and open `/analytics/` with DevTools Network open. Identify GraphQL calls by their tag filters: `App-Name` ending in `-Cache` discovers snapshots; the event namespace plus `Timestamp` indexes raw events. 2. Inspect the manifest query's HTTP/GraphQL response and `hasNextPage`. Verify `Source-App`, versions, requested Mondays, and corresponding Sundays. An empty successful response is different from a failed query. 3. Follow a returned transaction ID to `https://arweave.net/`. Inspect its manifest, referenced parts, content sizes, and redirected gateway host. A visible manifest tag does not guarantee all payloads are retrievable yet. 4. Record when the first counts render and when the selected range finishes. Raw-history requests should start within the head-start budget even when cache requests remain pending. Observe intentional aborts when verified raw results make those cache requests unnecessary. 5. Reload in the same profile. Inspect `jw-analytics-cache-v1` for complete windows and confirm immediate local rendering. A different origin, including GitHub versus Arweave, has a different browser-local cache. 6. If no shared snapshots appear, check whether an eligible live dashboard has completed a closed-week scan, whether privacy disables publishing, and whether upload receipts were accepted. Allow for gateway indexing delay. Opening an ordinary homepage does not run the dashboard's cache publisher. | Symptom | Check | | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | Today's data appears before old bars | Expected progressive rendering; check coverage/status before treating older zeros as verified | | GraphQL is fast but the dashboard is slow | Time the manifest/part GETs, redirects, and raw-query pagination separately | | Cache is public but not used | Validate namespace, week, versions, hashes, timestamps, byte caps, and whether fresher raw/local results already won | | No cache for this week | Expected until Sunday has ended in UTC | | Repeated identical uploads | Check stored receipt hash, 24-hour suppression, indexing delay, and whether different origins/tabs are involved | | Reads work but publication does not | Check HTTPS, privacy preferences/signals, Web Crypto, upload response, and payload limits | | Contact totals omit scheduled calls | Calendly is not currently a tracked target; this is an instrumentation gap, not a cache failure | The live investigation that motivated the 750 ms head start found fast GraphQL responses alongside slow transaction-subdomain downloads. Therefore, increasing cache timeouts alone does not solve the loading problem. The essential rule is to allow raw indexing to progress independently and reconcile whichever valid data arrives first. ## Example task for an implementing agent ```text Implement the analytics design in analytics.MD for this host project. Choose a unique application namespace unless this deployment intentionally shares an existing site's history. Reuse the five data modules and their tests or reproduce the documented wire formats and invariants. Integrate tracking only for explicitly supported events and adapt the UI to the host framework. Keep read-only loading enabled by default until the production/privacy gate is connected. Preserve progressive loading, bounded concurrency, cancellation, validation, and free-only publication. Complete the acceptance criteria and report the files changed, configuration choices, test results, and remaining limitations. If deployment is included in the task, use the host project's deployment workflow and verify the published build. ```