Files
broccolini-bot/docs/analytics/Part 2 Batch Analytics Report.md
2026-02-17 21:49:58 -06:00

10 KiB
Raw Blame History

Part 2 — Deep batch analytics report (Mode C, secondorder)

Source: Discord Ticket Transcripts/Drive2/ (722 transcripts).
Input: Parser-derived data from transcript HTML + decoded base64 payloads (timestamps, claim owner, game heuristic, attachments, staff involvement).
Guides: Part 1 Analysis (schemas), Part 2 Analysis (batch dimensions), Part 1 Batch Analytics Report (structure).
Tool: scripts/batch_transcript_analytics.py (extended with timestamps and claim_owner).

This report focuses on patterns, bottlenecks, and recommendations rather than raw counts. Dimensions that require Mode A perticket JSON (issue_types, tags, resolution.status, intake_gaps, wiki_solved_issue, frequency/impact, discord_handle) are called out; tables use parser-derived data where available and note where full analysis needs extraction.


Period (UTC month) Tickets % of total
2025-09 7 1.0
2025-10 288 39.9
2025-11 427 59.1

Game mix over time (parser heuristic; full trend needs Mode A):
From Part 1 report, top games in the batch are Project Zomboid (35.6%), Minecraft (24.8%), Satisfactory (10.9%), Palworld (6.4%). No permonth game breakdown was computed; recommend running Mode A and grouping by first_message_ts (or time.created) and issue.game_detected to detect spikes (e.g. Palworld postlaunch, MC modpack updates).

Narrative: Volume rises sharply from September to November (7 → 288 → 427). The jump suggests either growth in support load or a change in transcript export coverage. To interpret seasonality, spikes by game, or resolution.status over time, add Mode A extraction and bucket by week/month; then cross-tab with issue_types, game_detected, and tags to spot clusters (e.g. connectivity spikes for a specific game or platform).


2. Where tickets get stuck (bottlenecks)

Bottleneck Count % of 722 Notes
No staff ID in message payload 152 21.1 Ticket Toolonly or unclaimed; no Broccolini author in payload.
No attachments saved 325 45.0 Diagnostic quality unknown; mentions_logs/screenshots need Mode A.
Unclaimed (no claim_owner in channel name) 187 25.9 Channel not in staff-claimed-* format.
Long threads (60+ messages) 55 7.6 Likely complex or multi-step; good candidates for playbook review.
Escalation mentioned in text 1 0.1 Rare in this set.

Narrative: About one in five tickets have no staff author in the payload (either unclaimed or structural); a quarter show no claim in the channel name. Nearly half have zero attachments saved—correlating with resolution.status and intake_gaps.attachments (Mode A) would show whether missing diagnostics lengthen resolution or increase unresolved/escalated rates. Long threads (60+ messages) are a small but important slice; recommend tagging these and reviewing for missing reproduction steps, repeated “need more info,” or unclear priority so forms and macros can be improved.


3. Wiki and selfservice leverage

Dimension Parser-derived Mode A required
Wiki articles posted staff_and_wiki.wiki_articles_posted
wiki_solved_issue (true/false/unclear) Per ticket, then % by game_detected, issue_types
“Do it for me” vs walkthrough user_wanted_broccolini_to_do_it, user_wanted_broccolini_but_walkthrough per member

Narrative: Wiki usage and outcomes cannot be computed from the current parser. With Mode A JSON, build: (1) top wiki slugs by usage and by game_detected/issue_types; (2) wiki_solved_issue rate per article and per game/issue slice; (3) permember counts for “do it for me” vs “walkthrough.” Use that to find topics where users often want staff to “do it” and either add or improve wiki steps and macros so more tickets can be resolved via selfservice. Recommend a short table per major wiki category (e.g. connectivity, backups, mods) with usage, success rate, and suggested new or updated pages.


4. Stafflevel patterns and opportunities

Member (claim_owner from channel) Tickets claimed % of claimed Notes
indifferentchicken 407 76.0 Majority of claimed tickets in this batch.
indifferentketchup 128 24.0 Second by volume.
indifferentbroccoli, indifferentcat, indifferentstone 0 (in channel name) No claims in this export with their channel prefix.

Staff involved in payload (any message): 545 tickets have exactly one staff ID in payload, 25 have two, 152 have zero (see §2). Claim count above is from channel name only; staff_involved from payload can include secondary participants.

Narrative: indifferentchicken carries most of the claimed load (76%); indifferentketchup handles the rest. The other three Broccolini members do not appear as claim_owner in this channel-name set—they may be in other panels, or claims may use different naming. To get full workload, resolution rate, escalation rate, first-response time, and wiki success per member, use Mode A plus claim/unclaim message parsing. Recommendations: (1) Cross-check with Discord/Ticket Tool to ensure all support members claim patterns are visible; (2) use Mode A to compute tickets resolved per member and escalation rate per member; (3) if one member remains disproportionately loaded, consider routing or documentation so common game/issue combinations can be handled by others or by wiki/macros.


5. Repeat customers and recurring problems

Dimension Parser-derived Mode A required
Tickets per requester account.discord_handle or user ID; group by requester, count tickets.
Repeat customers (2+ tickets) Same; filter count ≥ 2.
Cluster by game, issue_types, mods, errors Per-ticket issue_types, reproduction.error_messages, environment.mods, game_detected.

Narrative: Requester identity is not in the parser output; repeat customers and recurring problems need Mode A (account.discord_handle, issue, reproduction, environment). With Mode A: (1) list requesters with 2+ tickets and their ticket IDs; (2) for each, cluster by game_detected, issue_types, mods, and error_messages to see if the same problem recurs or different games/issues; (3) suggest proactive actions—e.g. new guides for frequent error clusters, default configs or form questions for common mod/platform combos, or targeted announcements for repeat customers with the same issue type.


6. Design and process changes (recommendations)

Area Finding (from this report) Recommended change
Forms / bot questions 45% of tickets have 0 attachments saved; intake_gaps unknown without Mode A. After Mode A: if intake_gaps.reproduction or intake_gaps.attachments are often missing for high-impact or long-thread tickets, add required reproduction steps and “attach logs/screenshot” for selected issue types or tags.
Wiki coverage Wiki usage/success by game and issue not computable from parser. After Mode A: identify top issue_types and game_detected with low wiki_solved_issue or high “do it for me”; add or update wiki pages and link them in macros.
Tagging / routing issue_types and tags require Mode A. After Mode A: define routing rules for high-impact or high-volume game+issue combinations (e.g. Palworld connectivity → specific wiki + optional escalation path).
Staff playbooks / macros Long threads (60+ messages) and 152 tickets with no staff in payload suggest some tickets stall or lack clear handoff. (1) Add a macro or playbook for “request logs/screenshots” when reproduction is missing; (2) encourage claiming so every ticket has an owner; (3) document common flows for top game+issue pairs (e.g. PZ config, MC mods) so response time and consistency improve.
Lifecycle visibility No time to first response or time to resolution without Mode A timestamps. Run Mode A; extract time.created, time.claimed, time.closed (or first/last message timestamps) to compute response and resolution times and re-opens, then tune SLAs and staffing.

Narrative: This batch surfaces claim imbalance (one member holding most claims), missing diagnostics (many tickets with no attachments saved), and unclaimed or no-staff-in-payload tickets. Full intake_gaps, resolution.status, and wiki outcomes depend on Mode A. Prioritise: (1) Mode A on a representative sample (or full set) to fill Tables 3, 5, and Part 1 report gaps; (2) form and bot changes for reproduction and attachments where gaps correlate with poor outcomes; (3) wiki and macro improvements where “do it for me” is high; (4) routing and playbooks for high-volume game+issue combinations and long-thread tickets.


7. How to reproduce and extend

  1. Run the extended batch parser (timestamps + claim_owner):

    python3 scripts/batch_transcript_analytics.py "Discord Ticket Transcripts/Drive2"
    

    Use Discord Ticket Transcripts/Drive/ for the larger set.

  2. Add Mode A extraction for full Part 2 analysis: run Mode A on each transcript (or sample), collect JSON, then:

    • Bucket by time.created or first_message_ts for trends (issue_types, game_detected, resolution.status by week/month).
    • Correlate intake_gaps with resolution time, unresolved/escalated rate, and back_and_forth_turns for bottlenecks.
    • Aggregate wiki_solved_issue, wiki_articles_posted, and “do it” vs walkthrough for wiki & selfservice.
    • Use claim/unclaim + resolution + wiki for staff profiles (workload, response time, wiki success, escalations).
    • Group by account.discord_handle for repeat customers and cluster by game/issue/mod/error.
  3. Schema and dimensions: Part 1 Analysis and docs/TICKET-ANALYTICS-SCHEMA-PROMPTING.md are the source of truth for fields; Part 2 Analysis defines the batch dimensions (§§212).