broccolini-bot/docs/CRITICAL_FILES_AND_HOW_IT_WORKS.md

# Critical Files & How Broccolini Bot Works

This document identifies the **most critical files** for understanding the repo and gives a **thorough explanation** of how the bot works end-to-end.

---

## Most Critical Files (Read These First)

These are the files that give someone the fastest path to understanding the repo. Read in roughly this order.

### 1. [**README.md**](../README.md) (repo root)
- **Why:** Single source of truth for features, architecture diagram, config, commands, and troubleshooting.
- **What you get:** High-level picture, env vars, Discord commands, tag/panel systems, database schema summary, and links to other docs.

### 2. [**broccolini-discord.js**](../broccolini-discord.js) (entry point)
- **Why:** Where the bot starts and where all major pieces are wired together.
- **What you get:** Discord client setup, `interactionCreate` routing (buttons → commands → modals → context menus → autocomplete), `messageCreate` → Gmail reply handler, and `ready` logic: MongoDB connect, command registration, Gmail poll start, and background job intervals (auto-close, reminders, auto-unclaim). Also mounts the Express healthcheck and optional bOSScord API.

### 3. [**config.js**](../config.js)
- **Why:** All runtime configuration comes from here (env + defaults).
- **What you get:** Single `CONFIG` object: Discord IDs, Gmail/MongoDB settings, automation toggles, message templates, button labels, priority/game lists, and guild-specific options. Test env is supported via `ENV_FILE=.env.test`.

### 4. [**models.js**](../models.js) (Broccolini Bot section, ~line 793+)
- **Why:** Data model defines what the bot persists and how tickets are represented.
- **What you get:** Mongoose schemas for **Ticket** (gmailThreadId, discordThreadId, senderEmail, status, claimedBy, priority, escalation, etc.), **TicketCounter**, **Transcript**, **Tag**, **CloseRequest**, **GuildSettings**. Earlier in the file: **User**, **Host**, and other game/hosting models used by `/accountinfo` and external integrations.

### 5. [**gmail-poll.js**](../gmail-poll.js)
- **Why:** This is the “email → Discord” bridge: how support emails become ticket channels.
- **What you get:** `poll(client)` runs every 30s: lists unread primary inbox, skips messages from own address, parses From/Subject/body, strips quotes/footers, detects game from `GAME_LIST`, checks ticket limits and rate limits, gets next ticket number, creates Discord channel (or thread) and embed with Claim/Close buttons, saves Ticket + optional Transcript in MongoDB, marks email read. Overflow categories when a category hits 50 channels.

### 6. [**handlers/messages.js**](../handlers/messages.js)
- **Why:** This is the “Discord → email” bridge: staff messages in a ticket become Gmail replies.
- **What you get:** `handleDiscordReply(message)`: ignores bots and non-ticket channels; looks up Ticket by `discordThreadId`; skips if ticket is Discord-origin (`gmailThreadId.startsWith('discord-')`); for email tickets, gets Gmail thread, finds last customer message, builds reply with staff name and content, calls `sendGmailReply`, and updates `lastActivity`.

### 7. [**services/gmail.js**](../services/gmail.js)
- **Why:** All Gmail API usage and outbound email logic.
- **What you get:** OAuth2 client via `getGmailClient()`; `sendGmailReply()` (threaded reply with HTML, In-Reply-To/References); `sendTicketClosedEmail()` for closure notifications; optional `sendTicketNotificationEmail()` (e.g. priority high). Raw MIME construction and `users.messages.send`.

### 8. [**services/tickets.js**](../services/tickets.js)
- **Why:** Core ticket lifecycle and Discord channel/thread creation.
- **What you get:** Ticket numbers (`getNextTicketNumber`), channel naming, ticket limits and overflow category selection, rate limit for ticket creation per user, `createEmailTicketAsThread` / `createDiscordTicketAsThread`, auto-close/reminder/auto-unclaim jobs, and helpers like `updateTicketActivity`, `canRename` (retained as an always-ok shim — see `utils/renamer.js` and `services/channelQueue.js` for actual rename handling and primary-bot fallback), `makeTicketName`.

### 9. [**handlers/buttons.js**](../handlers/buttons.js)
- **Why:** Every button and ticket modal goes through here.
- **What you get:** “Open Ticket” panel → modal (email, game, description); email routing buttons (thread vs category); Claim / Unclaim / Close (including close confirmation flow); priority and tag selects; escalation/deescalation; and `handleTicketModal` for creating a ticket from the panel. Integrates with `commands.js` for escalation and with `tickets.js`/`gmail.js` for close and notifications.

### 10. [**handlers/commands.js**](../handlers/commands.js)
- **Why:** All slash commands and context menus are implemented here.
- **What you get:** Staff check (`requireStaffRole`), then routing for `/claim`, `/unclaim`, `/close`, `/priority`, `/topic`, `/escalate`, `/deescalate`, `/add`, `/remove`, `/transfer`, `/move`, `/tag`, `/response *`, `/panel`, `/email-routing`, `/accountinfo`, `/search`, `/stats`, `/backup`, `/export`, `/help`, and context menu “Create Ticket From Message”. Uses `tickets.js`, `guildSettings`, analytics, and accountinfo/setup handlers.

### 11. [**commands/register.js**](../commands/register.js)
- **Why:** Defines and registers every slash command and context menu with Discord.
- **What you get:** Full list of command names, options, permissions, and context types. Run at startup so Discord has the latest command definitions.

### 12. [**db-connection.js**](../db-connection.js)
- **Why:** MongoDB is required; this is the only place that connects and loads models.
- **What you get:** `connectMongoDB(uri)`, requires `models.js`, and wires `error` / `disconnected` / `reconnected` for resilience.

### 13. [**utils.js**](../utils.js)
- **Why:** Shared parsing and formatting used by Gmail poll, Gmail service, and commands.
- **What you get:** `getCleanBody`, `extractRawEmail`, `stripEmailQuotes`, `stripMobileFooter`, `detectGame` (from subject/body vs `GAME_LIST`), `replaceVariables` for tag/response templates (`{ticket.user}`, `{staff.name}`, etc.), `getPriorityEmoji`, `getFormattedDate`, `escapeHtml`, `htmlToTextWithBlocks`.

### 14. [**utils/ticketComponents.js**](../utils/ticketComponents.js)
- **Why:** Central place for Claim/Unclaim/Close (and related) button rows and embeds.
- **What you get:** `getTicketActionRow()` and related builders so ticket channels and panels show consistent buttons and styling.

### Supporting but still important
- [**services/guildSettings.js**](../services/guildSettings.js) – Guild-specific settings (e.g. email routing: thread vs category), cached and persisted in MongoDB.
- [**services/debugLog.js**](../services/debugLog.js) – Structured logging and optional Discord debugging channel.
- [**handlers/accountinfo.js**](../handlers/accountinfo.js) – `/accountinfo` and lookup logic (website user / Discord link).
- [**handlers/analytics.js**](../handlers/analytics.js) – In-memory interaction/error tracking and `/stats`.
- [**handlers/setup.js**](../handlers/setup.js) – Guild setup flow (buttons/modals/selects).
- [**game-options.json**](../game-options.json) – Game list used for dropdowns/options.
- [**QUICKSTART.md**](QUICKSTART.md) – Short path to first reply, panel, tags, priority.
- [**ENV_AND_SECURITY.md**](ENV_AND_SECURITY.md) – Test env workflow and security/agent rules.
- [**PROJECT_STRUCTURE.md**](PROJECT_STRUCTURE.md) – File/directory layout reference.

---

## How the Bot Works (End-to-End)

### Overview
Broccolini Bot is a **Node.js support-ticket bot** that connects **Gmail**, **Discord**, and **MongoDB**. Incoming support emails become Discord ticket channels (or threads); staff reply in Discord and their messages are sent back to the customer via Gmail. Tickets can also be created from Discord via a panel (no email). All ticket state is stored in MongoDB.

---

### Startup sequence

1. **Load config**  
   [config.js](../config.js) loads `.env` (or `ENV_FILE`), runs dotenv-expand, and exports `CONFIG`.

2. **Validate env**  
   [broccolini-discord.js](../broccolini-discord.js) checks required vars (e.g. `DISCORD_TOKEN`, `TICKET_CATEGORY_ID`, Gmail OAuth). Missing required ones cause exit.

3. **Create Discord client**  
   Client is created with intents: Guilds, GuildMessages, MessageContent, GuildMembers; Partials.Channel for ticket channels that might not be in cache.

4. **Register event handlers**
   - **`interactionCreate`** – Buttons (accountinfo, setup, ticket actions), modals (setup, ticket creation), slash commands, context menus, autocomplete. Order matters: prefix checks (e.g. accountinfo, setup) run before generic button/command handlers.
   - **`messageCreate`** – `handleDiscordReply`: Discord → Gmail for staff messages in ticket channels.

5. **`ready`**
   - Connect MongoDB via [db-connection.js](../db-connection.js) (and load [models.js](../models.js)).
   - Set debug log client and bOSScord API client.
   - If `BOSSCORD_API_KEY` is set, mount `/api` routes (e.g. bOSScord).
   - Call `registerCommands()` so slash commands and context menus are registered for the guild.
   - Start Gmail poll: `poll(client)` immediately and then `setInterval(..., 30000)`.
   - If enabled: start hourly auto-close, 30-minute reminders, hourly auto-unclaim.
   - Express server is already created; it listens on `CONFIG.PORT` and serves `GET /` → `"Active"` for healthchecks.

6. **No Gmail/MongoDB**  
   If `MONGODB_URI` is missing, the bot exits in `ready`. Gmail credentials are validated at startup but polling can fail later if tokens are bad.

---

### Email → Discord (new ticket from email)

1. **Gmail poll** ([gmail-poll.js](../gmail-poll.js))
   - Every 30s, `poll(client)` uses Gmail API `users.messages.list` with `is:unread category:primary`.
   - For each message, fetches full message, parses From/Subject/body. Skips if From is the support address (and marks it read).
   - Extracts sender email and name; cleans body (strip reply quotes, mobile footers) via [utils.js](../utils.js).
   - Detects game from subject/body using `GAME_LIST` (`utils.detectGame`).
   - Checks global and per-category ticket limits and per-user rate limit (`services/tickets.js`).
   - Gets next ticket number per sender from `TicketCounter` (`getNextTicketNumber`).
   - Decides where to create the ticket: **thread** vs **category channel** from `getEmailRouting()` (guild setting, can be set via `/email-routing`).
   - Creates the Discord channel or thread, posts an embed (subject, sender, game, ticket number) and action row (Claim, Close) from `ticketComponents.js`.
   - Saves a **Ticket** in MongoDB: `gmailThreadId`, `discordThreadId` (channel or thread id), `senderEmail`, `subject`, `ticketNumber`, game, status `open`, etc. Optionally creates **Transcript** placeholder.
   - Marks the Gmail message read so it is not processed again.

2. **Overflow**  
   Discord allows 50 channels per category. If the main ticket category is full, the bot uses `EMAIL_TICKET_OVERFLOW_CATEGORY_IDS` (and similar for Discord-origin tickets) to pick another category.

---

### Discord → Email (staff reply)

1. **Message event**  
   When a message is sent in a channel, `handleDiscordReply` in [handlers/messages.js](../handlers/messages.js) runs.

2. **Filter**  
   Ignores bots and interactions. Finds a Ticket by `discordThreadId === channel.id`. If none, or if `gmailThreadId.startsWith('discord-')`, does nothing (Discord-origin tickets have no Gmail thread).

3. **Build reply**  
   Uses Gmail API `users.threads.get` to get the thread. Finds the last message from the customer (not from support). Reads To/Reply-To, Subject, Message-ID.

4. **Send**  
   `sendGmailReply(threadId, content, recipientEmail, subject, discordUser, messageId)` in [services/gmail.js](../services/gmail.js) builds HTML (staff name, content, logo/signature), sets In-Reply-To/References for threading, and calls `users.messages.send` with `threadId` so the reply stays in the same Gmail thread.

5. **Activity**  
   `updateTicketActivity(ticket.gmailThreadId)` updates the ticket’s `lastActivity` for auto-close and reminder logic.

---

### Ticket creation from Discord (panel)

1. **Panel**  
   Staff runs `/panel` (optionally with channel, type, title, description). The bot sends a message with an “Open Ticket” button (and optional styling).

2. **Button click**  
   User clicks the button. [handlers/buttons.js](../handlers/buttons.js) shows a modal with: Account Email, Game, Description (and possibly priority).

3. **Modal submit**  
   `handleTicketModal` runs. Validates and applies rate limit (`checkTicketCreationRateLimit`). Creates a Ticket in MongoDB with `gmailThreadId = 'discord-' + ...` (no real Gmail thread). Gets next ticket number (by email or Discord user). Creates a Discord channel or thread (depending on panel type and guild settings), posts welcome embed and Claim/Close buttons, saves `discordThreadId` and other fields. No Gmail is involved for this ticket; `handleDiscordReply` explicitly skips when `gmailThreadId.startsWith('discord-')`.

---

### Claim / Unclaim / Close

- **Claim** (button or `/claim`)  
  Ticket is updated with `claimedBy` (user id or name). Channel may be renamed via the secondary-bot path (`utils/renamer.js`), falling back to the primary bot on 401/403/429. Claimed message is posted (template from CONFIG).

- **Unclaim** (button or `/unclaim`)  
  `claimedBy` is cleared; channel rename and message as above.

- **Close** (button or `/close`)  
  Close button often triggers a confirmation (e.g. “Are you sure?” with Confirm/Cancel). On confirm (or `/force-close`): build transcript of the channel, post it to the transcript channel, send closure email for **email tickets** via `sendTicketClosedEmail`, delete the Discord channel/thread, set ticket status to `closed` and clean up CloseRequest. Discord-origin tickets get no Gmail closure email.

---

### Automation (background jobs)

- **Auto-close**  
  If `AUTO_CLOSE_ENABLED`, `checkAutoClose` runs hourly. Finds open tickets whose `lastActivity` is older than `AUTO_CLOSE_AFTER_HOURS`. For each, same flow as manual close (transcript, closure email if email ticket, delete channel, update ticket).

- **Reminders**  
  If `REMINDER_ENABLED`, `checkReminders` runs every 30 minutes. Finds open tickets inactive longer than `REMINDER_AFTER_HOURS` and not yet reminded. Sends reminder message to the channel and sets `reminderSent`.

- **Auto-unclaim**  
  If `AUTO_UNCLAIM_ENABLED`, `checkAutoUnclaim` runs hourly. Clears `claimedBy` on tickets that have been inactive for `AUTO_UNCLAIM_AFTER_HOURS`.

All of these use the Ticket model’s `lastActivity` (and optional `reminderSent`) and live in [services/tickets.js](../services/tickets.js).

---

### Tags and saved responses

- **Tags**  
  `/tag` sets a ticket category (e.g. Server Down, Billing). Stored on the ticket and can be used in naming or display. Tag options come from `CONFIG.TICKET_TAGS` (from config / env).

- **Saved responses**  
  Stored in MongoDB (**Tag** collection for response name/content). `/response create|edit|delete|list|send`. When sending, `utils.replaceVariables` substitutes `{ticket.user}`, `{staff.name}`, `{date}`, etc. Autocomplete for response names is provided in `handlers/commands.js`.

---

### Priority and escalation

- **Priority**  
  If `PRIORITY_ENABLED`, tickets have low/normal/medium/high. Stored on Ticket; embeds and channel naming can show priority emoji. Optional: send email when set to high (`sendTicketNotificationEmail`).

- **Escalation**  
  `/escalate` and `/deescalate` (or buttons) change `escalationTier` (0 → 1 → 2) and move the channel to escalation categories (e.g. `EMAIL_ESCALATED_CATEGORY_ID`, `DISCORD_ESCALATED2_CHANNEL_ID`). Channel rename and “escalated” message are posted. Optional escalation notification email.

---

### Account info and other commands

- **`/accountinfo`**  
  Looks up a user by email or Discord ID (uses **User** and related models from `models.js`). Can post results to a dedicated channel; handler in `handlers/accountinfo.js`.

- **`/email-routing`**  
  Toggles where new **email** tickets are created: thread under a parent channel or channel in a category. Value is stored per guild in **GuildSettings** and read by [gmail-poll.js](../gmail-poll.js) via `getEmailRouting()`.

- **`/panel`**  
  Sends a message with an “Open Ticket” button; panel type (thread vs channel) and target channel are options.

- **`/search`, `/backup`, `/export`**  
  Query or export tickets (by status, limit, etc.) and post results (e.g. to backup channel).

- **`/stats`**  
  Returns in-memory analytics (interactions, errors) from `handlers/analytics.js`.

---

### Data flow summary

- **Gmail → Discord:** [gmail-poll.js](../gmail-poll.js) (poll) → [services/gmail.js](../services/gmail.js) (read), [services/tickets.js](../services/tickets.js) (create channel + Ticket), [utils.js](../utils.js) (parse/detect game).
- **Discord → Gmail:** [handlers/messages.js](../handlers/messages.js) → [services/gmail.js](../services/gmail.js) (`sendGmailReply`), [services/tickets.js](../services/tickets.js) (`updateTicketActivity`).
- **Discord-only ticket:** [handlers/buttons.js](../handlers/buttons.js) (modal) → [services/tickets.js](../services/tickets.js) (create channel + Ticket with `gmailThreadId: 'discord-...'`).
- **All interactions:** [broccolini-discord.js](../broccolini-discord.js) (routing) → [handlers/buttons.js](../handlers/buttons.js) or [handlers/commands.js](../handlers/commands.js) → [services/tickets.js](../services/tickets.js), [services/guildSettings.js](../services/guildSettings.js), [services/gmail.js](../services/gmail.js), and [models.js](../models.js) (Mongoose).

---

### Dependencies (high level)

- **discord.js** – Client, channels, embeds, buttons, modals, slash commands, context menus.
- **googleapis** – Gmail API (OAuth2, list/get messages, send, threads).
- **mongoose** – MongoDB connection and Broccolini Bot + game/hosting models.
- **express** – Healthcheck server and bOSScord API mount.

For full setup, config, and troubleshooting, use [README.md](../README.md), [QUICKSTART.md](QUICKSTART.md), and [ENV_AND_SECURITY.md](ENV_AND_SECURITY.md).