First upload
This commit is contained in:
samkintop
250
docs/CRITICAL_FILES_AND_HOW_IT_WORKS.md
Normal file
250
docs/CRITICAL_FILES_AND_HOW_IT_WORKS.md
Normal file
@@ -0,0 +1,250 @@
|
||||
# Critical Files & How Broccolini Bot Works
|
||||
|
||||
This document identifies the **most critical files** for understanding the repo and gives a **thorough explanation** of how the bot works end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## Most Critical Files (Read These First)
|
||||
|
||||
These are the files that give someone the fastest path to understanding the repo. Read in roughly this order.
|
||||
|
||||
### 1. [**README.md**](../README.md) (repo root)
|
||||
- **Why:** Single source of truth for features, architecture diagram, config, commands, and troubleshooting.
|
||||
- **What you get:** High-level picture, env vars, Discord commands, tag/panel systems, database schema summary, and links to other docs.
|
||||
|
||||
### 2. [**broccolini-discord.js**](../broccolini-discord.js) (entry point)
|
||||
- **Why:** Where the bot starts and where all major pieces are wired together.
|
||||
- **What you get:** Discord client setup, `interactionCreate` routing (buttons → commands → modals → context menus → autocomplete), `messageCreate` → Gmail reply handler, and `ready` logic: MongoDB connect, command registration, Gmail poll start, and background job intervals (auto-close, reminders, auto-unclaim). Also mounts the Express healthcheck and optional bOSScord API.
|
||||
|
||||
### 3. [**config.js**](../config.js)
|
||||
- **Why:** All runtime configuration comes from here (env + defaults).
|
||||
- **What you get:** Single `CONFIG` object: Discord IDs, Gmail/MongoDB settings, automation toggles, message templates, button labels, priority/game lists, and guild-specific options. Test env is supported via `ENV_FILE=.env.test`.
|
||||
|
||||
### 4. [**models.js**](../models.js) (Broccolini Bot section, ~line 793+)
|
||||
- **Why:** Data model defines what the bot persists and how tickets are represented.
|
||||
- **What you get:** Mongoose schemas for **Ticket** (gmailThreadId, discordThreadId, senderEmail, status, claimedBy, priority, escalation, etc.), **TicketCounter**, **Transcript**, **Tag**, **CloseRequest**, **GuildSettings**. Earlier in the file: **User**, **Host**, and other game/hosting models used by `/accountinfo` and external integrations.
|
||||
|
||||
### 5. [**gmail-poll.js**](../gmail-poll.js)
|
||||
- **Why:** This is the “email → Discord” bridge: how support emails become ticket channels.
|
||||
- **What you get:** `poll(client)` runs every 30s: lists unread primary inbox, skips messages from own address, parses From/Subject/body, strips quotes/footers, detects game from `GAME_LIST`, checks ticket limits and rate limits, gets next ticket number, creates Discord channel (or thread) and embed with Claim/Close buttons, saves Ticket + optional Transcript in MongoDB, marks email read. Overflow categories when a category hits 50 channels.
|
||||
|
||||
### 6. [**handlers/messages.js**](../handlers/messages.js)
|
||||
- **Why:** This is the “Discord → email” bridge: staff messages in a ticket become Gmail replies.
|
||||
- **What you get:** `handleDiscordReply(message)`: ignores bots and non-ticket channels; looks up Ticket by `discordThreadId`; skips if ticket is Discord-origin (`gmailThreadId.startsWith('discord-')`); for email tickets, gets Gmail thread, finds last customer message, builds reply with staff name and content, calls `sendGmailReply`, and updates `lastActivity`.
|
||||
|
||||
### 7. [**services/gmail.js**](../services/gmail.js)
|
||||
- **Why:** All Gmail API usage and outbound email logic.
|
||||
- **What you get:** OAuth2 client via `getGmailClient()`; `sendGmailReply()` (threaded reply with HTML, In-Reply-To/References); `sendTicketClosedEmail()` for closure notifications; optional `sendTicketNotificationEmail()` (e.g. priority high). Raw MIME construction and `users.messages.send`.
|
||||
|
||||
### 8. [**services/tickets.js**](../services/tickets.js)
|
||||
- **Why:** Core ticket lifecycle and Discord channel/thread creation.
|
||||
- **What you get:** Ticket numbers (`getNextTicketNumber`), channel naming and Discord rate limit handling (2 renames per 10 min), ticket limits and overflow category selection, rate limit for ticket creation per user, `createEmailTicketAsThread` / `createDiscordTicketAsThread`, auto-close/reminder/auto-unclaim jobs, and helpers like `updateTicketActivity`, `canRename`, `makeTicketName`.
|
||||
|
||||
### 9. [**handlers/buttons.js**](../handlers/buttons.js)
|
||||
- **Why:** Every button and ticket modal goes through here.
|
||||
- **What you get:** “Open Ticket” panel → modal (email, game, description); email routing buttons (thread vs category); Claim / Unclaim / Close (including close confirmation flow); priority and tag selects; escalation/deescalation; and `handleTicketModal` for creating a ticket from the panel. Integrates with `commands.js` for escalation and with `tickets.js`/`gmail.js` for close and notifications.
|
||||
|
||||
### 10. [**handlers/commands.js**](../handlers/commands.js)
|
||||
- **Why:** All slash commands and context menus are implemented here.
|
||||
- **What you get:** Staff check (`requireStaffRole`), then routing for `/claim`, `/unclaim`, `/close`, `/priority`, `/topic`, `/escalate`, `/deescalate`, `/add`, `/remove`, `/transfer`, `/move`, `/tag`, `/response *`, `/panel`, `/email-routing`, `/accountinfo`, `/search`, `/stats`, `/backup`, `/export`, `/help`, and context menu “Create Ticket From Message”. Uses `tickets.js`, `guildSettings`, analytics, and accountinfo/setup handlers.
|
||||
|
||||
### 11. [**commands/register.js**](../commands/register.js)
|
||||
- **Why:** Defines and registers every slash command and context menu with Discord.
|
||||
- **What you get:** Full list of command names, options, permissions, and context types. Run at startup so Discord has the latest command definitions.
|
||||
|
||||
### 12. [**db-connection.js**](../db-connection.js)
|
||||
- **Why:** MongoDB is required; this is the only place that connects and loads models.
|
||||
- **What you get:** `connectMongoDB(uri)`, requires `models.js`, and wires `error` / `disconnected` / `reconnected` for resilience.
|
||||
|
||||
### 13. [**utils.js**](../utils.js)
|
||||
- **Why:** Shared parsing and formatting used by Gmail poll, Gmail service, and commands.
|
||||
- **What you get:** `getCleanBody`, `extractRawEmail`, `stripEmailQuotes`, `stripMobileFooter`, `detectGame` (from subject/body vs `GAME_LIST`), `replaceVariables` for tag/response templates (`{ticket.user}`, `{staff.name}`, etc.), `getPriorityEmoji`, `getFormattedDate`, `escapeHtml`, `htmlToTextWithBlocks`.
|
||||
|
||||
### 14. [**utils/ticketComponents.js**](../utils/ticketComponents.js)
|
||||
- **Why:** Central place for Claim/Unclaim/Close (and related) button rows and embeds.
|
||||
- **What you get:** `getTicketActionRow()` and related builders so ticket channels and panels show consistent buttons and styling.
|
||||
|
||||
### Supporting but still important
|
||||
- [**services/guildSettings.js**](../services/guildSettings.js) – Guild-specific settings (e.g. email routing: thread vs category), cached and persisted in MongoDB.
|
||||
- [**services/debugLog.js**](../services/debugLog.js) – Structured logging and optional Discord debugging channel.
|
||||
- [**handlers/accountinfo.js**](../handlers/accountinfo.js) – `/accountinfo` and lookup logic (website user / Discord link).
|
||||
- [**handlers/analytics.js**](../handlers/analytics.js) – In-memory interaction/error tracking and `/stats`.
|
||||
- [**handlers/setup.js**](../handlers/setup.js) – Guild setup flow (buttons/modals/selects).
|
||||
- [**game-options.json**](../game-options.json) – Game list used for dropdowns/options.
|
||||
- [**QUICKSTART.md**](QUICKSTART.md) – Short path to first reply, panel, tags, priority.
|
||||
- [**ENV_AND_SECURITY.md**](ENV_AND_SECURITY.md) – Test env workflow and security/agent rules.
|
||||
- [**PROJECT_STRUCTURE.md**](PROJECT_STRUCTURE.md) – File/directory layout reference.
|
||||
|
||||
---
|
||||
|
||||
## How the Bot Works (End-to-End)
|
||||
|
||||
### Overview
|
||||
Broccolini Bot is a **Node.js support-ticket bot** that connects **Gmail**, **Discord**, and **MongoDB**. Incoming support emails become Discord ticket channels (or threads); staff reply in Discord and their messages are sent back to the customer via Gmail. Tickets can also be created from Discord via a panel (no email). All ticket state is stored in MongoDB.
|
||||
|
||||
---
|
||||
|
||||
### Startup sequence
|
||||
|
||||
1. **Load config**
|
||||
[config.js](../config.js) loads `.env` (or `ENV_FILE`), runs dotenv-expand, and exports `CONFIG`.
|
||||
|
||||
2. **Validate env**
|
||||
[broccolini-discord.js](../broccolini-discord.js) checks required vars (e.g. `DISCORD_TOKEN`, `TICKET_CATEGORY_ID`, Gmail OAuth). Missing required ones cause exit.
|
||||
|
||||
3. **Create Discord client**
|
||||
Client is created with intents: Guilds, GuildMessages, MessageContent, GuildMembers; Partials.Channel for ticket channels that might not be in cache.
|
||||
|
||||
4. **Register event handlers**
|
||||
- **`interactionCreate`** – Buttons (accountinfo, setup, ticket actions), modals (setup, ticket creation), slash commands, context menus, autocomplete. Order matters: prefix checks (e.g. accountinfo, setup) run before generic button/command handlers.
|
||||
- **`messageCreate`** – `handleDiscordReply`: Discord → Gmail for staff messages in ticket channels.
|
||||
|
||||
5. **`ready`**
|
||||
- Connect MongoDB via [db-connection.js](../db-connection.js) (and load [models.js](../models.js)).
|
||||
- Set debug log client and bOSScord API client.
|
||||
- If `BOSSCORD_API_KEY` is set, mount `/api` routes (e.g. bOSScord).
|
||||
- Call `registerCommands()` so slash commands and context menus are registered for the guild.
|
||||
- Start Gmail poll: `poll(client)` immediately and then `setInterval(..., 30000)`.
|
||||
- If enabled: start hourly auto-close, 30-minute reminders, hourly auto-unclaim.
|
||||
- Express server is already created; it listens on `CONFIG.PORT` and serves `GET /` → `"Active"` for healthchecks.
|
||||
|
||||
6. **No Gmail/MongoDB**
|
||||
If `MONGODB_URI` is missing, the bot exits in `ready`. Gmail credentials are validated at startup but polling can fail later if tokens are bad.
|
||||
|
||||
---
|
||||
|
||||
### Email → Discord (new ticket from email)
|
||||
|
||||
1. **Gmail poll** ([gmail-poll.js](../gmail-poll.js))
|
||||
- Every 30s, `poll(client)` uses Gmail API `users.messages.list` with `is:unread category:primary`.
|
||||
- For each message, fetches full message, parses From/Subject/body. Skips if From is the support address (and marks it read).
|
||||
- Extracts sender email and name; cleans body (strip reply quotes, mobile footers) via [utils.js](../utils.js).
|
||||
- Detects game from subject/body using `GAME_LIST` (`utils.detectGame`).
|
||||
- Checks global and per-category ticket limits and per-user rate limit (`services/tickets.js`).
|
||||
- Gets next ticket number per sender from `TicketCounter` (`getNextTicketNumber`).
|
||||
- Decides where to create the ticket: **thread** vs **category channel** from `getEmailRouting()` (guild setting, can be set via `/email-routing`).
|
||||
- Creates the Discord channel or thread, posts an embed (subject, sender, game, ticket number) and action row (Claim, Close) from `ticketComponents.js`.
|
||||
- Saves a **Ticket** in MongoDB: `gmailThreadId`, `discordThreadId` (channel or thread id), `senderEmail`, `subject`, `ticketNumber`, game, status `open`, etc. Optionally creates **Transcript** placeholder.
|
||||
- Marks the Gmail message read so it is not processed again.
|
||||
|
||||
2. **Overflow**
|
||||
Discord allows 50 channels per category. If the main ticket category is full, the bot uses `EMAIL_TICKET_OVERFLOW_CATEGORY_IDS` (and similar for Discord-origin tickets) to pick another category.
|
||||
|
||||
---
|
||||
|
||||
### Discord → Email (staff reply)
|
||||
|
||||
1. **Message event**
|
||||
When a message is sent in a channel, `handleDiscordReply` in [handlers/messages.js](../handlers/messages.js) runs.
|
||||
|
||||
2. **Filter**
|
||||
Ignores bots and interactions. Finds a Ticket by `discordThreadId === channel.id`. If none, or if `gmailThreadId.startsWith('discord-')`, does nothing (Discord-origin tickets have no Gmail thread).
|
||||
|
||||
3. **Build reply**
|
||||
Uses Gmail API `users.threads.get` to get the thread. Finds the last message from the customer (not from support). Reads To/Reply-To, Subject, Message-ID.
|
||||
|
||||
4. **Send**
|
||||
`sendGmailReply(threadId, content, recipientEmail, subject, discordUser, messageId)` in [services/gmail.js](../services/gmail.js) builds HTML (staff name, content, logo/signature), sets In-Reply-To/References for threading, and calls `users.messages.send` with `threadId` so the reply stays in the same Gmail thread.
|
||||
|
||||
5. **Activity**
|
||||
`updateTicketActivity(ticket.gmailThreadId)` updates the ticket’s `lastActivity` for auto-close and reminder logic.
|
||||
|
||||
---
|
||||
|
||||
### Ticket creation from Discord (panel)
|
||||
|
||||
1. **Panel**
|
||||
Staff runs `/panel` (optionally with channel, type, title, description). The bot sends a message with an “Open Ticket” button (and optional styling).
|
||||
|
||||
2. **Button click**
|
||||
User clicks the button. [handlers/buttons.js](../handlers/buttons.js) shows a modal with: Account Email, Game, Description (and possibly priority).
|
||||
|
||||
3. **Modal submit**
|
||||
`handleTicketModal` runs. Validates and applies rate limit (`checkTicketCreationRateLimit`). Creates a Ticket in MongoDB with `gmailThreadId = 'discord-' + ...` (no real Gmail thread). Gets next ticket number (by email or Discord user). Creates a Discord channel or thread (depending on panel type and guild settings), posts welcome embed and Claim/Close buttons, saves `discordThreadId` and other fields. No Gmail is involved for this ticket; `handleDiscordReply` explicitly skips when `gmailThreadId.startsWith('discord-')`.
|
||||
|
||||
---
|
||||
|
||||
### Claim / Unclaim / Close
|
||||
|
||||
- **Claim** (button or `/claim`)
|
||||
Ticket is updated with `claimedBy` (user id or name). Channel may be renamed (respecting Discord’s 2 renames per 10 min). Claimed message is posted (template from CONFIG).
|
||||
|
||||
- **Unclaim** (button or `/unclaim`)
|
||||
`claimedBy` is cleared; channel rename and message as above.
|
||||
|
||||
- **Close** (button or `/close`)
|
||||
Close button often triggers a confirmation (e.g. “Are you sure?” with Confirm/Cancel). On confirm (or `/force-close`): build transcript of the channel, post it to the transcript channel, send closure email for **email tickets** via `sendTicketClosedEmail`, delete the Discord channel/thread, set ticket status to `closed` and clean up CloseRequest. Discord-origin tickets get no Gmail closure email.
|
||||
|
||||
---
|
||||
|
||||
### Automation (background jobs)
|
||||
|
||||
- **Auto-close**
|
||||
If `AUTO_CLOSE_ENABLED`, `checkAutoClose` runs hourly. Finds open tickets whose `lastActivity` is older than `AUTO_CLOSE_AFTER_HOURS`. For each, same flow as manual close (transcript, closure email if email ticket, delete channel, update ticket).
|
||||
|
||||
- **Reminders**
|
||||
If `REMINDER_ENABLED`, `checkReminders` runs every 30 minutes. Finds open tickets inactive longer than `REMINDER_AFTER_HOURS` and not yet reminded. Sends reminder message to the channel and sets `reminderSent`.
|
||||
|
||||
- **Auto-unclaim**
|
||||
If `AUTO_UNCLAIM_ENABLED`, `checkAutoUnclaim` runs hourly. Clears `claimedBy` on tickets that have been inactive for `AUTO_UNCLAIM_AFTER_HOURS`.
|
||||
|
||||
All of these use the Ticket model’s `lastActivity` (and optional `reminderSent`) and live in [services/tickets.js](../services/tickets.js).
|
||||
|
||||
---
|
||||
|
||||
### Tags and saved responses
|
||||
|
||||
- **Tags**
|
||||
`/tag` sets a ticket category (e.g. Server Down, Billing). Stored on the ticket and can be used in naming or display. Tag options come from `CONFIG.TICKET_TAGS` (from config / env).
|
||||
|
||||
- **Saved responses**
|
||||
Stored in MongoDB (**Tag** collection for response name/content). `/response create|edit|delete|list|send`. When sending, `utils.replaceVariables` substitutes `{ticket.user}`, `{staff.name}`, `{date}`, etc. Autocomplete for response names is provided in `handlers/commands.js`.
|
||||
|
||||
---
|
||||
|
||||
### Priority and escalation
|
||||
|
||||
- **Priority**
|
||||
If `PRIORITY_ENABLED`, tickets have low/normal/medium/high. Stored on Ticket; embeds and channel naming can show priority emoji. Optional: send email when set to high (`sendTicketNotificationEmail`).
|
||||
|
||||
- **Escalation**
|
||||
`/escalate` and `/deescalate` (or buttons) change `escalationTier` (0 → 1 → 2) and move the channel to escalation categories (e.g. `EMAIL_ESCALATED_CATEGORY_ID`, `DISCORD_ESCALATED2_CHANNEL_ID`). Channel rename and “escalated” message are posted. Optional escalation notification email.
|
||||
|
||||
---
|
||||
|
||||
### Account info and other commands
|
||||
|
||||
- **`/accountinfo`**
|
||||
Looks up a user by email or Discord ID (uses **User** and related models from `models.js`). Can post results to a dedicated channel; handler in `handlers/accountinfo.js`.
|
||||
|
||||
- **`/email-routing`**
|
||||
Toggles where new **email** tickets are created: thread under a parent channel or channel in a category. Value is stored per guild in **GuildSettings** and read by [gmail-poll.js](../gmail-poll.js) via `getEmailRouting()`.
|
||||
|
||||
- **`/panel`**
|
||||
Sends a message with an “Open Ticket” button; panel type (thread vs channel) and target channel are options.
|
||||
|
||||
- **`/search`, `/backup`, `/export`**
|
||||
Query or export tickets (by status, limit, etc.) and post results (e.g. to backup channel).
|
||||
|
||||
- **`/stats`**
|
||||
Returns in-memory analytics (interactions, errors) from `handlers/analytics.js`.
|
||||
|
||||
---
|
||||
|
||||
### Data flow summary
|
||||
|
||||
- **Gmail → Discord:** [gmail-poll.js](../gmail-poll.js) (poll) → [services/gmail.js](../services/gmail.js) (read), [services/tickets.js](../services/tickets.js) (create channel + Ticket), [utils.js](../utils.js) (parse/detect game).
|
||||
- **Discord → Gmail:** [handlers/messages.js](../handlers/messages.js) → [services/gmail.js](../services/gmail.js) (`sendGmailReply`), [services/tickets.js](../services/tickets.js) (`updateTicketActivity`).
|
||||
- **Discord-only ticket:** [handlers/buttons.js](../handlers/buttons.js) (modal) → [services/tickets.js](../services/tickets.js) (create channel + Ticket with `gmailThreadId: 'discord-...'`).
|
||||
- **All interactions:** [broccolini-discord.js](../broccolini-discord.js) (routing) → [handlers/buttons.js](../handlers/buttons.js) or [handlers/commands.js](../handlers/commands.js) → [services/tickets.js](../services/tickets.js), [services/guildSettings.js](../services/guildSettings.js), [services/gmail.js](../services/gmail.js), and [models.js](../models.js) (Mongoose).
|
||||
|
||||
---
|
||||
|
||||
### Dependencies (high level)
|
||||
|
||||
- **discord.js** – Client, channels, embeds, buttons, modals, slash commands, context menus.
|
||||
- **googleapis** – Gmail API (OAuth2, list/get messages, send, threads).
|
||||
- **mongoose** – MongoDB connection and Broccolini Bot + game/hosting models.
|
||||
- **express** – Healthcheck server and bOSScord API mount.
|
||||
|
||||
For full setup, config, and troubleshooting, use [README.md](../README.md), [QUICKSTART.md](QUICKSTART.md), and [ENV_AND_SECURITY.md](ENV_AND_SECURITY.md).
|
||||
Reference in New Issue
Block a user