diff --git a/.env.example b/.env.example index 3da7409..8800802 100644 --- a/.env.example +++ b/.env.example @@ -65,7 +65,7 @@ DISCORD_ONLY_PORT=5000 # Port for healthcheck / Discord-only server # BOSSCORD_CORS_ORIGIN=* # Optional; default * (set to bOSScord origin in production) # --- Database --- -MONGODB_URI= # MongoDB connection string (e.g. mongodb+srv://user:pass@cluster/dbname) +MONGODB_URI= # MongoDB connection string (e.g. mongodb://broccoli_bot:CHANGE_ME@localhost:27017/broccoli_db?authSource=broccoli_db) # MONGODB_DATABASE= # Optional; DB name usually in MONGODB_URI path (not read by app currently) # --- Branding & copy --- diff --git a/.env.test.example b/.env.test.example index f4ac341..85386cf 100644 --- a/.env.test.example +++ b/.env.test.example @@ -64,7 +64,7 @@ DISCORD_ONLY_PORT=5000 # BOSSCORD_CORS_ORIGIN=* # --- Database (test cluster or local) --- -MONGODB_URI= +MONGODB_URI= # e.g. mongodb://broccoli_bot:CHANGE_ME@localhost:27017/broccoli_db_test?authSource=broccoli_db_test # MONGODB_DATABASE= # --- Branding & copy --- diff --git a/CLAUDE.md b/CLAUDE.md index 97a8f89..946d614 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,7 +21,7 @@ When the user asks for direct fixes, make them — but still avoid unsolicited r ## Project - **broccolini-bot** — Discord ticketing + support bot for Indifferent Broccoli (game hosting). - **Repo:** `/opt/broccolini-bot/` · **Gitea:** `ssh://git@100.114.205.53:2222/indifferentketchup/broccolini-bot.git` -- **DB:** MongoDB Atlas, database `broccoli_db`. +- **DB:** Self-hosted MongoDB on same host as bot, database `broccoli_db`. Dedicated user per DB. - **Host port 8892 → container port 5000** (`CONFIG.PORT`, env `DISCORD_ONLY_PORT`). - **Deploy:** `cd /opt/broccolini-bot && git pull && docker compose up --build -d` · tail: `docker logs broccolini-bot --tail 50 -f` @@ -36,7 +36,7 @@ When the user asks for direct fixes, make them — but still avoid unsolicited r Many files under `scripts/` are one-shot maintenance utilities (backups, user lookups, transcript mapping). They are not wired into CI or into the bot's runtime. ## Stack -Node.js **CommonJS** · Discord.js 14 · Express 5 · Mongoose 6 (MongoDB Atlas) · googleapis · express-rate-limit · p-queue · dotenv/dotenv-expand. +Node.js **CommonJS** · Discord.js 14 · Express 5 · Mongoose 6 · googleapis · express-rate-limit · p-queue · dotenv/dotenv-expand. ## Hard Rules @@ -76,7 +76,7 @@ Every `interactionCreate` branch runs through `runHandler(name, interaction, fn) ### Tickets (`services/tickets.js`, `models.js`) - `Ticket` schema has indexes on `{gmailThreadId}` (unique), `{status, lastActivity}`, `{senderEmail, status}`, `{discordThreadId}`. - **Discord-originated tickets** use `gmailThreadId` with prefix `discord-` / `discord-msg-` — skip the Gmail reply path entirely. -- Renames route through `utils/renamer.js` (RENAMER_BOT secondary token). On 401/403/429 from the secondary, `services/channelQueue.js` falls back to the primary bot via `channel.setName`. `canRename()` is retained as an always-ok shim for back-compat. `Ticket.renameCount` / `Ticket.renameWindowStart` remain in the schema but are now unread/unwritten orphan fields. +- Renames route through `utils/renamer.js` (RENAMER_BOT secondary token). On 401/403/429 from the secondary, `services/channelQueue.js` falls back to the primary bot via `channel.setName`. `canRename()` in `services/tickets.js` is retained as an always-ok shim for back-compat. `Ticket.renameCount` / `Ticket.renameWindowStart` remain in the schema but are now unread/unwritten orphan fields. - `getOrCreateTicketCategory()` handles Discord's 50-channels-per-category ceiling by creating `" (Overflow N)"` categories; `cleanupEmptyOverflowCategory()` removes empties. The primary category is never deleted. - Scheduled jobs in `ready`: `checkAutoClose`, `checkAutoUnclaim`, `reconcileDeletedTicketChannels`, plus `services/staffNotifications.js#notifyAllStaffUnclaimed` and the pattern/surge/chat checkers. diff --git a/README.md b/README.md index 1766ed7..1ab3d6e 100644 --- a/README.md +++ b/README.md @@ -120,7 +120,7 @@ See [Staff notification channels](#staff-notification-channels--reply-alerts) an |-------------|--------| | **Node.js** | **18+**; Docker image uses **20** (`Dockerfile`). | | **npm** | `npm install` locally; `npm ci --omit=dev` in Docker. | -| **MongoDB** | Atlas or self-hosted; `MONGODB_URI` required at startup. | +| **MongoDB** | Self-hosted; `MONGODB_URI` required at startup. | | **Discord application** | Bot token, application ID; intents: **Message Content**, **Server Members**; also Guilds + Guild Messages. | | **Google Cloud** | Gmail API enabled; OAuth2 client ID/secret + refresh token for the support mailbox. | @@ -498,7 +498,7 @@ This repo includes [`.gitlab-ci.yml`](.gitlab-ci.yml) with GitLab **SAST** and * |---------|--------| | **Commands missing** | Correct `DISCORD_APPLICATION_ID` + `DISCORD_GUILD_ID`; **restart** bot; Discord can take time to sync. | | **Gmail not ingesting** | `REFRESH_TOKEN`, API enablement, inbox auth; regenerate token if revoked. | -| **MongoDB errors** | `MONGODB_URI`, Atlas IP allowlist, `npm run test-mongodb`. | +| **MongoDB errors** | `MONGODB_URI`, `npm run test-mongodb`. | | **Channels not creating** | Bot **Manage Channels** in ticket categories; category not full (50) unless overflow set. | | **Modal / button no response** | Intents + permissions; bot online; check `DEBUGGING_CHANNEL_ID` / console. | | **Renames “too quickly”** | Discord rename cooldown; wait for channel queue / timestamp in bot message. | diff --git a/broccolini_bot_context.md b/broccolini_bot_context.md index 21b2a71..2fcd9c3 100644 --- a/broccolini_bot_context.md +++ b/broccolini_bot_context.md @@ -8,7 +8,7 @@ Node.js (CommonJS) Discord ticketing bot for Indifferent Broccoli. Single proces - A discord.js v14 client (ticket lifecycle, slash/button/modal handlers, context menus) - A Gmail bridge (~30s polling → Discord channels; staff replies → Gmail) -- A Mongoose/MongoDB Atlas layer (`broccoli_db`) for tickets + settings +- A Mongoose/self-hosted MongoDB layer (`broccoli_db`) for tickets + settings - Two Express servers: healthcheck + bOSScord API (`PORT`, default 5000 → host 8892), and an internal settings API (`INTERNAL_API_PORT`) - Background jobs: auto-close, unclaimed reminders, auto-unclaim, pattern detection, surge detection, chat monitoring, orphan reconciliation @@ -335,7 +335,7 @@ ticketSchema.index({ ticketNumber: 1 }); `discordThreadId` should be `unique, sparse` because Discord-only tickets set it immediately, email tickets may briefly lack it during creation, and no two tickets should share a channel. Confirm the sparse-unique behavior doesn't conflict with existing data before enabling (see Verify). **Verify:** - Before deploy, run `db.tickets.aggregate([{$group: {_id: "$discordThreadId", c: {$sum: 1}}}, {$match: {c: {$gt: 1}}}])` against `broccoli_db` to confirm no duplicate `discordThreadId` values exist. If any do, investigate (they indicate prior orphaning bugs) before adding the unique index. -- After redeploy, run `db.tickets.getIndexes()` in Atlas and confirm all five new indexes exist. +- After redeploy, run `db.tickets.getIndexes()` via mongosh and confirm all five new indexes exist. - Spot-check with `db.tickets.find({discordThreadId: ""}).explain("executionStats")` — should show `IXSCAN`, not `COLLSCAN`. --- diff --git a/docker-compose.yml b/docker-compose.yml index dafbf7f..efe938b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,4 +1,21 @@ services: + mongo: + image: mongo:7 + container_name: broccoli-mongo + restart: unless-stopped + environment: + MONGO_INITDB_ROOT_USERNAME: ${MONGO_ROOT_USERNAME} + MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD} + volumes: + - broccoli-mongo-data:/data/db + networks: + - broccoli-net + healthcheck: + test: ["CMD", "mongosh", "--quiet", "--eval", "db.runCommand({ping:1}).ok"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 20s broccolini: build: . image: broccolini-bot @@ -11,6 +28,9 @@ services: - ./.env:/app/.env:rw ports: - "100.114.205.53:8892:5000" + depends_on: + mongo: + condition: service_healthy healthcheck: test: ["CMD", "wget", "-qO-", "http://localhost:5000/"] interval: 30s @@ -23,3 +43,6 @@ networks: broccoli-net: name: broccoli-net external: true +volumes: + broccoli-mongo-data: + name: broccoli-mongo-data diff --git a/docs/setup/ENV_AND_SECURITY.md b/docs/setup/ENV_AND_SECURITY.md index 489b2f7..8935890 100644 --- a/docs/setup/ENV_AND_SECURITY.md +++ b/docs/setup/ENV_AND_SECURITY.md @@ -34,7 +34,7 @@ - **Secrets:** All secrets live in `.env` (or `.env.test` for test). Never commit them. `.gitignore` excludes `.env` and `.env.*` except `.env.example` and `.env.test.example`. - **Code:** No `eval()` or `new Function()` of user input. No hardcoded tokens, passwords, or API keys in source. - **Config:** Credentials are read from `process.env` via `config.js`; config is loaded once at startup from the file specified by `ENV_FILE` or default `.env`. -- **MongoDB:** Use a dedicated user and database; restrict network access (Atlas IP allowlist or VPC). For test, use a separate DB or cluster. +- **MongoDB:** Use a dedicated user and database; bind Mongo to loopback or docker network only; firewall 27017 from public interfaces. For test, use a separate DB or cluster. - **Discord / Google:** Use tokens with minimal required scopes; rotate if compromised. - **HTML in emails:** `LOGO_URL`, `EMAIL_SIGNATURE`, and closure messages are escaped in outbound HTML to prevent injection. - **Healthcheck:** Optional `HEALTHCHECK_HOST=127.0.0.1` in `.env` binds the healthcheck server to localhost only; omit to listen on all interfaces. diff --git a/docs/setup/MONGODB_SETUP.md b/docs/setup/MONGODB_SETUP.md index 4e1083b..538d510 100644 --- a/docs/setup/MONGODB_SETUP.md +++ b/docs/setup/MONGODB_SETUP.md @@ -20,10 +20,23 @@ Broccolini Bot uses **MongoDB only** for persistent storage (tickets, transcript Add to your `.env` file: ```env -MONGODB_URI=mongodb://localhost:27018/broccolini_bot +MONGODB_URI=mongodb://broccoli_bot:CHANGE_ME@localhost:27017/broccoli_db?authSource=broccoli_db ``` -**Note:** Uses port `27018` to match your existing setup (as defined in docker-compose.yml). +**Note:** Mongo runs self-hosted on the same host as the bot. A **dedicated user per database** is required — create `broccoli_bot` with `readWrite` on `broccoli_db` only (no admin/root, no cross-DB access). For test, create a separate user with `readWrite` on `broccoli_db_test` only. + +Example mongosh setup: + +```javascript +use broccoli_db +db.createUser({ + user: "broccoli_bot", + pwd: "CHANGE_ME", + roles: [ { role: "readWrite", db: "broccoli_db" } ] +}) +``` + +Bind Mongo to loopback (`bindIp: 127.0.0.1`) or the internal docker network only; firewall `27017` from public interfaces. ### 2. Install Dependencies @@ -141,11 +154,12 @@ process.on('SIGINT', async () => { ### Connection refused - Check MongoDB is running: `docker ps` or `systemctl status mongodb` -- Verify port 27018 is correct in `.env` +- Verify port 27017 is correct in `.env` (or whatever port your mongod is bound to) - Check MongoDB logs for errors ### Authentication failed -- If MongoDB requires auth, update URI: `mongodb://username:password@localhost:27018/broccolini_bot` +- Verify the user exists in the correct DB's `authSource` (URI must include `?authSource=broccoli_db`) +- Confirm the user has `readWrite` on `broccoli_db`: `db.getUser("broccoli_bot")` in mongosh ### Schema validation errors - Check required fields are provided when creating documents diff --git a/gmail-poll.js b/gmail-poll.js index 6adb1e5..3df5d0d 100644 --- a/gmail-poll.js +++ b/gmail-poll.js @@ -33,7 +33,10 @@ let authErrorNotified = false; let pollSuspended = false; let pollCount = 0, totalProcessed = 0, totalSkipped = 0, totalErrors = 0; -function setPollSuspended(val) { pollSuspended = !!val; } +function setPollSuspended(val) { + pollSuspended = !!val; + if (!pollSuspended) authErrorNotified = false; +} function isPollSuspended() { return pollSuspended; } /** @@ -362,18 +365,16 @@ async function poll(client) { } authErrorNotified = false; } catch (e) { - const isAuthError = - (e.message && ( - e.message.includes('invalid_grant') || - e.message.includes('unauthorized') || - e.message.includes('Invalid Credentials') - )) || - e.status === 401 || - e.code === 401; + // Only treat Google-reported permanent-grant failures as reasons to suspend + // the loop. Transient 401/403/429/5xx/network errors fall through to the + // next interval tick naturally. The OAuth error codes come back on the + // response body, not the message string. + const oauthError = e && e.response && e.response.data && e.response.data.error; + const isPermanentAuth = oauthError === 'invalid_grant' || oauthError === 'invalid_client'; - if (isAuthError) { + if (isPermanentAuth) { pollSuspended = true; - const suspendMsg = 'Gmail OAuth token invalid or expired. Polling SUSPENDED — will not retry automatically. Re-authenticate to resume.'; + const suspendMsg = `Gmail OAuth ${oauthError}. Polling SUSPENDED — will not retry automatically. Re-authenticate and POST /internal/gmail/reload to resume.`; console.error('[gmail-poll]', suspendMsg); logError('Gmail OAuth', { message: suspendMsg, stack: e.stack || e.message || String(e) }, null, client).catch(() => {}); try { require('./broccolini-discord').clearGmailPollInterval?.(); } catch (_) {} diff --git a/routes/internalApi.js b/routes/internalApi.js index 045b7b4..c3f0583 100644 --- a/routes/internalApi.js +++ b/routes/internalApi.js @@ -228,6 +228,35 @@ router.post('/notifications/toggle', express.json(), async (req, res) => { res.json({ state: getNotificationState() }); }); +// POST /gmail/reload — hot-swap Gmail OAuth creds after weekly reauth without +// restarting the process. Reads REFRESH_TOKEN from .env via configPersistence, +// probes Google with users.getProfile, and on success clears pollSuspended and +// re-installs the poll interval. On failure returns 400 with Google's error so +// the operator can see why (e.g. still invalid_grant). +router.post('/gmail/reload', express.json(), async (req, res) => { + const { reloadGmailClient } = require('../services/gmail'); + const { setPollSuspended } = require('../gmail-poll'); + try { + const { emailAddress } = await reloadGmailClient(); + setPollSuspended(false); + // Lazy require — same reason as /restart above (module scope cycle). + const parent = require('../broccolini-discord'); + if (parent.setGmailPollInterval) { + parent.setGmailPollInterval(CONFIG.GMAIL_POLL_INTERVAL_MS); + } + await logSystem('Gmail OAuth reloaded', [ + { name: 'Account', value: emailAddress, inline: false } + ]).catch(() => {}); + res.json({ ok: true, email: emailAddress }); + } catch (err) { + const oauthError = err && err.response && err.response.data && err.response.data.error; + res.status(400).json({ + ok: false, + error: oauthError || err.code || err.message || 'reload failed' + }); + } +}); + // Expose the allowlist for the Phase 8 schema smoke test. Attached to the // router function object; doesn't show up as a route. router._allowedKeys = Array.from(ALLOWED_CONFIG_KEYS); diff --git a/services/gmail.js b/services/gmail.js index 9c1f6a5..b0d9857 100644 --- a/services/gmail.js +++ b/services/gmail.js @@ -6,6 +6,7 @@ const { CONFIG } = require('../config'); const { extractRawEmail, escapeHtml } = require('../utils'); const { getStaffSignatureBlocks } = require('./staffSignature'); const { logError } = require('./debugLog'); +const { readEnvFile } = require('./configPersistence'); function sanitizeHeaderValue(v) { return String(v || '').replace(/[\r\n]+/g, ' ').trim(); } const EMAIL_RE = /^[^@\s]+@[^@\s]+$/; @@ -19,6 +20,31 @@ function getGmailClient() { return google.gmail({ version: 'v1', auth }); } +/** + * Re-read REFRESH_TOKEN from .env, update in-memory config, and probe Google. + * Used by the internal /gmail/reload endpoint so the weekly reauth chore does + * not require a full container restart. + * + * Throws if the env file is missing the token, or if the probe call (getProfile) + * fails — the caller surfaces the error so the UI can see why. + * + * @returns {Promise<{emailAddress: string}>} + */ +async function reloadGmailClient() { + const envMap = readEnvFile(); + const newToken = envMap.get('REFRESH_TOKEN'); + if (!newToken) { + const err = new Error('REFRESH_TOKEN not set in .env'); + err.code = 'ENOTOKEN'; + throw err; + } + process.env.REFRESH_TOKEN = newToken; + CONFIG.REFRESH_TOKEN = newToken; + const gmail = getGmailClient(); + const profile = await gmail.users.getProfile({ userId: 'me' }); + return { emailAddress: profile.data.emailAddress }; +} + async function sendTicketClosedEmail(ticket, discordDisplayName) { try { const gmail = getGmailClient(); @@ -318,6 +344,7 @@ async function sendGmailReply( module.exports = { getGmailClient, + reloadGmailClient, sendGmailReply, sendTicketClosedEmail, sendTicketNotificationEmail diff --git a/settings-site/CLAUDE.md b/settings-site/CLAUDE.md index 970ea89..dbb50e5 100644 --- a/settings-site/CLAUDE.md +++ b/settings-site/CLAUDE.md @@ -42,6 +42,9 @@ browser ──► settings server.js (:SETTINGS_PORT, default 12752) | `GET /api/discord/guild` | `GET /internal/discord/guild` | | `POST /api/restart` | `POST /internal/restart` | | `GET /api/restart/status` | `GET /internal/restart/status` | +| `GET /api/notifications/alerts` | `GET /internal/notifications/alerts` | +| `GET /api/notifications/state` | `GET /internal/notifications/state` | +| `POST /api/notifications/toggle` | `POST /internal/notifications/toggle` | Every response-shape change in the bot's `/internal/*` handlers (`routes/internalApi.js`) is a breaking change here. The bot also gates `POST /internal/config` on an `ALLOWED_CONFIG_KEYS` allowlist — **adding a new field to the UI requires adding the key to that Set in the bot first**, otherwise the save returns 400 for that key. @@ -49,11 +52,11 @@ Every response-shape change in the bot's `/internal/*` handlers (`routes/interna `server.js:20-26` sets `cookie.secure: true`. Browsers will refuse to persist the session cookie over plain HTTP, so login silently fails when not behind an HTTPS reverse proxy (`SETTINGS_DOMAIN` is the deployed domain). If you're reproducing a login bug, check this first before debugging auth logic. The `session secret` falls back to `'fallback-secret-change-me'` when `INTERNAL_API_SECRET` is unset — don't rely on the fallback in any environment that matters. ### Client-side routing -`public/index.html` is a single page with all sections rendered; `public/js/app.js` toggles `.hidden` on sections based on `location.pathname`. Routes live in the `ROUTES` map (`app.js:425`). The server has `app.get('*', requireAuth, …)` as a catch-all back to `index.html` (`server.js:97`), so any new client route works without server changes as long as it's added to `ROUTES`. +`public/js/` is split into focused modules (phase 4 refactor): `app.js` (bootstrap), `router.js`, `fields.js`, `notifications.js`, `discord.js`, `login.js`, `util.js` — no bundler, loaded via `