feat(newsletter): add delivery pipeline #70

Closed

exe-dev-bot wants to merge 25 commits from exe-dev-bot/market:feat/newsletter-delivery into main

pull from: exe-dev-bot/market:feat/newsletter-delivery

merge into: kwila:main

kwila:main

kwila:exe-dev-bot-feat/newsletter-delivery

kwila:exe-dev-bot-feat/connections-system

kwila:add-invite-validation-system

kwila:set-up-supabase-email-auth

kwila:setup-supabase-storage

kwila:feat/db-migrations

kwila:feat-docker-compose

kwila:mvp-add-docker-compose

kwila:claude/mvp-spec-next-01JjffeFb97yuuUWk8o6J2VV

Conversation 1 Commits 25 Files changed 22 +1070 -22

exe-dev-bot commented 2026-02-06 04:55:04 -05:00

Contributor

Copy link

Summary

Implements the Newsletter Delivery section of spec-003, adding the complete email delivery pipeline for the weekly newsletter system.

Changes

New Files

• src/lib/email.ts -- SMTP email utility using cloudflare:sockets TCP API. Implements the full SMTP protocol (EHLO, STARTTLS, AUTH LOGIN, MIME multipart) directly since nodemailer is unavailable on Cloudflare Workers. Auto-detects server capabilities: skips STARTTLS and AUTH when not advertised (e.g. local Mailpit).
• src/lib/jwt.ts -- JWT sign/verify using Web Crypto API (HS256). No Node.js crypto dependency. Used for unsubscribe tokens.
• src/lib/newsletter-delivery.ts -- Core delivery logic: iterates all users, generates personalized newsletters, sends emails to opted-in contacts, logs results, and cleans up old newsletters (keeps 4 most recent per user).
• src/worker.ts -- Custom Cloudflare Worker entry point with scheduled handler for cron triggers.
• src/pages/newsletter/unsubscribe.astro -- Public unsubscribe page. Validates JWT token, disables newsletter_enabled on the contact record. Shows success/error UI.
• src/pages/api/newsletter/trigger.ts -- Manual trigger endpoint for the newsletter delivery, protected by NEWSLETTER_SECRET bearer token. Works around wrangler --test-scheduled issue with Astro adapter.

Modified Files

• src/components/react/ContactInfoForm.tsx -- Added newsletter preference checkbox (email contacts only)
• src/pages/api/user/contact-info.ts -- Handle newsletter_enabled in create/update operations
• src/lib/auth.ts -- Added /newsletter/unsubscribe and /api/newsletter/trigger to publicRoutes
• astro.config.mjs -- Configured workerEntryPoint for custom worker with scheduled export
• wrangler.jsonc -- Added cron trigger (0 12 * * 1 -- Monday 12:00 UTC)
• .env.example -- Documented new env vars: SUPABASE_SERVICE_ROLE_KEY, SMTP_*, NEWSLETTER_SECRET, SITE_URL
• specs/003-weekly-newsletter.md -- Marked 10 delivery tasks as complete

Spec Progress

After this PR, spec-003 has all Database Schema, Newsletter Generation, and Newsletter Delivery tasks complete. Remaining: Testing (3 tasks) and Rollout (2 tasks).

Testing

Code quality

• Verify lint passes: just lint
• Verify unit tests pass: just test-unit

Newsletter preference toggle

• Edit an email contact info in Profile settings -- confirm the "Subscribe to weekly newsletter" checkbox appears
• Verify the checkbox does NOT appear for phone-type contacts
• Toggle the newsletter checkbox on and save -- verify newsletter_enabled = true in Supabase Studio (contact_info table)

Unsubscribe page

• Visit /newsletter/unsubscribe with no token -- verify error page shows
• Generate a test JWT and visit /newsletter/unsubscribe?token=<jwt> -- verify success page and newsletter_enabled is set to false

Cron job (local testing)

Note: The wrangler dev --test-scheduled / /__scheduled approach does not work with the Astro Cloudflare adapter — the scheduled handler never actually executes. This is a known limitation. Instead, use the POST /api/newsletter/trigger endpoint described below.

1. Ensure backend is running: just start-backend
2. Set up .env with the required vars (SUPABASE_SERVICE_ROLE_KEY, SMTP_*, NEWSLETTER_SECRET, SITE_URL). Defaults in .env.example point to local Mailpit.
3. In Supabase Studio (http://localhost:54323), set newsletter_enabled = true on at least one email-type contact in the contact_info table
4. Build and start wrangler:

   just build-frontend
   npx wrangler dev --port 8787
   

5. Trigger delivery via API:

   curl -X POST http://localhost:8787/api/newsletter/trigger \
     -H "Authorization: Bearer <your-NEWSLETTER_SECRET>" \
     -H "Origin: http://localhost:8787"
   

   The Origin header is required to bypass Astro's CSRF protection for non-browser requests.
6. Verify results:
   • newsletter table in Supabase Studio -- a new row per user
   • newsletter_send_log table -- a row per opted-in email contact with status: sent
   • Mailpit (http://localhost:54324) -- newsletter email for each opted-in address

• Cron endpoint returns {"success": true} with status 200
• newsletter rows created (one per user)
• newsletter_send_log rows created with status: sent
• Emails arrive in Mailpit with correct subject ("The Good Folk Market Weekly"), HTML body, and unsubscribe link
• Unauthorized requests (wrong/missing bearer token) return 401

New Environment Variables

Variable	Purpose
SUPABASE_SERVICE_ROLE_KEY	Service role key for newsletter delivery (bypasses RLS)
SMTP_HOST	SMTP server hostname
SMTP_PORT	SMTP server port (587 for STARTTLS, 465 for implicit TLS)
SMTP_USER	SMTP authentication username
SMTP_PASS	SMTP authentication password
SMTP_FROM	Newsletter sender email address
NEWSLETTER_SECRET	HS256 secret for JWT unsubscribe tokens + delivery trigger auth
SITE_URL	Base URL for links in emails

## Summary Implements the Newsletter Delivery section of spec-003, adding the complete email delivery pipeline for the weekly newsletter system. ## Changes ### New Files - **`src/lib/email.ts`** -- SMTP email utility using `cloudflare:sockets` TCP API. Implements the full SMTP protocol (EHLO, STARTTLS, AUTH LOGIN, MIME multipart) directly since `nodemailer` is unavailable on Cloudflare Workers. Auto-detects server capabilities: skips STARTTLS and AUTH when not advertised (e.g. local Mailpit). - **`src/lib/jwt.ts`** -- JWT sign/verify using Web Crypto API (HS256). No Node.js crypto dependency. Used for unsubscribe tokens. - **`src/lib/newsletter-delivery.ts`** -- Core delivery logic: iterates all users, generates personalized newsletters, sends emails to opted-in contacts, logs results, and cleans up old newsletters (keeps 4 most recent per user). - **`src/worker.ts`** -- Custom Cloudflare Worker entry point with `scheduled` handler for cron triggers. - **`src/pages/newsletter/unsubscribe.astro`** -- Public unsubscribe page. Validates JWT token, disables `newsletter_enabled` on the contact record. Shows success/error UI. - **`src/pages/api/newsletter/trigger.ts`** -- Manual trigger endpoint for the newsletter delivery, protected by `NEWSLETTER_SECRET` bearer token. Works around `wrangler --test-scheduled` issue with Astro adapter. ### Modified Files - **`src/components/react/ContactInfoForm.tsx`** -- Added newsletter preference checkbox (email contacts only) - **`src/pages/api/user/contact-info.ts`** -- Handle `newsletter_enabled` in create/update operations - **`src/lib/auth.ts`** -- Added `/newsletter/unsubscribe` and `/api/newsletter/trigger` to `publicRoutes` - **`astro.config.mjs`** -- Configured `workerEntryPoint` for custom worker with `scheduled` export - **`wrangler.jsonc`** -- Added cron trigger (`0 12 * * 1` -- Monday 12:00 UTC) - **`.env.example`** -- Documented new env vars: `SUPABASE_SERVICE_ROLE_KEY`, `SMTP_*`, `NEWSLETTER_SECRET`, `SITE_URL` - **`specs/003-weekly-newsletter.md`** -- Marked 10 delivery tasks as complete ## Spec Progress After this PR, spec-003 has all **Database Schema**, **Newsletter Generation**, and **Newsletter Delivery** tasks complete. Remaining: Testing (3 tasks) and Rollout (2 tasks). ## Testing ### Code quality - [x] Verify lint passes: `just lint` - [x] Verify unit tests pass: `just test-unit` ### Newsletter preference toggle - [x] Edit an email contact info in Profile settings -- confirm the "Subscribe to weekly newsletter" checkbox appears - [x] Verify the checkbox does NOT appear for phone-type contacts - [x] Toggle the newsletter checkbox on and save -- verify `newsletter_enabled = true` in Supabase Studio (`contact_info` table) ### Unsubscribe page - [x] Visit `/newsletter/unsubscribe` with no token -- verify error page shows - [ ] Generate a test JWT and visit `/newsletter/unsubscribe?token=<jwt>` -- verify success page and `newsletter_enabled` is set to false ### Cron job (local testing) > **Note:** The `wrangler dev --test-scheduled` / `/__scheduled` approach does **not** work with the Astro Cloudflare adapter — the scheduled handler never actually executes. This is a known limitation. Instead, use the `POST /api/newsletter/trigger` endpoint described below. 1. Ensure backend is running: `just start-backend` 2. Set up `.env` with the required vars (`SUPABASE_SERVICE_ROLE_KEY`, `SMTP_*`, `NEWSLETTER_SECRET`, `SITE_URL`). Defaults in `.env.example` point to local Mailpit. 3. In Supabase Studio (http://localhost:54323), set `newsletter_enabled = true` on at least one email-type contact in the `contact_info` table 4. Build and start wrangler: ```bash just build-frontend npx wrangler dev --port 8787 ``` 5. Trigger delivery via API: ```bash curl -X POST http://localhost:8787/api/newsletter/trigger \ -H "Authorization: Bearer <your-NEWSLETTER_SECRET>" \ -H "Origin: http://localhost:8787" ``` The `Origin` header is required to bypass Astro's CSRF protection for non-browser requests. 6. Verify results: - **`newsletter` table** in Supabase Studio -- a new row per user - **`newsletter_send_log` table** -- a row per opted-in email contact with `status: sent` - **Mailpit** (http://localhost:54324) -- newsletter email for each opted-in address - [x] Cron endpoint returns `{"success": true}` with status 200 - [x] `newsletter` rows created (one per user) - [x] `newsletter_send_log` rows created with `status: sent` - [x] Emails arrive in Mailpit with correct subject ("The Good Folk Market Weekly"), HTML body, and unsubscribe link - [x] Unauthorized requests (wrong/missing bearer token) return 401 ## New Environment Variables | Variable | Purpose | |---|---| | `SUPABASE_SERVICE_ROLE_KEY` | Service role key for newsletter delivery (bypasses RLS) | | `SMTP_HOST` | SMTP server hostname | | `SMTP_PORT` | SMTP server port (587 for STARTTLS, 465 for implicit TLS) | | `SMTP_USER` | SMTP authentication username | | `SMTP_PASS` | SMTP authentication password | | `SMTP_FROM` | Newsletter sender email address | | `NEWSLETTER_SECRET` | HS256 secret for JWT unsubscribe tokens + delivery trigger auth | | `SITE_URL` | Base URL for links in emails |

exe-dev-bot added 1 commit 2026-02-06 04:55:05 -05:00

feat(newsletter): add delivery pipeline ... b98f700bd8

- SMTP email utility using cloudflare:sockets for CF Workers
- JWT sign/verify using Web Crypto API (HS256)
- Newsletter cron job: generate, send, and cleanup per user
- Custom worker entry with scheduled handler for cron triggers
- Newsletter preference toggle on email contact info
- Unsubscribe endpoint with JWT validation (public route)
- Configure workerEntryPoint in astro.config.mjs
- Add cron trigger to wrangler.jsonc (Monday 12:00 UTC)
- Document new env vars in .env.example

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-06 05:03:16 -05:00

fix: use exact public route for unsubscribe, fix CI lint errors ... 81c01bc378

- Use /newsletter/unsubscribe instead of /newsletter prefix in publicRoutes
- Add Cloudflare Worker globals to ESLint config for src/worker.ts
- Fix console.log -> console.warn in newsletter-cron.ts

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-06 05:04:15 -05:00

fix: replace ambiguous unicode with ASCII in source files ... 3798aa6e97

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-06 05:18:39 -05:00

fix(worker): use exported handler path from @astrojs/cloudflare ... 1cd13e0a4e

Import from '@astrojs/cloudflare/handler' instead of internal
dist path that isn't a valid package export.

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 07:21:53 -05:00

fix: resolve type-check errors for Cloudflare Worker types ... 59ee0adeb1

- Add local interfaces for ExecutionContext and ScheduledEvent in worker.ts
  instead of relying on @cloudflare/workers-types globals
- Add @ts-expect-error for cloudflare:sockets import in email.ts
- Cast handle() call to any to avoid Request type mismatch

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 07:24:57 -05:00

fix: correct .env.example comment for service role key ... 554eea2692

The key is obtained via `npx supabase status` (listed as "Secret"
under Authentication Keys), not from `just start-backend` output.

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 07:27:08 -05:00

chore: expose Mailpit SMTP port and default .env.example to local dev ... 1f9a657cf8

- Uncomment smtp_port = 54325 in supabase/config.toml so Mailpit
  SMTP is available for newsletter testing out of the box
- Default SMTP_* values in .env.example to localhost:54325 (Mailpit)
  with comments explaining local vs production usage

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 07:35:02 -05:00

feat: show newsletter subscription status on contact info card ... 4faaba38cf

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 4 commits 2026-02-07 07:59:31 -05:00

feat: add /api/cron/newsletter endpoint for manual trigger ... 70f4b89ed9

Works around the known issue where wrangler dev --test-scheduled
doesn't invoke the scheduled handler with the Astro Cloudflare adapter.
Also useful for manual triggering in production.

Protected by NEWSLETTER_SECRET bearer token.

Co-authored-by: Shelley <shelley@exe.dev>

fix(email): skip STARTTLS and AUTH when server doesn't advertise them ... e1363f2814

Parse EHLO capabilities before attempting STARTTLS or AUTH LOGIN.
This allows the SMTP client to work with local test servers like
Mailpit/Inbucket that don't support TLS or authentication.

Co-authored-by: Shelley <shelley@exe.dev>

chore(worker): clean up debug logging in scheduled handler ... e7c1cd5bc4

Remove verbose console.warn statements added during debugging.
Use ctx.waitUntil() for non-blocking execution.

Co-authored-by: Shelley <shelley@exe.dev>

style: apply prettier formatting ... bae40169e2

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 08:06:39 -05:00

refactor: rename newsletter-cron to newsletter-delivery ... 45571d9bd2

- src/lib/newsletter-cron.ts → src/lib/newsletter-delivery.ts
- runNewsletterCron() → deliverNewsletters()
- /api/cron/newsletter → /api/newsletter/trigger

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 08:10:34 -05:00

docs: add Cloudflare Worker secrets to infrastructure docs ... 18b87971dc

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot force-pushed feat/newsletter-delivery from 18b87971dc to 4a2a919744 2026-02-07 08:12:21 -05:00 Compare

exe-dev-bot added 1 commit 2026-02-07 08:13:30 -05:00

fix(eslint): allow service-client import in newsletter trigger endpoint ... 3504d7bc84

Update ESLint config instead of using eslint-disable-next-line.

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 08:14:54 -05:00

docs(agents): add learnings from newsletter delivery work ... e570470ac7

- Never use eslint-disable comments; update eslint.config.js instead
- Document restricted import pattern for service-client
- Note that db-reset clears manual data changes
- Add newsletter local testing gotchas (wrangler scheduled, SMTP, CSRF)

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 2 commits 2026-02-07 08:21:16 -05:00

docs: update task list 0f2763460c

docs(agents): align lint-suppress guidance with shelley-extras ... 579d976654

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot force-pushed feat/newsletter-delivery from 579d976654 to 2ebe15f1d5 2026-02-07 08:26:06 -05:00 Compare

exe-dev-bot added 3 commits 2026-02-07 11:32:01 -05:00

docs: expand testing task list in spec-003 with granular test items ... f6300d63bc

Co-authored-by: Shelley <shelley@exe.dev>

refactor: replace custom JWT with jose library ... 44d4d77a21

Co-authored-by: Shelley <shelley@exe.dev>

docs: restructure testing task list as checklist with sub-points ... 0a3dc1cad2

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 3 commits 2026-02-07 16:16:31 -05:00

fix(worker): replace eslint-disable with proper type declaration for handle ... 2880b40ad7

Co-authored-by: Shelley <shelley@exe.dev>

fix(newsletter): remove hardcoded SITE_URL fallbacks ... 1836f94689

SITE_URL must be set as an environment variable. Failing loudly on
misconfiguration is better than silently using a wrong URL.

Co-authored-by: Shelley <shelley@exe.dev>

chore(nix): add wrangler CLI to dev shell PATH ... 300626e84a

Wrangler is already a devDependency in package.json. Adding
node_modules/.bin to PATH makes it (and all other local binaries)
available directly without the npx prefix.

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 16:19:26 -05:00

chore(nix): add wrangler CLI to dev shell ... 3dca285984

Use pkgs.wrangler from nixpkgs instead of a node_modules/.bin PATH hack.

Co-authored-by: Shelley <shelley@exe.dev>

addison commented 2026-02-07 16:25:40 -05:00

Owner

Copy link

Squashed locally.

Squashed locally.

addison closed this pull request 2026-02-07 16:25:40 -05:00

Some checks failed

Hide all checks

CI / Unit Tests (pull_request) Successful in 2m8s

Required

Details

CI / Lint, Type Check & Format (pull_request) Failing after 2m41s

Required

Details

CI / E2E Tests (pull_request) Failing after 3m49s

Required

Details

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kwila/market!70

No description provided.