fix(newsletter): process one user per invocation to respect subrequest limit #74

Merged

addison merged 7 commits from exe-dev-bot/market:fix/newsletter-one-per-invocation into main 2026-02-07 17:58:43 -05:00

Conversation 0 Commits 7 Files changed 5 +173 -113

exe-dev-bot commented 2026-02-07 17:35:33 -05:00

Contributor

Copy link

Problem

Cloudflare Workers limits each invocation to 50 subrequests (free tier). Each Supabase REST API call counts as one subrequest. Processing a single user requires ~10 subrequests, so the previous approach of processing all users in one invocation hit the limit at ~5 users (out of 17).

See Cloudflare Workers Limits.

Solution

Process one user per invocation via a frequent cron trigger. Each invocation:

1. Finds the next user without a newsletter in the last 7 days (2 subrequests)
2. Generates and delivers the newsletter for that user (~10 subrequests)
3. Cleans up old newsletters for that user (~3 subrequests)
4. When all users are up-to-date, returns immediately (2 subrequests)

At every 5 minutes Mon–Sat, this supports up to 1,728 users/week.

Changes

File	Change
src/lib/newsletter-delivery.ts	Refactored to process one user per call. Added findNextUser() with 7-day dedup guard. Sunday code guard. Cleanup scoped to single user.
wrangler.jsonc	Cron: 0 12 * * MON → */5 * * * MON-SAT
src/worker.ts	Updated comments
specs/003-weekly-newsletter.md	New design decision (subrequest limit), updated task list

Testing

1. just start-backend
2. Enable newsletter_enabled on a test email contact in Supabase Studio
3. Trigger manually:

   echo -n "Newsletter secret: " && read -s SECRET && echo && echo "Sending..." && \
   curl -s -X POST https://market.kwila.cloud/api/newsletter/trigger \
     -H "Authorization: Bearer $SECRET" \
     -H "Origin: https://market.kwila.cloud" \
     -w "\nHTTP %{http_code}\n"; unset SECRET
   

4. Verify in Supabase Studio:
   • newsletter table has one new row
   • newsletter_send_log has send entries
5. Trigger again — should pick up the next user (not the same one)
6. Trigger on a Sunday (UTC) — should be a no-op

## Problem Cloudflare Workers limits each invocation to **50 subrequests** (free tier). Each Supabase REST API call counts as one subrequest. Processing a single user requires ~10 subrequests, so the previous approach of processing all users in one invocation hit the limit at ~5 users (out of 17). See [Cloudflare Workers Limits](https://developers.cloudflare.com/workers/platform/limits/#subrequests). ## Solution Process **one user per invocation** via a frequent cron trigger. Each invocation: 1. Finds the next user without a newsletter in the last 7 days (2 subrequests) 2. Generates and delivers the newsletter for that user (~10 subrequests) 3. Cleans up old newsletters for that user (~3 subrequests) 4. When all users are up-to-date, returns immediately (2 subrequests) At every 5 minutes Mon–Sat, this supports up to **1,728 users/week**. ## Changes | File | Change | |---|---| | `src/lib/newsletter-delivery.ts` | Refactored to process one user per call. Added `findNextUser()` with 7-day dedup guard. Sunday code guard. Cleanup scoped to single user. | | `wrangler.jsonc` | Cron: `0 12 * * MON` → `*/5 * * * MON-SAT` | | `src/worker.ts` | Updated comments | | `specs/003-weekly-newsletter.md` | New design decision (subrequest limit), updated task list | ## Testing 1. `just start-backend` 2. Enable `newsletter_enabled` on a test email contact in Supabase Studio 3. Trigger manually: ```bash echo -n "Newsletter secret: " && read -s SECRET && echo && echo "Sending..." && \ curl -s -X POST https://market.kwila.cloud/api/newsletter/trigger \ -H "Authorization: Bearer $SECRET" \ -H "Origin: https://market.kwila.cloud" \ -w "\nHTTP %{http_code}\n"; unset SECRET ``` 4. Verify in Supabase Studio: - `newsletter` table has one new row - `newsletter_send_log` has send entries 5. Trigger again — should pick up the **next** user (not the same one) 6. Trigger on a Sunday (UTC) — should be a no-op

exe-dev-bot added 5 commits 2026-02-07 17:35:33 -05:00

fix: process one newsletter user per invocation to respect subrequest limit ... 5e6b5d2873

Cloudflare Workers free tier allows 50 subrequests per invocation.
Each user needs ~10 Supabase calls, so processing all users at once
hits the limit after ~5 users.

Changes:
- deliverNewsletters() now processes ONE user per invocation instead
  of looping through all users
- 7-day dedup guard: skips users who already have a recent newsletter
- Sunday guard: refuses to run on Sunday (UTC) as belt-and-suspenders
- Cleanup scoped to the single processed user (saves subrequests)
- Cron changed from '0 12 * * MON' to '*/5 * * * MON-SAT'
- Updated spec and worker comment to match new schedule

Co-authored-by: Shelley <shelley@exe.dev>

docs: add pre-commit hook guidance to AGENTS.md ... b140210138

Instruct agents to run 'just setup' once per session instead of
manually running lint/format/type-check. Pre-commit hooks handle
these automatically on every commit.

Co-authored-by: Shelley <shelley@exe.dev>

docs(spec): add subrequest limit design decision and update task list ... 2e0d454173

Co-authored-by: Shelley <shelley@exe.dev>

docs: clarify just setup only needs to run once ... 55c9af9b99

Co-authored-by: Shelley <shelley@exe.dev>

docs(spec): clarify practical user limit for one-per-invocation approach ... c6ade8f5eb

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 17:54:02 -05:00

fix: batch image URL signing, add display_name to logs, document race condition ... 33c55e0940

Co-authored-by: Shelley <shelley@exe.dev>

exe-dev-bot added 1 commit 2026-02-07 17:58:25 -05:00

fix: comment 956b590931

addison merged commit 2d32b9c88b into main 2026-02-07 17:58:43 -05:00

addison deleted branch fix/newsletter-one-per-invocation 2026-02-07 17:58:43 -05:00

addison referenced this pull request from a commit 2026-02-07 17:58:44 -05:00

fix(newsletter): process one user per invocation to respect subrequest limit (#74)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

kwila/market!74

No description provided.