Community PulseWeekly developer intelligence

How it works

From community threads to a Friday brief

Every Friday at 07:00 IST a pipeline reads the week’s discussions on Hacker News, Stack Overflow and DEV, counts them with code, asks Gemini to group and explain them, checks every claim against the source, and publishes the brief here and by email.

The pipeline

Ten steps, one run, usually two to three minutes. Each step records its status, time and detail, which the run log below shows. The same run, actor by actor, is in the components map.

  1. 1 · Trigger

    1. Vercel Cron

      Schedule

      Every Friday at 07:00 IST, or on demand with “Run now” below.

    2. Load topic config

      Supabase

      Active categories, keywords, context terms and sources, read fresh each run.

  2. 2 · Collect & count

    1. Collectors

      Public APIs

      Four weeks of posts, in parallel. Timeouts and one retry; a failing source is isolated.

      • HN
      • Stack Overflow
      • DEV

    2. Matcher

      Code

      Keywords plus context terms, word-boundary safe; generic keywords need a context term.

    3. Stats

      Code

      Mentions and threads per keyword, category and week; change and trend labels.

  3. 3 · Understand

    1. AI classify

      AI · Gemini

      Is each thread on-topic, and which pain point? Cached per thread.

    2. AI synthesize

      AI · Gemini

      Clusters pain points and drafts underserved problems, objections and sprint ideas.

    3. Validator

      Code

      Citations must exist in the right week, quotes must be verbatim, counts are recomputed.

    Gemini unavailable? A keyword-based fallback brief is built instead and marked as such.

  4. 4 · Publish

    1. Supabase briefs

      Supabase

      One brief per week, upserted; every step logged to pipeline_runs.

    2. Delivery

      Email + web

      Email with the headline and top pain points, and the full brief on this site.

      • Email · Resend
      • Web
One weekly run, start to finish.AI steps (Gemini): words and groupings onlyDeterministic code, storage and delivery: every number

Reliability rules

What keeps a brief trustworthy when the model, a community or the network misbehaves.

  • Numbers come from code, words from the AI

    The model never outputs a count. It groups thread keys; code counts them, computes the week-over-week change and labels the trend with fixed rules.

  • A model fallback chain

    gemini-3.6-flashthengemini-3-flash-previewthengemini-3.5-flash-liteRate limits, server errors, timeouts and invalid JSON move on to the next model; configuration errors (400/401/403) fail fast. Replies must be schema-valid JSON.

  • Every citation is checked

    Cited threads must exist in the collected data and fall in the right week; quotes must appear word for word in the post. Anything else is dropped and counted in the brief’s quality panel.

  • A brief even when something breaks

    If Gemini is unavailable, the brief is built from keyword signals instead and says so at the top. A failing community is isolated: the brief is marked partial and names it, rather than failing.

  • Prompt-injection safe

    Posts reach the model as delimited, untrusted data with a standing rule never to follow instructions inside them. Output is schema-constrained, so injected text cannot change its shape, and citations are re-checked.

  • Read-only by default

    Supabase row-level security lets the website’s key only read. Writes go through token-gated database functions (only the token’s SHA-256 is stored). Manual runs need a passcode and have a cooldown; the cron route needs its secret.

Data sources

Three public communities with free, terms-friendly APIs, chosen because that is where developers talk about review and CI pain in the open.

Hacker News

Algolia HN Search API

Stories and comments, searched per keyword.

Signals: Points and comments.

Stack Overflow

Stack Exchange API

Recent questions in the configured tags.

Signals: Answers, views and accepted answers, which power “Open questions”.

DEV Community

Forem API

Recent articles in the configured tags.

Signals: Reactions and comments.

Not monitored

Reddit answers server requests without credentials with HTTP 403, and new API apps need Reddit’s approval.

X and Discord need paid or privileged API access.

Sources are pluggable, so one can be added later without touching the rest of the pipeline.

Recent runs

Live from the pipeline_runs table. Hover a step icon for its time and detail.

Loading recent runs…

Run the pipeline now

Regenerates the brief for a week on demand. Passcode-protected, one run per week at a time, with a short cooldown to protect the free API quotas.