Event-Driven Agent Jobs: Fire on Change, Produce Information

A nightly job runs whether or not anything changed, and its failures are investigated hours later by someone with no memory of what happened. Events do neither.

Scheduled jobs are the wrong default

The first instinct when automating agent work is a nightly cron: run the dependency upgrade at 3am, run the test-coverage backfill on Sundays. It works, and it has two properties that get worse as you add more of them.

Scheduled work runs whether or not anything changed, burning tokens on no-ops. And it arrives detached from the thing that prompted it — a failure at 3am is investigated at 10am by someone with no memory of what happened yesterday.

Event-driven work fires when something actually happened, on the artifact it happened to, with the context still fresh.

Which events are worth wiring

Every output in the right column is a comment or a draft. That is the pattern worth holding to: event-driven agent jobs produce information, not changes. The events are frequent and the supervision is thin, so the blast radius should be a notification.

The output contract

Since these jobs produce text a human skims, format matters more than for work nobody reads. Two conventions save people time.

Verdict first. A comment opening with three paragraphs of context gets skipped. One opening with Risk: low gets read to the end when it says high.

State what was not checked. An assessment that lists only findings reads as complete. One that says "did not review the vendored test fixtures" tells the reader where their own attention is still required.

A job worth having

name: Assess dependency PR
on:
  pull_request:
    types: [opened]

jobs:
  assess:
    if: github.actor == 'dependabot[bot]'
    runs-on: agent-runner
    timeout-minutes: 10
    permissions:
      contents: read
      pull-requests: write     # comment only — cannot push
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }

      - name: Assess
        env:
          AGENT_MAX_TOKENS: 150000
        run: |
          ./ci/agent-task.sh > assessment.md <<'PROMPT'
          A dependency upgrade PR was opened. Determine:
            1. Which of OUR code uses this package. List the files.
            2. Does the changelog contain breaking changes that
               affect those call sites specifically?
            3. Risk: low / medium / high, with one sentence why.
          Do not modify any file. Output markdown only.
          PROMPT

      - name: Comment
        run: gh pr comment ${{ github.event.number }} -F assessment.md

Note the permissions. contents: read and pull-requests: write means the job can read the repository and post a comment, and can do nothing else — no push, no deploy, no publishing. That is the whole safety design, and it is two lines.

Where the job should run

One decision shapes everything else: does the agent job run on your CI provider's shared runners, or somewhere you control?

For jobs that only read and comment, shared runners are fine — the permissions block is the control. The moment a job needs a credential to anything real, a dedicated runner is worth the afternoon, because it lets you apply egress filtering that the shared pool cannot.

Debounce, or you will pay for it

Events arrive in bursts. Ten pushes to a branch in twenty minutes fires ten jobs, nine of which are wasted.

concurrency:
  group: agent-assess-${{ github.event.pull_request.number }}
  cancel-in-progress: true

Pair it with a path filter so a documentation-only push does not trigger a code-assessment job:

on:
  pull_request:
    types: [opened, synchronize]
    paths-ignore:
      - '**/*.md'
      - 'docs/**'

Three lines, and it converts a burst into one run against the latest state. Without it, an agent job on a busy repository is one of the more expensive things in your CI bill, and most of the spend is on states nobody ever saw.

Idempotence, so comments do not pile up

A job that comments on every run leaves eleven comments on an active PR. Update one instead.

MARKER="<!-- agent-assessment -->"
BODY="$MARKER"$'\n'"$(cat assessment.md)"

ID=$(gh pr view "$PR" --json comments \
     --jq ".comments[] | select(.body | startswith(\"$MARKER\")) | .id" \
     | head -1)

if [ -n "$ID" ]; then
  gh api -X PATCH "repos/$REPO/issues/comments/$ID" -f body="$BODY"
else
  gh pr comment "$PR" --body "$BODY"
fi

The failure mode to design against

Event-driven jobs fail differently from scheduled ones. A nightly job that breaks is noticed the next morning because its output stops arriving. An event job that breaks produces nothing on an event nobody was watching for, and can be dead for weeks.

Two cheap defences. Emit a heartbeat — a metric or a log line on every run, including successful ones — so an absence is detectable. And check periodically that the job has run at all:

gh run list --workflow=dependency-review.yml --limit 20   --json conclusion,createdAt   --jq 'group_by(.conclusion)
        | map({outcome: .[0].conclusion, n: length})'

Run that monthly. An empty result on a repository with recent dependency PRs means the trigger condition stopped matching — usually because a bot's account name changed, or a branch protection rule now blocks the event. Neither produces an error anywhere.

Make it easy to switch off

Event-driven jobs become background noise faster than scheduled ones, because they fire constantly and each individual comment is plausible. Two things prevent that.

An opt-out label. A no-agent label that the workflow checks costs one line and gives people a way to silence a job that is being unhelpful on a particular change — which is much better than them asking for the whole job to be deleted.

A usefulness review. Once a quarter, ask whether anyone has acted on these comments. A job producing output nobody reads is a job that should be turned off. This is easy to check and almost never checked, and event-driven automation accumulates precisely because switching things off requires someone to decide to.