Async generation job polled with no deadline or cancel

A billable job polled with no deadline and no way to cancel it.

Rule orphaned-async-jobCost audit

Why it costs money

Image, video and batch generation jobs are billed by the provider for the full run once they are submitted, whether or not anyone reads the result. The polling loop that waits for them is free, which is why this cost does not show up next to any call in a trace.

When the poller dies (a serverless timeout, a closed browser tab, a container restart) the job keeps running and billing. The user sees nothing, clicks generate again, and pays for a second run. Every timeout can become a double charge.

Example

import Replicate from "replicate";

const replicate = new Replicate();

export async function POST(req: Request) {
  const { prompt } = await req.json();
  let prediction = await replicate.predictions.create({
    model: "black-forest-labs/flux-schnell",
    input: { prompt },
  });
  while (prediction.status !== "succeeded" && prediction.status !== "failed") {
    await new Promise((r) => setTimeout(r, 1000));
    prediction = await replicate.predictions.get(prediction.id);
  }
  return Response.json(prediction.output);
}

import Replicate from "replicate";

const replicate = new Replicate();
const DEADLINE_MS = 60_000;

export async function POST(req: Request) {
  const { prompt } = await req.json();
  let prediction = await replicate.predictions.create({
    model: "black-forest-labs/flux-schnell",
    input: { prompt },
  });
  const started = Date.now();
  while (!["succeeded", "failed", "canceled"].includes(prediction.status)) {
    if (Date.now() - started > DEADLINE_MS || req.signal.aborted) {
      await replicate.predictions.cancel(prediction.id);
      return new Response("Generation timed out", { status: 504 });
    }
    await new Promise((r) => setTimeout(r, 1000));
    prediction = await replicate.predictions.get(prediction.id);
  }
  return Response.json(prediction.output);
}

How to fix it

Give the wait a deadline that is shorter than your platform's function timeout, and on timeout or client disconnect call the provider's cancel endpoint before returning. Cancelling stops further billing for the remaining run.

Where the provider supports it, use its blocking wait mode or a webhook instead of a poll loop, and store the job id so a retry from the same user can pick up the existing job rather than start a new one.

What the audit detects

• A loop whose body checks a job's status (status !== "succeeded", "processing", "queued", a poll() call and similar) in a file that talks to an AI provider.
• The loop header and body must have no deadline or attempt bound: no Date.now() - start, deadline, elapsed, maxAttempts, maxPollTime or range(n).
• The whole file must have no cancel or abort path (cancel, abort, AbortController, terminate). Any of these silences the rule. One finding per file; test, example and docs folders are skipped.

Research

Replicate docs: Create a prediction. Async generation jobs are billed for the run once submitted, independent of whether the result is fetched. Providers expose an explicit cancel endpoint so an abandoned job can be stopped.