Loading Accelerated Logic…
GA Release

The Multi-Agent WebGPU Engine & Interactions API are now generally available.

Part 8: Prompt Queues, Forking & Request Recovery

Queue follow-up prompts, branch conversations, and understand which recovery settings are saved versus active.

Category: Getting Started • Read Time: 25 min read • Updated: 2026-11-10

1. Three Workflow Bottlenecks This Part Solves

A. Waiting on Generations

Sending a prompt forces you to wait before queuing follow-up instructions. Prompt Queue enables continuous batching.

B. Risk of Overwriting Progress

Trying alternative prompt directions can ruin a working thread. Forking creates non-destructive branch points.

C. Rate Limits & 429 Errors

If a request fails, check the provider error, select another equipped model, and retry manually. Saved model fallback order is not currently applied by request execution.

2. Prompt Queue — Your Assembly Line

While an active generation is running, type your next instruction into the composer input. The main action button shifts to an indigo Queue Response button.

Prompt Queue Manager Features:
  • Isolated Staged Assets: Each queued prompt preserves its own attached files, screenshots, or CSVs independently.
  • Reorder & Delete: Drag queued cards to adjust execution order or click trash to remove.
  • Pause Queue: Complete current active turn and temporarily halt remaining queued tasks.
  • Queue Backoff Interval: Configure a 0–60 second delay in Settings → AI Engine to pace execution and prevent rate limits.
  • Last Failed Recovery: Failed queued tasks are moved to a recovery drawer for instant 1-click retry.

3. Forking / Branching — Never Lose a Good Version

Hover under any user or assistant message and click Fork (GitBranch icon).

Non-Destructive Branching: Creates a new chat thread containing history up to that exact fork point while preserving your original chat intact.

Comparison Workflows: Fork a prompt 3 times to test different design directives (e.g. Minimal vs Brutalist vs Glassmorphic) or compare multiple LLM models side by side.

4. Saved fallback order and current limits

The Model Organizer can save an ordered list of alternate models, but request execution does not currently read that list to retry a failed request. Do not rely on it for automatic failover.

  • Saved order: The Model Organizer stores the preferred sequence for a model; this is configuration data only.
  • Current retry behavior: A failed request reports its provider error. Select another model and retry manually.
  • Provider sign-in: Authentication prompts and retry behavior depend on the provider integration in use.

5. Putting All Three Together — A Real Workflow

Batch Production Setup Example:
  1. Equip the models you plan to use. Saved fallback order does not automatically switch requests between them.
  2. Create a Folder (e.g., Client Landing Pages) with Folder Memory: React + Tailwind + lucide-react.
  3. Stage client1.pdf + prompt "Build landing page for Client 1" → click Queue Response.
  4. Stage client2.png + prompt "Build landing page for Client 2" → click Queue Response.
  5. Review queued prompts and results as they finish. If a request fails, choose another model and retry it.

6. Queue Edge Cases & Failure Recovery — The Fine Print

  • A queued prompt fails mid-batch: it lands in the recovery drawer with its error; subsequent queued items continue unless you pause the queue — a single provider failure does not freeze the whole assembly line.
  • Dependent steps do not auto-chain safely: step 2 is sent with the chat context that existed when it was enqueued, not with step 1's fresh output spliced in. If step 2 needs step 1's exact result, run them sequentially in one chat instead of queueing both blindly.
  • Attachments are per-item, not global: files staged for queued item A are not visible to item B. Re-stage the file on each item that needs it, or pin it to the chat (Part 5).
  • Backoff vs. order: the 0–60s backoff delays start times between queued items; it does not reorder the queue. Pair backoff with a free/limited provider to avoid 429 storms during unattended batches.
  • Stopping mid-queue: Stop halts the active generation; pause the queue to keep remaining items from starting. Clearing the queue discards staged payloads — you cannot "unclear" it, so export or re-stage first if the items were expensive to prepare.

7. Fork Hygiene — Naming, Comparing, and Deciding

Forks multiply fast; without a decision habit you end up with nine near-identical threads and no idea which is canonical. A workable protocol:

  1. Name branches at creation. Right after forking, rename both threads with the discriminating decision: pricing – minimal vs pricing – glass. Unnamed forks are lost forks.
  2. One variable per fork. Change either the model, the prompt, or the directive — not two — or the comparison teaches you nothing.
  3. Score with a rubric. For visual work: correctness of layout, responsiveness, polish. For code: passes tests / no type errors / meets spec. Pick the winner on evidence, not vibe.
  4. Declare and declutter. Rename the winner pricing – WINNER, delete or export-and-delete the losers, and pin the final artifact to the chat for the next iteration.

8. Forking vs. Queueing — a Decision Table

Situation Use Why
Exploring 2–3 design directions from the same starting pointForkIndependent branches; you compare final artifacts side by side
Running the same checklist against several inputs (5 client PDFs, 10 bug reports)QueueUnattended batch processing with per-item attachments
Step 2 must consume step 1's exact output verbatimSequential chatQueued items don't see sibling items' fresh output
You hit a rate limit and have more prompts readyQueue + backoffBackoff paces starts so free tiers stop 429-ing mid-batch
Answer may strand you (wrong direction) and you fear losing a good stateFork first, then proceedThe original thread is your guaranteed-good checkpoint
Comparing models on an identical promptFork per modelIdentical context, one variable — the cleanest A/B test (Part 2)