Workflow Automation

markdown# Ajolla Workflow Automation

Automations that live in your app: **trigger** (an event, a schedule, or a manual run) → optional **conditions** on the trigger payload → ordered **steps** (send a notification, emit an event, call a signed webhook, wait, set a variable). Runs are durable with no extra infrastructure: two tables and a cron route.

## What was installed

**Routes** (all require `workflows.manage`)
- `/admin/workflows` — list: trigger, on/off toggle, last run status, runs today vs. the daily cap.
- `/admin/workflows/new`, `/admin/workflows/[workflowId]` — builder: trigger picker, conditions, step list (add / reorder / remove), dry-run tester with a sample payload, "Run now", recent runs, delete.
- `/admin/workflows/[workflowId]/runs` — run history with status filter, retry and cancel.
- `/admin/workflows/[workflowId]/runs/[runId]` — per-step, per-attempt logs (input, output, error, duration), trigger data, variables, retry/cancel.
- `GET /api/ajolla/workflows/tick` — cron (Bearer `CRON_SECRET`): starts scheduled runs, advances due runs, prunes old history.

**Code**
- `lib/ajolla/workflows/core.ts` — CRUD, run history, retry/cancel, event-type catalog; re-exports everything below. Framework-free.
- `engine.ts` — enqueueing (event / schedule / manual), lease-based execution, retries, delays, dry runs.
- `definitions.ts` (zod schemas + `LIMITS`), `conditions.ts`, `template.ts`, `schedule.ts`, `ssrf.ts` — pure helpers, unit-tested.
- `http.ts` — Node webhook transport (DNS-pinned SSRF guard). `server.ts` — env wiring. `view.ts` — UI helpers.
- `lib/ajolla/subscribers/workflows.ts` — `"*"` subscriber. `lib/ajolla/admin/workflows.ts` — admin registration.
- `app/admin/workflows/**`, `components/ajolla/workflows/*`.

**Tables** (`ajolla_workflows_`)
- `workflows` — definition (trigger, `conditions` and `steps` as validated JSON), `enabled`, `daily_run_cap`, `next_run_at` (schedules), last run status.
- `runs` — one execution: a snapshot of the steps, trigger `context`, `vars`, step `outputs`, `status` (`pending → running → waiting/succeeded/failed/cancelled`), `step_index`, `attempts`, `next_run_at`, `lease_until` + `lease_version` (claiming), `chain`/`depth` (loop detection). `unique (workflow_id, dedupe_key)` makes enqueueing idempotent per event id / schedule slot.
- `step_runs` — one row per step attempt (written only by the engine with the secret key).
- `event_types` — event types seen on the bus with a small sample payload (feeds the trigger picker).

**RLS** — `workflows.manage` only (read/write; `step_runs` read-only). Other users and anonymous visitors see nothing. Run execution uses the secret-key client from the tick route, the subscriber and "Run now".

**Events** — emits `workflows.run_completed` `{ workflowId, workflowName, runId, trigger, eventId, eventType, durationMs }` and `workflows.run_failed` `{ …, error, stepId }`, plus whatever steps emit (`notifications.requested`, custom types). Every event a run emits carries `_workflow: { workflowId, runId, depth, chain }`.

**Env**
- `WORKFLOWS_WEBHOOK_SECRET` — HMAC key for the `ajolla-signature` header (kernel format `t=…,v1=…`, verify with `verifyWebhook` from `@ajolla/kernel`). Webhook steps fail without it.
- `WORKFLOWS_RUN_RETENTION_DAYS` — finished runs older than this are pruned by the tick (default 30).
- Kernel's `CRON_SECRET`.

## Setup
1. Apply migrations: `supabase db push`.
2. Schedule the tick, e.g. in `vercel.json` (Hobby plans allow daily crons only, which delays schedules/retries accordingly):
   ```json
   { "crons": [
     { "path": "/api/ajolla/events", "schedule": "*/5 * * * *" },
     { "path": "/api/ajolla/workflows/tick", "schedule": "*/5 * * * *" }
   ] }
   ```
3. Set `WORKFLOWS_WEBHOOK_SECRET` (e.g. `openssl rand -hex 32`) if you use webhook steps, and share it with the receiver.
4. Open `/admin/workflows`, create a workflow, test it with a sample payload, then turn it on.
5. Install Notifications (`npx ajolla add notifications`) for "Send notification" steps to be delivered.

## How execution works
- **Event triggers**: the `"*"` subscriber records the event type, evaluates each matching enabled workflow's conditions, and inserts a run keyed by `event:<event id>` (redelivery is a no-op). New runs then get up to 8s of inline execution; everything else is the tick's job.
- **Schedules** (UTC): every N minutes (≥5, aligned to the epoch), hourly at :MM, daily at HH:MM. The tick claims a slot by moving `next_run_at` forward with a conditional update; the run is keyed `schedule:<slot>`. Granularity is your cron interval.
- **Claiming**: a worker claims a run with `update … where id = ? and status = ? and lease_version = ?` (bumping `lease_version`, setting `lease_until`). Every later write is conditional on that version, so a run is never executed by two workers; a crashed worker's run is reclaimed once its lease expires (renewed after each step). Cancel/retry bump the version, stopping any holder.
- **Retries**: a failing step is retried with backoff 1, 2, 4… minutes (max 1h) up to 3 attempts; webhook 408/429/5xx and network errors are retryable, 4xx and invalid input are not. Then the run fails and emits `workflows.run_failed`. "Retry run" resumes at the same step position with the original trigger data and the workflow's current steps.
- **Delays** park the run as `waiting` until `next_run_at` (max 30 days).
- **Delivery semantics**: at-least-once per step (a crash between a side effect and its log can repeat it). Webhooks send `ajolla-delivery: <runId>:<stepId>` so receivers can dedupe.

## Limits (edit `LIMITS` in `definitions.ts`)
- 25 steps per workflow and per run; 20 conditions; daily run cap per workflow (default 100, max 10,000; manual runs bypass it but count).
- **Loops**: an event emitted by a run never re-triggers a workflow already in its `chain`, and chains stop at depth 5 — a workflow that emits its own trigger event runs once.
- **Webhooks**: POST JSON, `https://` only, no credentials in URLs, no redirects followed, 10s timeout, response read capped at 64 KB (2 KB kept in logs). Hostnames are resolved inside the connection and every address must be public (private, loopback, link-local/metadata, CGNAT, multicast, reserved, IPv6 ULA/link-local, and v4-mapped/NAT64/6to4 forms are blocked), so DNS rebinding can't bypass the check.

## Templates and conditions
- `{{event.payload.x}}`, `{{event.type}}`, `{{event.actor_id}}`, `{{vars.name}}`, `{{steps.<stepId>.status}}`, `{{steps.<stepId>.body.field}}`, `{{workflow.name}}`, `{{run.id}}`. Array items: `{{event.payload.items.0.sku}}`. A JSON value that is exactly one placeholder keeps its type.
- Conditions compare a payload field (dotted path) with `equals`, `not_equals`, `contains` (case-insensitive substring or array item), `exists`, `not_exists`, `gt`, `lt` (numbers or ISO dates). Combine with all/any. Manual runs skip conditions.

## Customising
- Add a step type: extend `stepSchema` in `definitions.ts`, `renderStep`/`runStep` in `engine.ts`, and `StepFields` in `components/ajolla/workflows/workflow-builder.tsx`.
- Trigger picker events come from `MODULE_EVENTS` in `core.ts` (filtered by installed modules) plus types seen on the bus; any custom type can be typed in.
- Styles: `components/ajolla/workflows/workflows.module.css` and the kit's `--aj-*` variables.

## Switching to Vercel Workflow (or another durable engine)
This module deliberately uses a cron + tables so it runs anywhere Next.js and Postgres do, with no extra dependency. To move to [Vercel Workflow](https://vercel.com/docs/workflow): keep `definitions.ts`, `conditions.ts`, `template.ts` and `renderStep()`; write a `"use workflow"` function that loops over `run.definition.steps`, calling each step's side effect as a `"use step"` function (webhook → `sendWebhook`, notify/emit → `emit`) and `sleep()` for delays; start it from the subscriber instead of `enqueueEventRuns`'s insert (keep inserting the run row for history and idempotency), and drop the tick route's `processDueRuns` call. Retries and leases then come from the platform.