std::jobs
Background jobs that survive a restart. A Job<A> pairs a kind, the
name a job is stored under, with a handler for its arguments A.
jobs.enqueue writes a job on the caller’s Conn, so it commits or
rolls back with the caller’s own writes, and a Worker claims due jobs
from a Queue and runs them.
struct EmailArgs post_id: Int end derive ToJson for EmailArgs derive FromJson for EmailArgs fn send_email(args: EmailArgs) -> Result<Unit, Failure> ... end once email_job = Job.new("email", send_email, max_attempts: 25) fn publish(post_id: Int) -> Result<Unit, sql.Error> sql.transaction(App.db, fn tx -> _ = Conn.exec(tx, Sql"UPDATE posts SET published = true WHERE id = ${post_id}")? _ = jobs.enqueue(App.queue, tx, email_job, EmailArgs{post_id:})? Ok(Unit) end) end
Queue is the interface a backend implements: lib/jobs/postgres
keeps jobs in a Postgres table. std::jobs::testing has Inline, a queue
kept in memory for tests, run_queued, which runs the due jobs in the
test’s own task, and conformance, the suite every backend passes.
Import with import std::jobs.
Exports
Section titled “Exports”Types
Interfaces
Delivery
Section titled “Delivery”A job runs at least once. If its process dies after the handler’s side effect but before the job is marked done, it runs again, so a handler must be safe to run twice.
Running jobs
Section titled “Running jobs”A Worker is built while boot builds the app value, since it creates
two supervisors, and is started in main with the handlers it runs:
fn boot() -> Result<App, String> queue = pgjobs.queue(db, url) |> Result.map_err(fn e -> "${e}")? Ok(App{..., queue:, worker: Worker.new(queue, max_running: 4)}) end fn main() Worker.start(App.worker, [email_job, batch_job])! ... end
A handler returns Ok(Unit), or a Failure: Retry after a backoff of
attempt^4 + 15 seconds plus up to a tenth more, Snooze after a delay
of its own without counting an attempt, or Discard. A trap counts as a
Retry, and so does running past the job’s timeout, which cancels the
handler’s task. A job that reaches max_attempts is discarded. At
shutdown the worker gives running jobs its shutdown_timeout, then
hands back the ones still running, so another process takes them at
once.
Unique jobs and priorities
Section titled “Unique jobs and priorities”An enqueue with unique stores nothing when a job holding the same key
is stored, and returns that job’s id with duplicate: True. A key is
the job’s kind and the Unique’s key; by default a job holds it while
it is available or running, and with across: .Kept for as long as the
queue keeps it. A period makes the key one per period, counted from
the Unix epoch:
unique = Unique{key: "post:${post_id}", period: Some(Duration"10m")} enqueued = jobs.enqueue(App.queue, tx, email_job, EmailArgs{post_id:}, unique: Some(unique))?
A worker claims the due job of the highest priority first, then the
oldest. A Job gives its kind a priority, 0 unless given, and an
enqueue may give another.
Schedules
Section titled “Schedules”A Schedule enqueues a job on a fixed interval or at the times a Cron
expression names, and a worker started with schedules enqueues each
one’s jobs as their times come:
once nightly = Schedule.cron( "nightly report", Cron"0 3 * * *", report_job, ReportArgs{}, zone: "America/New_York", )! fn main() Worker.start(App.worker, [report_job], schedules: [nightly])! ... end
Each tick’s job carries a unique key of the schedule’s name and the
tick’s time, held while the queue keeps the job, so any number of
processes can run the same schedules and each tick is enqueued once,
with no process leading. The queue also records each schedule’s latest
tick. A scheduler that comes back after missing several ticks enqueues
one job, for the latest of them, or none with missed: .Skip; one that
starts a schedule the queue has no record of enqueues nothing until its
next tick.
A cron expression is read on a wall clock in zone, UTC unless given.
A wall time that a fall-back change makes happen twice runs once, at
its first occurrence. A wall time that a spring-forward change skips
runs at the instant the clocks jump, so a 02:30 job in New York runs at
03:00 that day, once. An interval counts elapsed time from the Unix
epoch and ignores the zone.
Arguments over time
Section titled “Arguments over time”A stored job’s arguments are A’s JSON. Derived FromJson fills a
missing key from the field’s default or None and ignores a key it does
not know, so a field added with a default reads old jobs. Arguments
that do not decode are discarded with the decode error. Renaming a kind
leaves its stored jobs under the old name: Job.renamed_from gives a
handler for the old name too.
Writing a backend
Section titled “Writing a backend”Everything after Worker on this page is for a backend or a test queue:
the Queue interface and the values it takes and returns (NewJob,
Claimed, Settle), Handler, which a worker runs, and the pieces a
worker is built from (run_one, settle_for, backoff,
handlers_by_kind, wake, enqueue_due).
struct Job
Section titled “struct Job”struct Job<A> kind: String queue: String max_attempts: Int timeout: Duration priority: Int handler: (A) -> Result<Unit, Failure> end
One kind of job: the name it is stored under, the queue it goes to,
how often it may run and for how long, its priority, and its handler,
typed by its arguments A.
Job.new
Section titled “Job.new”fn new(kind: String, queue: String = "default", max_attempts: Int = 20, timeout: Duration = Duration.minutes(5), priority: Int = 0, handler: (A) -> Result<Unit, Failure>) -> Job<A>
type function on Job
A job of kind, handled by handler. A program declares each kind
once, as a file-level once, and enqueues and starts that value.
The handler comes last, so it goes in its slot whichever options a
call names: Job.new("email", send_email, max_attempts: 25).
max_attempts counts runs, including the first, and timeout
bounds each run. Of the due jobs, a worker claims those of the
highest priority first; an enqueue may give another.
job = Job.new("greet", max_attempts: 3, fn _: Int -> Ok(Unit))
assert job.kind == "greet"
assert job.queue == "default"
assert job.max_attempts == 3
assert job.timeout == Duration.minutes(5)
assert job.priority == 0
Job.renamed_from
Section titled “Job.renamed_from”fn renamed_from(job: Job<A>, old_kind: String) -> Job<A>
type function on Job
The same job under a kind it was stored under before a rename. Start a worker with both until no job of the old kind is left.
job = Job.new("send_email", fn _: Int -> Ok(Unit))
old = Job.renamed_from(job, "email")
assert old.kind == "email"
assert old.queue == job.queue
Job.kind
Section titled “Job.kind”impl Handler for Job<A>
fn kind(handler: Job<A>) -> String
impl Handler.kind
Job.timeout
Section titled “Job.timeout”impl Handler for Job<A>
fn timeout(handler: Job<A>) -> Duration
impl Handler.timeout
Job.run
Section titled “Job.run”impl Handler for Job<A>
fn run(handler: Job<A>, args: Json) -> Result<Unit, Failure>
impl Handler.run
Job.inspect
Section titled “Job.inspect”impl Debug for Job<A>
fn inspect<A>(value: Job<A>) -> String where A: Debug
impl Debug.inspect
enum Failure
Section titled “enum Failure”enum Failure Retry String Snooze { message: String, after: Duration } Discard String end
Why a job did not complete: the handler’s Err.
Variants
-
RetryRuns the job again after the backoff. The run counts as an attempt.
-
SnoozeRuns the job again no sooner than
after, and the run does not count as an attempt: for a downstream that said to come back later. -
DiscardNever runs the job again.
assert "${Failure.Retry("timed out")}" == "retry: timed out"
Failure.to_string
Section titled “Failure.to_string”impl Display for Failure
fn to_string(failure: Failure) -> String
impl Display.to_string
Failure.equal
Section titled “Failure.equal”impl Equatable for Failure
fn equal(a: Failure, b: Failure) -> Bool
impl Equatable.equal
Failure.inspect
Section titled “Failure.inspect”impl Debug for Failure
fn inspect(value: Failure) -> String
impl Debug.inspect
type JobId
Section titled “type JobId”type JobId Int
A stored job’s id, as its queue numbers it.
JobId.to_string
Section titled “JobId.to_string”impl Display for JobId
fn to_string(JobId(n)) -> String
impl Display.to_string
JobId.equal
Section titled “JobId.equal”impl Equatable for JobId
fn equal(a: JobId, b: JobId) -> Bool
impl Equatable.equal
JobId.inspect
Section titled “JobId.inspect”impl Debug for JobId
fn inspect(value: JobId) -> String
impl Debug.inspect
jobs.enqueue
Section titled “jobs.enqueue”fn enqueue<A>(queue: Queue, conn: Conn, job: Job<A>, args: A, run_at: Maybe<Instant> = None, priority: Maybe<Int> = None, unique: Maybe<Unique> = None) -> Result<Enqueued, sql.Error> where A: ToJson
Enqueues a job of job’s kind for args on conn, due at once or at
run_at, at job’s priority or priority, and returns its id. conn
may be a transaction, and the job is stored only if the transaction
commits. With unique, an enqueue that finds a stored job holding the
same key stores nothing and returns that job’s id, with duplicate
set. A failed write is the queue’s sql.Error.
import std::sql::testing.Fake
import std::jobs::testing.Inline
queue = Inline.new()
job = Job.new("greet", fn _: Int -> Ok(Unit))
first = enqueue(queue, Fake.new(), job, 1, unique: Some(Unique{key: "ada"}))?
again = enqueue(queue, Fake.new(), job, 2, unique: Some(Unique{key: "ada"}))?
assert not first.duplicate
assert again == Enqueued{id: first.id, duplicate: True}
struct Enqueued
Section titled “struct Enqueued”struct Enqueued id: JobId duplicate: Bool end
What an enqueue stored, or found: the job’s id, and whether a stored job already held the enqueue’s unique key, so nothing new was stored.
Enqueued.equal
Section titled “Enqueued.equal”impl Equatable for Enqueued
fn equal(a: Enqueued, b: Enqueued) -> Bool
impl Equatable.equal
Enqueued.inspect
Section titled “Enqueued.inspect”impl Debug for Enqueued
fn inspect(value: Enqueued) -> String
impl Debug.inspect
struct Unique
Section titled “struct Unique”struct Unique key: String across: UniqueStates = UniqueStates.Live period: Maybe<PositiveDuration> = None end
What makes an enqueue unique: a key, which names the job among those
of its kind, the states in which a stored job holds it, and an optional
period, which makes the key one per period. The period is a
PositiveDuration, since a zero or negative one has no periods to
count: a typed literal, Duration"10m", is checked when the program is
compiled, and a Duration computed at run time goes through
PositiveDuration.try_from.
Unique.key_for
Section titled “Unique.key_for”fn key_for(unique: Unique, kind: String, now: Instant) -> String
type function on Unique
The key a job of kind stores for unique when enqueued at now:
the kind, the key, and with a period the start of the period now
falls in, counted from the Unix epoch.
u = Unique{key: "post:7", period: Some(Duration"1h")}
at = Instant.from_seconds(3_600 * 24 + 1_800)
assert Unique.key_for(u, "email", at) == "[\"email\",\"post:7\",\"1970-01-02T00:00:00Z\"]"
assert Unique.key_for(Unique{key: "post:7"}, "email", at) == "[\"email\",\"post:7\"]"
Unique.equal
Section titled “Unique.equal”impl Equatable for Unique
fn equal(a: Unique, b: Unique) -> Bool
impl Equatable.equal
Unique.inspect
Section titled “Unique.inspect”impl Debug for Unique
fn inspect(value: Unique) -> String
impl Debug.inspect
enum UniqueStates
Section titled “enum UniqueStates”enum UniqueStates Live Kept end
The states in which a stored job holds its unique key.
Variants
-
LiveWhile the job is available or running: once it completes or is discarded, the key is free.
-
KeptIn every state, for as long as the queue keeps the job.
UniqueStates.equal
Section titled “UniqueStates.equal”impl Equatable for UniqueStates
fn equal(a: UniqueStates, b: UniqueStates) -> Bool
impl Equatable.equal
UniqueStates.inspect
Section titled “UniqueStates.inspect”impl Debug for UniqueStates
fn inspect(value: UniqueStates) -> String
impl Debug.inspect
struct Schedule
Section titled “struct Schedule”struct Schedule
opaque — construction surface is private to its defining module
A job a worker enqueues on a fixed interval or at the times a cron
expression names. Its name keys its ticks in the queue, so every
process that runs a schedule gives it the same name, and two schedules
never share one.
Schedule.every
Section titled “Schedule.every”fn every<A>(name: String, interval: PositiveDuration, job: Job<A>, args: A, missed: Missed = Missed.RunLatest) -> Schedule where A: ToJson
type function on Schedule
A schedule that enqueues job for args every interval, at the
whole multiples of it since the Unix epoch, so every process
agrees on the ticks. The interval is a PositiveDuration: a typed
literal, Duration"15m", is checked when the program is compiled,
and a Duration computed at run time goes through
PositiveDuration.try_from.
job = Job.new("sweep", fn _: Int -> Ok(Unit))
sweep = Schedule.every("sweep", Duration"15m", job, 0)
assert Schedule.name(sweep) == "sweep"
at = Instant.from_seconds(60 * 20)
assert Schedule.next_after(sweep, at) == Instant.from_seconds(60 * 30)
Schedule.cron
Section titled “Schedule.cron”fn cron<A>(name: String, cron: Cron, job: Job<A>, args: A, zone: String = "UTC", missed: Missed = Missed.RunLatest) -> Result<Schedule, calendar.Error> where A: ToJson
type function on Schedule
A schedule that enqueues job for args at the times cron
names on a wall clock in zone, an IANA zone name, or the
calendar.Error for a zone the time zone database does not know,
as DateTime.from_instant_in gives. Write ! after it for a zone
written in the program.
job = Job.new("report", fn _: Int -> Ok(Unit))
nightly = Schedule.cron("nightly", Cron"30 2 * * *", job, 0, zone: "America/New_York")!
assert Schedule.cron("nightly", Cron"30 2 * * *", job, 0, zone: "Mars/Olympus") |> Result.is_err()
// On 8 March 2026 New York's clocks jump from 02:00 to 03:00, so
// that day's 02:30 runs at 03:00.
at = DateTime"2026-03-08T00:00:00-05:00[America/New_York]"
assert Schedule.next_after(nightly, Anchored.to_instant(at))
== Anchored.to_instant(DateTime"2026-03-08T03:00:00-04:00[America/New_York]")
Schedule.name
Section titled “Schedule.name”fn name(schedule: Schedule) -> String
type function on Schedule
The name the schedule’s ticks are recorded under.
Schedule.next_after
Section titled “Schedule.next_after”fn next_after(schedule: Schedule, at: Instant) -> Instant
type function on Schedule
The schedule’s first tick after at.
Schedule.inspect
Section titled “Schedule.inspect”impl Debug for Schedule
fn inspect(value: Schedule) -> String
impl Debug.inspect
enum Missed
Section titled “enum Missed”enum Missed RunLatest Skip end
What a scheduler does when it finds that it missed several of a schedule’s ticks, as after a restart.
Variants
-
RunLatestEnqueue one job, for the latest tick missed.
-
SkipEnqueue nothing for the ticks missed, and wait for the next one.
Missed.equal
Section titled “Missed.equal”impl Equatable for Missed
fn equal(a: Missed, b: Missed) -> Bool
impl Equatable.equal
Missed.inspect
Section titled “Missed.inspect”impl Debug for Missed
fn inspect(value: Missed) -> String
impl Debug.inspect
struct Cron
Section titled “struct Cron”struct Cron
opaque — construction surface is private to its defining module
A cron expression: five fields, separated by spaces, for the minute
(0 to 59), the hour (0 to 23), the day of the month (1 to 31), the
month (1 to 12, or JAN to DEC) and the day of the week (0 to 7,
where 0 and 7 are Sunday, or SUN to SAT). A field is *, a value,
a range a-b, any of those with a step (*/15, 1-5/2, 10/20 from
10 to the end), or a list of them separated by commas. Names may be
written in any case. @yearly, @monthly, @weekly, @daily and
@hourly stand for their usual expressions.
When both day fields are given, a day matches either one, as in
Vixie cron: 0 0 1 * MON runs on the first of the month and on each
Monday. A field written */2 counts as given.
Cron"0 3 * * *" is checked when the program is checked, and a
bad expression is a compile error; Cron.parse reads one at run time.
assert "${Cron"*/15 9-17 * * MON-FRI"}" == "*/15 9-17 * * MON-FRI"
assert Cron.parse("0 3 * * *") == Ok(Cron"0 3 * * *")
Cron.to_string
Section titled “Cron.to_string”impl Display for Cron
fn to_string(cron: Cron) -> String
impl Display.to_string
Cron.inspect
Section titled “Cron.inspect”impl Debug for Cron
fn inspect(cron: Cron) -> String
impl Debug.inspect
Cron.parse
Section titled “Cron.parse”fn parse(text: String) -> Result<Cron, CronError>
type function on Cron
Reads a cron expression, or returns what is wrong with it and where.
assert Result.is_ok(Cron.parse("@hourly"))
assert Cron.parse("0 3 * *") == Err(CronError{
offset: 0,
message: "a cron expression has five fields (minute, hour, day of month, month, day of week), not 4",
})
assert Cron.parse("0 24 * * *")
== Err(CronError{offset: 2, message: "hour 24 is out of range 0-23"})
assert Cron.parse("0 0 30 FEB *") == Err(CronError{
offset: 4,
message: "no month chosen has the days of the month chosen",
})
Cron.next_after
Section titled “Cron.next_after”fn next_after(cron: Cron, at: Instant, zone: String = "UTC") -> Result<Instant, calendar.Error>
type function on Cron
The first time after at that cron names on a wall clock in
zone, or the zone’s error when the time zone database does not
know it. See Schedule.cron for the days the clocks change.
zone = "America/New_York"
at = Anchored.to_instant(DateTime"2026-11-01T00:30:00-04:00[America/New_York]")
first = Anchored.to_instant(DateTime"2026-11-01T01:30:00-04:00[America/New_York]")
assert Cron.next_after(Cron"30 1 * * *", at, zone:) == Ok(first)
assert Result.is_err(Cron.next_after(Cron"30 1 * * *", at, zone: "Mars/Base"))
Cron.check
Section titled “Cron.check”impl CheckedLiteral<String, Cron> for Cron
fn check(statics: List<String>) -> Result<Unit, LiteralError>
impl CheckedLiteral.check
A Cron literal is a Cron, checked when the program is checked:
an expression Cron.parse refuses is a compile error, and so is a
slot.
Cron.build
Section titled “Cron.build”impl CheckedLiteral<String, Cron> for Cron
fn build(fragments: List<Fragment<String>>) -> Cron
impl CheckedLiteral.build
Cron.equal
Section titled “Cron.equal”impl Equatable for Cron
fn equal(a: Cron, b: Cron) -> Bool
impl Equatable.equal
struct CronError
Section titled “struct CronError”struct CronError offset: Int message: String end
Why Cron.parse refused an expression: message, about the text
offset code points in.
CronError.to_string
Section titled “CronError.to_string”impl Display for CronError
fn to_string(error: CronError) -> String
impl Display.to_string
CronError.equal
Section titled “CronError.equal”impl Equatable for CronError
fn equal(a: CronError, b: CronError) -> Bool
impl Equatable.equal
CronError.inspect
Section titled “CronError.inspect”impl Debug for CronError
fn inspect(value: CronError) -> String
impl Debug.inspect
struct Rate
Section titled “struct Rate”struct Rate count: PositiveInt per: Duration burst: Int = 1 end
A start rate: at most count jobs start per per, with up to burst
started at once after a quiet spell. It holds back claims in one worker
process; several processes each get the whole rate.
Rate.equal
Section titled “Rate.equal”impl Equatable for Rate
fn equal(a: Rate, b: Rate) -> Bool
impl Equatable.equal
Rate.inspect
Section titled “Rate.inspect”impl Debug for Rate
fn inspect(value: Rate) -> String
impl Debug.inspect
struct Worker
Section titled “struct Worker”struct Worker
opaque — construction surface is private to its defining module
Claims due jobs from a queue and runs them, up to max_running at
once. Build one with Worker.new while boot builds the app value, and
start it in main with Worker.start.
Worker.new
Section titled “Worker.new”fn new(queue: Queue, max_running: PositiveInt, queues: List<String> = ["default"], rate: Maybe<Rate> = None, poll: Duration = Duration.seconds(5), lease: Duration = Duration.seconds(30), shutdown_timeout: Duration = Duration.seconds(10), logger: Logger = Logger.discard()) -> Worker
type function on Worker
A worker for queue’s queues queues, running up to max_running
jobs at once. Call it while boot builds the app value, since it
creates the worker’s two supervisors: one for its claim loop and
the queue’s watcher, one for the jobs.
rateholds back claims to a start rate.pollis how often an idle worker looks for due jobs when the queue does not wake it.leaseis how long a claimed job is held before another worker may rescue it; a running job’s lease is renewed every third of it.shutdown_timeoutis how long running jobs get to finish at shutdown before they are cancelled and handed back.loggergets a line for each job claimed, completed, retried, snoozed or discarded, and for each queue call that failed.
Worker.id
Section titled “Worker.id”fn id(worker: Worker) -> String
type function on Worker
The name this worker claims jobs under.
Worker.start
Section titled “Worker.start”fn start(worker: Worker, handlers: List<Handler>, schedules: List<Schedule> = []) -> Result<Unit, String>
type function on Worker
Starts claiming jobs of the kinds handlers handle, and enqueuing
the jobs of schedules as their ticks come, and returns Ok at
once; the worker runs until the program drains. A schedule’s job
need not be one of handlers: another worker may run it. Two
handlers of one kind are an Err naming the kind, and so are two
schedules of one name, and the worker does not start. A main whose
lists are written in the program writes ! after it.
Worker.inspect
Section titled “Worker.inspect”impl Debug for Worker
fn inspect(value: Worker) -> String
impl Debug.inspect
interface Queue
Section titled “interface Queue”interface Queue fn insert(queue: self, conn: Conn, job: NewJob) -> Result<Enqueued, sql.Error> fn claim(queue: self, worker: String, queues: List<String>, kinds: List<String>, limit: Int, lease: Duration) -> Result<List<Claimed>, sql.Error> fn renew(queue: self, worker: String, job: Claimed, lease: Duration) -> Result<Bool, sql.Error> fn settle(queue: self, worker: String, job: Claimed, how: Settle) -> Result<Bool, sql.Error> fn release(queue: self, worker: String) -> Result<Int, sql.Error> fn rescue(queue: self) -> Result<Int, sql.Error> fn prune(queue: self) -> Result<Int, sql.Error> fn watch(queue: self, supervisor: Supervisor, wake: Sender<Unit>) -> Unit fn last_tick(queue: self, schedule: String) -> Result<Maybe<Instant>, sql.Error> fn record_tick(queue: self, schedule: String, tick: Instant, job: Maybe<NewJob>) -> Result<Maybe<Enqueued>, sql.Error> end
Where jobs are stored. lib/jobs/postgres and std::jobs::testing.Inline
implement it, and std::jobs::testing’s conformance suite checks each
backend.
claim, renew, settle and release act for the worker named
worker, and renew and settle change a job only while that worker
still holds it at that attempt, so a worker that lost a job to a rescue
cannot overwrite the run that took it over.
A stored job with a unique_key holds it while it is available or
running, or, when its unique_across is Kept, while the queue keeps
it. No two stored jobs hold one key.
Queue.insert
Section titled “Queue.insert”fn insert(queue: self, conn: Conn, job: NewJob) -> Result<Enqueued, sql.Error>
interface Queue
Stores job on conn, which may be a transaction: the job exists
only if conn’s writes commit. When a stored job holds job’s
unique key, stores nothing and returns that job’s id with
duplicate: True. A unique enqueue in a transaction still open
holds its key against the others until the transaction ends.
Queue.claim
Section titled “Queue.claim”fn claim(queue: self, worker: String, queues: List<String>, kinds: List<String>, limit: Int, lease: Duration) -> Result<List<Claimed>, sql.Error>
interface Queue
Marks up to limit due jobs of these queues and kinds as running
for worker, held for lease, each with its attempt counted, and
returns them highest priority first, then oldest. Two claims never
return the same job.
Queue.renew
Section titled “Queue.renew”fn renew(queue: self, worker: String, job: Claimed, lease: Duration) -> Result<Bool, sql.Error>
interface Queue
Extends job’s lease. False when worker no longer holds it.
Queue.settle
Section titled “Queue.settle”fn settle(queue: self, worker: String, job: Claimed, how: Settle) -> Result<Bool, sql.Error>
interface Queue
Records how job ended. False when worker no longer holds it.
Queue.release
Section titled “Queue.release”fn release(queue: self, worker: String) -> Result<Int, sql.Error>
interface Queue
Makes every job worker still holds available at once, with this
run’s attempt not counted. Returns how many.
Queue.rescue
Section titled “Queue.rescue”fn rescue(queue: self) -> Result<Int, sql.Error>
interface Queue
Makes every job whose lease ran out available again, or discarded when it used its last attempt. Returns how many.
Queue.prune
Section titled “Queue.prune”fn prune(queue: self) -> Result<Int, sql.Error>
interface Queue
Deletes finished jobs older than the queue keeps them. Returns how many.
Queue.watch
Section titled “Queue.watch”fn watch(queue: self, supervisor: Supervisor, wake: Sender<Unit>) -> Unit
interface Queue
Sends Unit on wake when a job may have become due, from a task
under supervisor. A queue with no way to know does nothing; the
worker also claims on a timer.
Queue.last_tick
Section titled “Queue.last_tick”fn last_tick(queue: self, schedule: String) -> Result<Maybe<Instant>, sql.Error>
interface Queue
The latest instant record_tick recorded for the schedule named
schedule, or None when it recorded none.
Queue.record_tick
Section titled “Queue.record_tick”fn record_tick(queue: self, schedule: String, tick: Instant, job: Maybe<NewJob>) -> Result<Maybe<Enqueued>, sql.Error>
interface Queue
Records tick for the schedule named schedule, unless a later
one is recorded, and stores job as insert does, on the queue’s
own connection and in the same transaction. Returns the insert, or
None without a job.
struct NewJob
Section titled “struct NewJob”struct NewJob queue: String kind: String args: Json run_at: Maybe<Instant> max_attempts: Int priority: Int = 0 unique_key: Maybe<String> = None unique_across: UniqueStates = UniqueStates.Live end
A job as enqueue hands it to a queue to store. unique_key is the
whole key, as Unique.key_for writes it.
NewJob.equal
Section titled “NewJob.equal”impl Equatable for NewJob
fn equal(a: NewJob, b: NewJob) -> Bool
impl Equatable.equal
NewJob.inspect
Section titled “NewJob.inspect”impl Debug for NewJob
fn inspect(value: NewJob) -> String
impl Debug.inspect
struct Claimed
Section titled “struct Claimed”struct Claimed id: JobId queue: String kind: String args: Json attempt: Int max_attempts: Int end
A job a worker claimed: attempt counts this run, from 1.
Claimed.equal
Section titled “Claimed.equal”impl Equatable for Claimed
fn equal(a: Claimed, b: Claimed) -> Bool
impl Equatable.equal
Claimed.inspect
Section titled “Claimed.inspect”impl Debug for Claimed
fn inspect(value: Claimed) -> String
impl Debug.inspect
enum Settle
Section titled “enum Settle”enum Settle Completed Retry { at: Instant, error: String } Snoozed { at: Instant, error: String } Discarded String end
How a claimed job ended, as a worker hands it to its queue.
Variants
-
CompletedThe handler returned
Ok. -
RetryAvailable again at
at; the run counted as an attempt. -
SnoozedAvailable again at
at; the run did not count as an attempt. -
DiscardedNever runs again.
Settle.equal
Section titled “Settle.equal”impl Equatable for Settle
fn equal(a: Settle, b: Settle) -> Bool
impl Equatable.equal
Settle.inspect
Section titled “Settle.inspect”impl Debug for Settle
fn inspect(value: Settle) -> String
impl Debug.inspect
interface Handler
Section titled “interface Handler”interface Handler fn kind(handler: self) -> String fn timeout(handler: self) -> Duration fn run(handler: self, args: Json) -> Result<Unit, Failure> end
A job with its argument type erased: what a worker runs. Every
Job<A> whose A is FromJson is one, and its run discards
arguments that do not decode as A.
Handler.kind
Section titled “Handler.kind”fn kind(handler: self) -> String
interface Handler
Handler.timeout
Section titled “Handler.timeout”fn timeout(handler: self) -> Duration
interface Handler
Handler.run
Section titled “Handler.run”fn run(handler: self, args: Json) -> Result<Unit, Failure>
interface Handler
jobs.wake
Section titled “jobs.wake”fn wake(wake: Sender<Unit>) -> Unit
Wakes a worker whose wake channel is wake without waiting: a wake
already pending absorbs this one. A queue’s watch calls it.
jobs.run_one
Section titled “jobs.run_one”fn run_one(handler: Handler, args: Json) -> Result<Unit, Failure>
Runs handler on args under the handler’s timeout. The handler runs
in a task of its own: a trap returns Retry with the trap’s message at
once, and running past the timeout cancels the task and returns
Retry("timed out").
ok = Job.new("ok", fn _: Int -> Ok(Unit))
assert run_one(ok, Json.Int(1)) == Ok(Unit)
assert run_one(ok, Json.String("x")) is Err(.Discard(_))
jobs.backoff
Section titled “jobs.backoff”fn backoff(attempt: Int) -> Duration
The delay before a job that failed attempt attempt runs again:
attempt^4 + 15 seconds. settle_for adds up to a tenth more.
assert backoff(1) == Duration.seconds(16)
assert backoff(3) == Duration.seconds(96)
jobs.settle_for
Section titled “jobs.settle_for”fn settle_for(claimed: Claimed, answer: Result<Unit, Failure>, now: Instant) -> Settle
How claimed settles when its handler returned answer at now. A
retry waits the backoff plus a share of it, up to a tenth, taken from
now’s nanoseconds, so jobs that failed together do not all come back
at once. A retry of the last attempt is a discard.
claimed = Claimed{
id: JobId(1),
queue: "default",
kind: "k",
args: Json.Null,
attempt: 2,
max_attempts: 3,
}
now = Instant.from_seconds(0)
assert settle_for(claimed, Ok(Unit), now) == Settle.Completed
assert settle_for(claimed, Err(.Retry("x")), now) == Settle.Retry{at: now + backoff(2), error: "x"}
assert settle_for({..claimed, attempt: 3}, Err(.Retry("x")), now) == Settle.Discarded("x")
assert settle_for(claimed, Err(.Discard("bad")), now) == Settle.Discarded("bad")
snooze = Failure.Snooze{message: "busy", after: Duration.seconds(60)}
assert settle_for(claimed, Err(snooze), now) == Settle.Snoozed{
at: now + Duration.seconds(60),
error: "busy",
}
jobs.handlers_by_kind
Section titled “jobs.handlers_by_kind”fn handlers_by_kind(handlers: List<Handler>) -> Result<Map<String, Handler>, String>
The handlers keyed by kind, or an Err naming a kind two of them
share.
a = Job.new("a", fn _: Int -> Ok(Unit))
b = Job.new("b", fn _: Int -> Ok(Unit))
assert Result.map(handlers_by_kind([a, b]), &Map.size(&1)) == Ok(2)
assert handlers_by_kind([a, b, a]) == Err("two handlers for job kind a")
jobs.enqueue_due
Section titled “jobs.enqueue_due”fn enqueue_due(queue: Queue, schedule: Schedule, now: Instant) -> Result<Maybe<Enqueued>, sql.Error>
Enqueues schedule’s job for its latest tick at or before now, when
that tick is after the latest one the queue recorded for it, and
records the tick: what a worker does for each of its schedules at each
tick, and what a test calls to step a schedule through time.
- When the queue recorded no tick, it records
nowand enqueues nothing, so a new schedule starts with its next tick. - When ticks were missed between the one recorded and the latest, the
latest is enqueued, or nothing with
missed: .Skip. - The job is due at its tick and unique by the schedule’s name and the tick, held for as long as the queue keeps it, so a second process that reaches the same tick stores nothing.
import std::sql::testing.Fake
import std::jobs::testing.Inline
queue = Inline.new()
sweep = Schedule.every("sweep", Duration"1h", Job.new("sweep", fn _: Int -> Ok(Unit)), 0)
nine = Instant.from_seconds(9 * 3_600)
assert enqueue_due(queue, sweep, nine) == Ok(None)
// Down from 09:00 to 12:30: one job, for 12:00.
assert enqueue_due(queue, sweep, nine + Duration.minutes(210)) is Ok(Some(_))
assert Iter.map(Inline.jobs(queue), &(&1.run_at)) |> Iter.to_list() == [nine + Duration.hours(3)]