Skip to content

Concurrency

In Nomi, every task has an owner. The owner makes sure the task finishes, is cancelled, or has its failure reported. You can’t start a task that nothing owns.

There are two kinds of owner. Pick by asking how long the work should live:

  • If you need the result here, use a concurrent … end block and Task.spawn(fn -> …).
  • If the work should keep running after this call returns, use Supervisor.spawn(supervisor, fn -> …), which Background Work covers.

One function can use both. A request handler might compute its response in a concurrent block, then give an audit write to a supervisor. The response goes back without waiting for the audit.

Each task runs as a Go goroutine, so tasks are cheap to start. Channels work much like Go channels. When you limit how many tasks run at once, the reason is usually what the work talks to: a database pool, an API’s rate limit, a single worker, or the number of CPU cores.

Start with a block:

import std::tasks.Task

fn double(n: Int) -> Int
    n * 2
end

fn main()
    total = concurrent
        a = Task.spawn(fn -> double(10))
        b = Task.spawn(fn -> double(20))
        c = Task.spawn(fn -> double(30))
        Task.await(a) + Task.await(b) + Task.await(c)
    end

    dbg total
end
  • concurrent … end owns every task started inside it. Its value is its last expression.
  • Task.spawn(fn -> expr) starts the lambda as a task and returns a handle to it, a Task<T>, right away. A function that takes no arguments can be passed by name: Task.spawn(refresh).
  • Task.await(task) waits for the task and returns its result.

The compiler checks two rules. You must use each task’s handle before the block ends, and the handle can’t leave the block. Task.await is one way to use a handle. Task.outcome, later on this page, is the other.

Task.spawn doesn’t name an owner, because the enclosing concurrent block is the only one it could be. Supervisors (Background Work) are owners with names, and they live longer.

Fine print

  • A Task.spawn that can run outside every concurrent block is the error spawn outside concurrent block. A helper that spawns is fine when it is called from inside one.
  • A handle that is never awaited is Task bound to 't' is never awaited, and one that leaves its block, returned or sent on a channel, is Task<T> cannot escape its enclosing concurrent block.
  • return, ?, break and continue in the block leave the block, as in a lambda, not the function around it.
  • A task body is a lambda, so it can’t assert. An assert in the block’s own body needs the block’s value to be a Result<_, AssertionFailure>.

Spec: concurrent and tasks, Static guarantees.

A Channel passes values between tasks. It has two halves. Producers send on the Sender. A consumer receives on the Receiver until the channel is closed.

import
    std::channels.{Channel, Receiver, Sender}
    std::io
    std::tasks.Task
end

fn produce(out: Sender<Int>, n: Int) -> Unit
    case Sender.send(out, n)
        Ok(_)  -> Unit
        Err(_) -> io.print("send failed")
    end
end

fn consume(inbox: Receiver<Int>) -> Int
    Iter.loop(0, fn sum ->
        case Receiver.receive(inbox)
            Some(v) -> sum + v
            None    -> break sum
        end
    end)
end

fn main()
    ch = Channel.buffered<Int>(4)

    total = concurrent
        p1 = Task.spawn(fn -> produce(ch.sender, 10))
        p2 = Task.spawn(fn -> produce(ch.sender, 20))
        p3 = Task.spawn(fn -> produce(ch.sender, 30))
        cons = Task.spawn(fn -> consume(ch.receiver))

        // Wait for the producers, then close so the consumer's loop ends on None.
        Task.await(p1)
        Task.await(p2)
        Task.await(p3)
        Sender.close(ch.sender)
        Task.await(cons)
    end

    dbg total
end

Receiver.receive waits until a value arrives and returns Some(v). Once the channel is closed and empty, it returns None instead. That None ends the consumer’s loop, so the example closes the sender only after all three producers have finished.

consume gets only the Receiver<Int>, so it can’t send or close the channel. Closing is done with Sender.close. Closing means “nothing more is coming”, and only the sending side can promise that.

There are two ways to make a channel:

  • Channel.buffered<T>(n) holds up to n values. A producer can get n values ahead of its consumer before send waits. Choose n by how far ahead is useful and how much you are willing to hold in memory.
  • Channel.unbuffered<T>() holds nothing. Each send waits until another task receives the value.

With an unbuffered channel, the producer runs only as fast as the consumer. That makes deadlock easy: a task that sends and then receives on the same channel waits forever. Use buffered by default. Use unbuffered when you want the sender to wait for the hand-off.

Channel.select waits on several channels, or a timer, and runs the handler of whichever arm finishes first. Each arm’s handler turns its own outcome into the one result:

import
    std::channels.{Arm, Channel, Sender}
    std::duration.Duration
    std::tasks.Task
end

fn main()
    replies = Channel.buffered<String>(1)
    failures = Channel.buffered<Int>(1)

    answer = concurrent
        worker = Task.spawn(fn -> Sender.send(failures.sender, 503))
        _ = Task.await(worker)

        Channel.select([
            Arm.receive(replies.receiver, fn reply -> Maybe.with_default(reply, "closed")),
            Arm.receive(failures.receiver, fn code -> "failed: ${Maybe.with_default(code, 0)}"),
            Arm.after(Duration.seconds(1), fn -> "timed out"),
        ])
    end

    dbg answer
end

When several arms are ready at once, the first one listed wins. For one channel and a timeout, Receiver.receive_within returns Ok(Some(v)), Ok(None) when closed, or Err(TimedOut).

Fine print

  • Sender.send on a closed channel returns Err(ChannelClosed). Receiver.receive keeps returning values already sent after the close, and None once they are gone.
  • There is no Receiver.close. Only the sending side closes.
  • The capacity is a PositiveInt, so Channel.buffered(0) does not compile. A channel with no buffer is Channel.unbuffered().
  • A channel belongs to no block. It can be made in one block and used in a later one.
  • A send, receive or select that is waiting when its task is cancelled stops like any other wait.
  • Timers in select run on the clock in force, so under clock Clock.Virtual a test of a timeout takes no real time.

Spec: Channels.

Suppose one task in a block returns an Err. When a ? sees that Err from Task.await, the block stops. It cancels the other tasks, waits for them to stop, and returns the error:

import
    std::duration.Duration
    std::tasks.Task
    std::timer
end

fn short_fail() -> Result<Int, String>
    timer.sleep(Duration.milliseconds(10))
    Err("boom")
end

fn long_sleep() -> Result<Int, String>
    timer.sleep(Duration.seconds(5))
    Ok(0)
end

fn main()
    outcome = concurrent
        short = Task.spawn(short_fail)
        long = Task.spawn(long_sleep)
        n = Task.await(short)?
        _ = Task.await(long)?
        Ok(n)
    end

    dbg outcome
end

The block returns after about 10 ms. The five-second sleep is cancelled; it doesn’t keep running after the error.

The order of the awaits matters. The block only notices a problem when you await the task that has it. It doesn’t watch all its tasks in the background. Swap the two awaits above:

_ = Task.await(long)? // blocks the full five seconds

n = Task.await(short)? // the Err has been sitting here the whole time

Now the program takes five seconds. It still returns Err("boom"), but only after waiting for the slow task.

A runtime failure, such as a ! on None, works the same way. You find out about it when you await that task. So await first the task you most want to hear from.

Task.await_all is different. It waits for a whole batch, so a failure in any task of the batch shows up right away.

Fine print

  • A cancelled task doesn’t see the cancellation as a value. It stops at its next safe point, running its deferred calls on the way out.
  • Safe points are every function call, every loop round and every operation that waits, so pure computation stops too.
  • A single call into Go isn’t interrupted. The task stops at the first safe point after the call returns.
  • A task that returns Err completed: its outcome is Completed(Err(e)). Failed means it trapped.

Spec: Errors and cancellation, Safe points.

You could start one task per item:

// Ten thousand items, ten thousand simultaneous database queries.
concurrent
    handles =
        ids
        |> Iter.map(fn id -> Task.spawn(fn -> fetch_user(id)))
        |> Iter.to_list()

    handles
    |> Iter.map(Task.await)
    |> Iter.to_list()
end

The tasks are cheap, but the queries are not. Ten thousand database queries at once will use up the connection pool.

Task.spawn_all starts a batch with a limit on how many run at once. Task.await_all collects the results in order:

// FILE: main.nomi
import
    std::fs
    std::io
    std::tasks.Task
end

// How many words a file holds.
fn words(path: String) -> Result<Int, io.Error>
    path
    |> fs.read_text()?
    |> String.words()
    |> Iter.count()
    |> Ok()
end

fn main()
    counts = concurrent
        ["b.txt", "a.txt", "missing.txt"]
        |> Task.spawn_all(words, max_running: 8)
        |> Task.await_all()
    end

    dbg counts
end

// FILE: a.txt
the cat sat
on the mat

// FILE: b.txt
a dog

At most eight run at a time, however many inputs you pass. With ten thousand ids, the rest wait their turn instead of opening ten thousand connections.

Results come back in the order the tasks were started, not the order they finished: b.txt, a.txt, then missing.txt, the order of the input. words returns a Result, so the file that isn’t there is an Err in the list, and the other two still finish.

max_running is required, and it is a PositiveInt: max_running: 0 does not compile. For work that only uses the CPU, the number of cores is a good choice. For most other work, the limit protects something the tasks talk to: a pool, a queue, an API, or a single worker. If you don’t know, start low; a limit that is too low is only slower, and you can measure it and raise it.

The limit goes on the spawn, because by the time you await, the work is already running. Each batch has its own limit, because one block may talk to more than one service:

concurrent
    users =
        ids
        |> Task.spawn_all(fetch_user, max_running: 8)
        |> Task.await_all()

    users
    |> Task.spawn_all(send_digest, max_running: 50)
    |> Task.await_all()
end

This runs eight database queries at a time, then fifty API calls at a time. One limit for the whole block couldn’t do both.

A function that returns a Result gives a list of Results, as words did. A task that returns Err still finished normally, so it doesn’t stop the batch. You decide what to do when some succeed and some don’t:

outcomes = concurrent
    ids
    |> Task.spawn_all(fetch_user, max_running: 8)
    |> Task.await_all()
end

failures = Iter.filter(outcomes, Result.is_err)

The stopping in Errors cancel siblings happens because a ? sees an Err, or because a task breaks. Here, nothing uses ? on the results, so each Err is a value in the list.

Task.cancel stops one task and leaves the others running. The task stops at its next call, loop round or wait. Task.outcome tells you whether a task finished or was cancelled:

import
    std::duration.Duration
    std::tasks.Task
    std::timer
end

fn serve_conn() -> Int
    // Would take half a minute if left alone.
    timer.sleep(Duration.seconds(30))
    1
end

fn main()
    outcome = concurrent
        conn = Task.spawn(serve_conn)
        Task.cancel(conn)
        // `Task.await` on a cancelled task would cancel this block too;
        // `Task.outcome` reports it instead.
        Task.outcome(conn)
    end

    dbg outcome
end

This returns immediately, even though the task was in a 30-second sleep. Cancelling interrupts a task that is waiting.

Cancelling a task also cancels everything it started. If serve_conn ran its read and write loops in its own concurrent block, cancelling the task stops those loops too.

When several tasks race, you can cancel the ones that lose. You still have to await or inspect each handle, but a cancelled task returns quickly, so the block doesn’t wait for the slowest one.

Inside a task, the app’s context field, Config.context in these examples, is that task’s own context, not the one boot built. This has two effects.

First, you don’t need to check whether your task was cancelled. The runtime checks for you at every loop iteration, every function call, and every operation that waits, and stops the task there. Your own check would run later than that anyway. So you write a worker loop as if it runs forever:

fn outbox_worker()
    Iter.loop(fn ->
        drain_pending(Config.outbox)
        timer.sleep(Duration.seconds(1))
    end)
end

Cancelling it interrupts the sleep, ends the loop, and makes Task.outcome return Cancelled. Put cleanup in a defer, which runs however the work ends.

A Context holds a deadline and any typed values you add to it. It has no “cancelled” flag. Context.deadline_remaining returns Some of a zero Duration once the deadline has passed, but that only tells you about the clock. A task never reads whether it was cancelled; the runtime stops it.

Second, you can give the work a deadline by deriving a new context:

fn send_welcome(user: User) -> Unit
    // The deadline covers `deliver` and everything it calls.
    with Config.context = Context.with_timeout(Config.context, Duration.seconds(30))
    deliver(user)
end

A task in a concurrent block gets the caller’s deadline, because the caller is waiting for it. The task stops at whichever comes first: the deadline or Task.cancel.

A deadline can only get shorter: a context made inside it can’t extend it.

When a deadline passes, you don’t get a value back. The work stops at the next point where the runtime checks, and the rest of the code under that deadline is skipped. In main, the program ends. In a task, the task ends.

So this attempt to try for a while and then fall back does not work:

fn attempt() -> String
    // Broken: the deadline covers the `case` too, so the `case` never runs.
    with Config.context = Context.with_timeout(Config.context, Duration.seconds(1))

    outcome = concurrent
        t = Task.spawn(slow)
        Task.outcome(t)
    end

    case outcome
        .Completed(v) -> "got ${v}"
        .Cancelled    -> "timed out, falling back"
        .Failed(_)    -> "broke"
    end
end

Instead, set the deadline inside the task, and check the result from outside it:

import
    std::duration.Duration
    std::tasks.Task
    std::timer
end

struct Config
    context: Context
end

fn boot() -> Result<Config, String>
    Ok(Config{context: Context.root()})
end

fn slow() -> Int
    timer.sleep(Duration.minutes(5))
    1
end

// The deadline covers `slow` and nothing else.
fn slow_bounded() -> Int
    with Config.context = Context.with_timeout(Config.context, Duration.milliseconds(50))
    slow()
end

fn main()
    outcome = concurrent
        t = Task.spawn(slow_bounded)
        Task.outcome(t)
    end

    // Reached because `Task.outcome` sits outside the deadline.
    result = case outcome
        .Completed(v) -> "got ${v}"
        .Cancelled    -> "timed out, falling back"
        .Failed(_)    -> "broke"
    end

    dbg result
end

The five-minute sleep ends after 50 milliseconds, and the caller gets an ordinary value to case on.

So far, tasks ended because they finished or because you stopped them. A task can also break: a bug, a bad FFI call, an index past the end.

A task ends in one of three ways:

OutcomeMeaning
CompletedThe function returned normally.
CancelledSomething stopped it: Task.cancel, a deadline, or shutdown.
FailedIt broke.

Task.await returns the value of a completed task. If the task was cancelled or failed, await passes that on to the code that awaited it, as if it had happened there.

A task that returns Err completed; ? passes the Err on, as in Errors cancel siblings. A failed or cancelled task is passed on by Task.await itself, up through owners until Task.outcome, a supervisor or the top of the program stops it. Nothing at the call site shows this. Task.await on a Task<Int> returns an Int. A failed task is like an index past the end, not like an Err your code chose to return.

To look at what happened instead of having it passed on, ask for the outcome:

case Task.outcome(worker)
    .Completed(v) -> io.print("done: ${v}")
    .Cancelled    -> io.print("we stopped it")
    .Failed(e)    -> io.print("broke: ${e}")
end

Task.await is Task.outcome with the Cancelled and Failed cases passed on for you. Use outcome when you cancelled the task yourself, or when you want to check on background work.

So programs can still crash, but an owner always sees the failure:

  • A failure under a supervisor doesn’t stop the program. The supervisor reports it, restarts the task, or gives up, as you told it to (Background Work).
  • A failure that nothing catches ends the program with a report, not a crash dump.
  • You can catch it at any owner along the way with Task.outcome.

Nomi values are immutable, so a task that dies halfway can’t leave in-memory data half written. Writes it made outside the program, to a file or a database, stay made.

Calling Go is next. It calls functions from Go packages.