Stackful coroutines in C.
Task & Future coroutines which can pause, waiting for a future.ASleep an example of pausing Tasks using Futures.Generator coroutines used as generators for loops.Coroutine the base coroutine engine.Your code doesn't need to do anything special to be a coroutine. Only standard, or commonly available libraries are needed.
These libraries rely on as much as possible on C's cross-platform comfort zone. C's standard libraries are used as far as possible, but, as threads.h is not usually supported, pthread.h has been used instead.
You will need to build & link the code, coroutine/*.c, as part of your system, and ensure the headers, include/*, are available on your include path.
If your system doesn't have pthread, all the system-specific bits have been collected into cor_platform.c & .h. Replace these with versions which work with your platform.
To run Tasks:
#include "coroutine.h"
#include "task.h"
main(){
Coroutine_StartSystem();
size_t mytask_stack_size = 8192 * sizeof(void *);
void *res = NULL;
bool canceled = Task_Run(mytask_stack_size, maintask, ¶m, &res);
Coroutine_StopSystem();
}
Task_Run runs tasks, switching between them when the current task waits on an Future. maintask() is run as a task. The start function for any task looks like this:
bool mytask(void *param, void **res){
// do your thing here
return canceled;
}
When Task returns from its start function, it returns whether it was canceled. Canceled Tasks are assumed to have not finished what they were doing.
Within your main task, create Tasks and Task_Await() them when you want to wait for their result:
Task task1;
Task_ctor(&task1, mytask_stack_size, adifferenttask, &task1param);
void *result;
bool canceled = Task_Await(&task1, &result);
Task_dtor(&task1);
// use the result
When a task needs to wait for something, and wants to allow other tasks to run, it should use a Future:
Future future;
Future_ctor(&future);
// pass the future to the background-thing-which-might-take-a-while
void *res;
bool canceled = Future_Await(&future, &res);
Future_dtor(&future);
When the background-thing-which-might-take-a-while has a result:
Future_SetResult(future, false, result);
ASleep() needs its own system to be started to work:
ASleep_StartSystem()
Coroutine_StartSystem();
// Run tasks here which may now use ASLeep()
Coroutine_StopSystem();
ASleep_StopSystem();
Note that ASleep_StartSystem() / ASleep_StopSystem() is only needed once per process, whereas Coroutine_StartSystem() / Coroutine_StopSystem() is needed on each thread where coroutines are used.
Sleeping in a task:
bool mytask(void *param, void **result){
..
ASleep(time_to_sleep);
..
}
The coroutine system needs to be started:
Coroutine_StartSystem();
// you can use generators now
Coroutine_StopSystem();
or
void *mygenuser(void *){
// use generators here
}
if (Coroutine_Run(coroutine_stack_size, mygenuser, NULL, NULL)){
// handle the failure
}
Note that you need to start the coroutine system on each thread you want to use them.
You will need a generator function:
void *yield_my_things(void *param){
bool domore = true;
// loop/call functions to find more values to yield, and when you have one:
domore = Generator_Yield(thing);
// .. if domore is false, exit your generator - it is being destructed
// not actually used by generators, but this is a useful convention for bubbling
// the flag out to calling functions.
return (void *)domore;
}
And to use it:
Generator gen;
Generator_ctor(&gen, generator_stack_size, yield_my_things, "..");
void *thing;
while(Generator_Next(&gen, &thing)){
// use thing - a value yielded by your generator
}
Generator_dtor(&gen);
While you can use coroutines directly, it's designed as a system to support more useful patterns, like Async and Generators.
The Coroutines system must be started:
Coroutine_StartSystem();
// use coroutines here
Coroutine_StopSystem();
Your coroutine will need to have a start function:
void *start(void *param){
...
}
When there is no coroutine running, start your 'main' coroutine:
// Coroutine_StartSystem() is optional.
// Wrap with Coroutine_StartSystem() & Coroutine_StopSystem() to get the report
size_t coroutine_stack_size = 8192 * sizeof(void *);
void *result;
if (Coroutine_Run(coroutine_stack_size, comain, param, &result)){
// handle the failure
}
Create other coroutines like this:
Coroutine *cor = Coroutine_New(start);
When you want a Coroutine to run, or to return from a yield:
Coroutine_Continue(cor, value, run_early);
value will be start function's parameter, or the value returned from the yield.
Within the Coroutine, to yield a value:
void *Coroutine_Yield(value, on_yield, void *me);
The on_yield function is called after the coroutine has been 'wait'ed, but before the next coroutine is resumed.
The coroutine system uses the stack divided into smaller stacks for the coroutines. This means you may need to consider whether each coroutine's stack size, is right for each Coroutine, and whether your C stack size is enough for the number of coroutines you might run.
Each of your threads has its own stack - the coroutine system can be run (or not) independantly on each thread. For some special cases, you may want to adjust each of your thread's stack sizes depending on how it is used.
The style is influenced by C++. For example, where possible, a Something *Something_New(a, b, c) and Something_Delete(Something *) will have corresponding Somthing_ctor(Somthing *, a, b, c) and Something_dtor(Something *) to initialise and finalise a Something on the stack, or within another object. Using .._ctor() and .._dtor() will be faster as they avoid the malloc() and free().
Something *oneofthem = Something_New();
// use oneofthem
Something_Delete(oneofthem);
Can be also be done like this, and this will run faster:
Something oneofthem;
Something_ctor(&oneofthem);
// use oneofthem
Something_dtor(&oneofthem);
The exception is Coroutine_New() and Coroutine_Delete(). The returned Coroutine is somewhere on your thread's stack - its memory is managed by the coroutine system, and is allocated and freed quickly.
When you are using coroutines or generators:
void *myfunc(void *){
// your function here
}
Coroutine_StartSystem();
size_t coroutine_stack_size = 8192 * sizeov(void *);
if (Coroutine_Run(coroutine_stack_size, myfunc, (void *)myparam, NULL)){
// handle the failure
}
Coroutine_StopSystem();
You can make many calls to Coroutine_Run() or Task_Run(). Coroutine_Run() ensures the system is started, and that myfunc is called
from inside a Coroutine. In paeticular, if the Coroutine system is running and Coroutine_Run() is called from inside a coroutine, then myfunc is simply called.
The C stack is divided down into smaller stacks. There's one to give some work room between ..StartSystem() and ..Run(), and one for each coroutine. These have guard markers which are checked to see if the stack has overrun. If there is a stack overrun, the system cannot continue - a message is output and the programe exited. There's a number of ways to avoid this issue:
Use less stack. This is, sometimes, the right advice, especially if the startup stack overruns. The expectation is that very little is done between .._StartSystem() and ..Run(). If your situation needs more doing, you can...
increase the stack size for your Coroutine. If your use case is even more demanding, such as if you want 1000s of coroutines (so you need small stack chunks), /and/ some of them can recurse an unknown amount (so you need a deep stack for that coroutine), then you can...
monitor stack headroom, and add another stack chunk if you need to:
In this last case you'll need to add some code at key points:
void *myfunction(void *param){
if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
void *result;
bool fail = Coroutine_Chain(my_stack_size, myfunction, param, &result);
if (fail){
// handle failure
}
return result;
}
// do everything normally
}
More realistically:
struct myfunctionparams {
int a;
char *b;
struct dog *d;
}
void *mychain(void *param){
struct myfunctionparams *myparams = (struct myfunctionparams *)params;
return (void *)myfunction(myparams->a, myparams->b, *myparams->d);
}
int myfunction(int a, char *b, struct dog d){
if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
struct myfunctionparams params = {
a,
b,
&d
};
void *result;
bool fail = Coroutine_Chain(my_stack_size, mychain, ¶ms, &result);
if (fail){
// handle failure
}
return (int)(intptr_t)result;
}
}
And if you want to panic if the C stack overruns:
if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_COROUTINE_STACK){
if (Coroutine_HasCoroutinesInFreePool() ||
(char *)Coroutine_GetCStackTop() - c_stack_end >= MIN_ALLOWED_C_STACK) {
struct myfunctionparams params = {
a,
b,
&d
};
void *result;
if (Coroutine_Chain(my_stack_size, mychain, ¶ms, &result)){
// handle failure
}
return (int)(intptr_t)result;
}
// panic now
}
There's a number of adjustments which you may need to make for your situation. This are, mostly, in cor_platform.h.
There's two options in coroutine.h which you may need to adjust:
Coroutine_StartSystem() and Coroutine_Run()COROUTINE_MINIMUM_STACK_SIZE of stack, then the whole free block is given to your new Coroutine, instead
of being split into two.cor_platform.h focuses on customisation for your particular use case.
#incude <alloca.h>.The pattern for using async is:
bool mymaintask(void *param, void **result){
// do your main task things here, like starting more tasks
}
Coroutine_StartSystem();
void *res = NULL;
bool canceled = Task_Run(mymaintask, NULL, &res);
Coroutine_StopSystem();
To create and wait for a task:
Task task1;
Task_ctor(&task1, asynctask1, &task1param);
void *res = NULL;
bool canceled = Task_Await(&task1, void **res)
Task_dtor(&task1);
or, if you prefer new & delete:
Task *task1 = Task_New(asynctask1, &task1param);
void *res = NULL;
bool canceled = Task_Await(task1, void **res)
Task_Delete(task1);
Inside your task, when there is something to wait for and you want other tasks to run while your task is waiting, you will need a future:
Future future;
Future_ctor(&future);
// keep &future to hand for when the background thing completes
bool canceled = Future_Await(&future, NULL);
Future_dtor(&future);
Future_New() and Future_Delete() are also available if you prefer that style.
Inside the callback when the background thing is complete:
// result is a void *
Future_SetResult(future, result, false);
or, if something went wrong:
// exception is a void *
Future_SetResult(future, exception, true);
Back in the task, you can respond to the future:
... Future_Await has returned
if (canceled){
// exit quickly - you've been canceled
// you could, for example, use the future's result as an exception, or error code here
}
// carry on - the future's result may be an actual result, that's up to you
Future being constructedInitialise a future. When you no longer need it, use Future_dtor().
Allocates and initialises a future, When you no longer need it, use Future_Delete().
Future being destructedDestruct a future previously constructed with Future_ctor().
Future to be destructed and freedDelete (finalise and free) a future previously new'ed with Future_New()
Future whose result is being setcanceled settingvalueSet the result of a future. This has an effect only the first time its done, ie a completed future can't be canceled and a canceled future can't be completed. When an Future has a result, its watchers are called back.
The value of a future might be a result if the future completes (when canceled == false), or could be some sort of exception value if canceled == true. The interpretation of a future's value is up to the user - as far as the async system is concerned, it's only a void *.
canceled value of the Future.Future. This may be NULL.Get the result of a future.
A Future_Watcher is a callback called when a future has a result. The me parameter is the one passed to Future_AddWatcher(). fut is the future which has just got its result.
Future to add a watcher toFuture has a result.me value to pass to watcher when it is called back.Add a watcher (callback) to be called when the future has a result. If the future is already complete, watcher is immediately called. The me value is passed to the watcher as its me parameter. It is assumed that a watcher, identified by the (watcher, me) pair, will only be added once.
Future to remove a watcher fromme value of the watcher to remove.Remove a watcher from a future. It is not an error if no watcher matching (watcher, me) is found - it has probably already been called back.
Future was canceled.Future to wait for.value of the future when it is has a result. May be NULL.The current Task is paused until the Future has a result. Other Tasks are run while this one is waiting.
The entry function to an Task.
param to pass to entry.Initialises an Task. When you have finished with an Task you must finalise it using Task_dtor()
Task tsk;
Task_ctor(&tsk, mytask, myparam);
// tsk will run if you wait for a task or future
Task_Await(&tsk, NULL);
Task_dtor(&tsk);
Task.param to pass to entry.This allocates and initialises a new Task. When you have finished with your task, you must Task_Delete() it.
Task *tsk = Task_New(mytask, myparam);
// tsk will run if you wait for a task or future
Task_Await(tsk, NULL);
Task_Delete(tsk);
Task to destruct.This finalises an Task you ealier initalised with Task_ctor(). It is an error to attempt to destruct a task which is running.
Task tsk;
Task_ctor(&tsk, mytask, myparam);
// use tsk
Task_dtor(&tsk);
Task to delete.This finalises and frees an Task you ealier new'ed with Task_New(). It is an error to attempt to delete a task which is running.
Task *tsk = Task_New(mytask, myparam);
// use tsk
Task_Delete(tsk);
Task to wait for.Task's value when it finishes. This may be NULL.The current Task waits for tsk to finish, and returns the result.
This marks a task as canceled. When that task waits on a future that future will be canceled too, using cancel_value.
Return the future a task is waiting on.
start was canceled.start.start.Runs start as an Task. When start returns all other tasks must have been destructed, using Task_dtor() or Task_Delete().
You must start the ASleep system to use it. This needs to happen per process (whereas Coroutine_StartSystem() needs to happen per-thread). Once you've finished with ASleep you must ASleep_StopSystem().
ASleep_StartSystem();
// Now you can use ASleep() on any thread
ASleep_StopSystem();
Call this to stop the ASleep system.
Sleep for delay seconds. *value will be set to NULL if the sleep is successful, and the cancel_value if the task is canceled.
The pattern for a Generator is:
Generator gen;
Generator_ctor(&gen, generator_stack_size, mygen, ¶m);
void *value;
while(Generator_Next(&gen, &value)){
// use value here
}
// value is now the return value from the Generator
Generator_dtor(&gen);
Or:
Generator *gen = Generator_New(generator_stack_size, mygen, ¶m);
void *value;
while(Generator_Next(gen, &value)){
// use value here
}
Generator_Delete(gen);
Generators yield a series of void *s - what the void *s mean is up to you. Generator_Next() returns a bool to indicate whether the Generator has finished.
The generator_stack_size is the stack amount made available to your generator.
void *mygen(void *param){
bool domore = true;
// The parameter is a pointer to a string of chars
for (char *str = param; *str; ++str) {
// The value yielded is a pointer to a character in the string
domore = Generator_Yield(str);
if (!domore){
break;
}
}
return (void *)domore;
}
The bool returned from Generator_Yield() indicates whether the generator function should yield more values. When it is false the Generator is being finalised - your generator function should close files, and release any other resources it has claimed, before exiting.
Generator to construct.Generator.start.Initialise a Generator. When you no longer need the Generator, use Generator_dtor() to destruct it.
Generator gen;
Generator_ctor(&gen, generator_stack_size, mystart, ¶ms);
// Generator is used
// ... later:
Generator_dtor(&gen);
Generator.start.new a Generator - malloc, and initialise it. When you no longer need the Generator use Generator_dtor to finalise it.
Generator *gen = Generator_New(mystart, ¶ms);
// Generator is used
// ... later:
Generator_Delete(gen);
Generator to destruct.Finalise a Generator. Once a Generator is no longer needed, it must be finalised:
// earlier...
Generator gen;
Generator_ctor(&gen, generator_stack_size, mystart, ¶ms);
// Generator is used
// the Generator is no longer needed
Generator_dtor(&gen);
Generator to delete.Finalise then free() a Generator. Once a newed Generator is no longer needed, it must be deleted:
// earlier...
Generator *gen = Generator_New(mystart, ¶ms);
// Generator is used
// the Generator is no longer needed
Generator_Delete(gen);
true - there is a next value; false - the Generator has finishedGenerator to get the next value from.Get the next value yielded by the Generator.
void *value;
while(Generator_Next(gen, &value)){
// use value here
}
The Generator feeds values to its client using Generator_Yield() - it is these values which Generator_Next() sets, in the example, value to.
When a Generator is finished it returns from start. When you call Generator_Yield() on a finished Generator it returns false and value will be the return value from start.
Generator should do more.Generator's next value.Yield a value from a Generator.
bool domore = Generator_Yield(value);
value is then provided by Generator_Next() as the next value from the generator.
The bool returned by Generator_Yield() says whether more values should be provided by your generator function. true - provide more values if there are any. false - close files, free memory, free up any other resources and return. false is returned when the Generator is being finalised before it has finished, ie the client has exited its for-loop early.
Start the coroutine system on this thread. When you've finished with Coroutine must call Coroutine_StopSystem().
Coroutine_StartSystem();
// prepare
void *result;
if (Coroutine_Run(..., &result)){
// handle the failure
}
// use result
Coroutine_StopSystem();
Coroutine can be started & stopped many times. While Coroutine is started, Coroutine_Run() or Coroutine_RunCoroutine() can be called any number of times.
The total stack allowed for all coroutines running on any thread is the size of the call stack on that thread.
Set the limit of the stack. This is used to determine more accurately whether Coroutine_CanStartCoroutine()
A report from this run of the Coroutine system.
typedef struct Coroutine_Report { unsigned coroutines_created; unsigned coroutines_pool_size; unsigned lowest_headroom; } Coroutine_Report;
Coroutine objects) when the system stopped. This is also the peak number of active coroutines. This will give you an idea of how much stack was needed for your coroutines.Coroutine.Stop the coroutine system on this thread.
Check the integrity of the coroutine system, and printf() any problems.
void *(*)(void *param)
The entry function for a coroutine. The param is the value passed to Coroutine_Continue, and the void * return value can be accessed through the Coroutine object using Coroutine_GetValue().
(returns)
A new Coroutine, or NULL if there was a failure, such as insufficient stack for the new Coroutine.
Create a new Coroutine. The Coroutine system must be started to create a Coroutine. The stack size available to the coroutine will be COROUTINE_STACK_SIZE defined in coroutine.h. When you have finished with your Coroutine, use Coroutine_Delete() to delete it. If there is not enough space for a new Coroutine on your stack, NULL will be returned.
start routine.Run the Coroutine and return when it returns. This is how to start coroutines running in the coroutine system. It is an error for the run coroutine to return before all other coroutines have completed, and the coroutine system must be started to call this.
false if nothing went wrong, or true if there was some type of failure, such as insufficent stack
to create the Coroutine.start().start(value). This may be NULL.start(value) is called from within a coroutine and its value returned in *result.
If this completes without any failure, false is returned, otherwise, typically
because Coroutine_New() returned NULL, true is returned. result may be NULL if you don't
need the resturn value from start().
When the coroutine system is active - you are already running in a coroutine - start(value)
is simply called and its result returned in *result. When the Coroutine system is not running,
Coroutine_Run() starts it, creates a Coroutine and runs that Coroutine to call start(calue)
and return value is returned in *result, then stops the Coroutine system. If you need to force
a new Coroutine to be created, with a particular stack size to call start(value), then use
Coroutine_Chain() instead.
Use Coroutine_Delete() to delete a coroutine when it is no longer needed. It is an error to attempt to delete a coroutine which is running.
false when there's no problem, or true if something wasn't right.cor's yield function.cor early (true), or late (false). Early means before other Coroutines which are waiting
to be called, whereas late means after them.Continue the given Coroutine. value is passed to the coroutine, as param to the start function, or as the return value from Coroutine_Yield. early determines whether the continued coroutine will be run next, or after all the other, currently runnable, coroutines. If the Coroutine is already runnable, nothing is done, and false is returned. If the Coroutine is free, or complete, nothing is done and true is returned to show there was a problem.
on_yield.Yield value from the current coroutine; this coroutine is moved to the list of coroutines waiting to be continued. The next runable coroutine is run - either by its start routine being called with value as its param, or by valuebeing returned from its Coroutine_Yield().
Return the Coroutine's value - the value last yielded, or returned by its start routine.
Return whihc coroutine is currently running, ie the caller's Coroutine.
cor is running - it's the active coroutine or waiting to be continued.Return whether the given coroutine is still running - it may be running, ready to run, or waiting to be continued, but won't have returned from its start function.
cor is complete, ie has returned from start()./Return whether the given coroutine is complete - is has returned from its start function.
Return the headroom available in the current coroutine's stack. This can be used to detect when your coroutine is nearing its stack limit, and then use Coroutine_Chain() to continue in a new chunk of coroutine stack.
Return whether the coroutine system can start a new coroutine. This check can only be done with the coroutine system active (currently running
a coroutine). If there's a free coroutine, or enough space on the stack for a new one, then this will return true. To set the limit of the
stack use Coroutine_SetStackLimit()
Find out where this coroutine's guard patterns end. This is intended as a part of the tools to measure how much stack something is using:
Coroutine_ClearStackForHWM();
char *before = (char *)Coroutine_GetStackHWM();
// do the thing you want to measure here
char *after = (char *)Coroutine_GetStackHWM();
intptr_t amount_used = before - after;
Fill the unused stack in this coroutine with a guard pattern. This is intended as a part of the tools to measure how much stack something is using:
Coroutine_ClearStackForHWM();
char *before = (char *)Coroutine_GetStackHWM();
// do the thing you want to measure here
char *after = (char *)Coroutine_GetStackHWM();
intptr_t amount_used = before - after;
Return an address which is near to the top of used C stack.
false - start(value) was run; true - there was a problem.start()start(value). This may be NULL.Run start with value on a new coroutine, and return its return value in *result. result
may be NULL. Coroutine_Run() returns false if nothing fails, and true if something went wrong,
usually when Coroutine_New() ran out of stack. stack_size is the amount of stack made available
to the chained Coroutine.
It is expected that Coroutine_Chain() will be used when your coroutine is running short
of stack - it is not an alternative to Coroutine_Run().
Do not use this function in production code
This prints the current state of the Coroutine system. It is used for development, and is not part of the official interface.