0 branches 0 tags
27
Documentation experiment
jonroach 19 Sep 2025 08:11
2 contributors
514 lines18.2 KB
Markdown Text Raw Blame
  • Delete
1Stackful coroutines in C.
2
3* `Async` coroutines which can pause, waiting for a future.
4* `Generator` coroutines used as generators for loops.
5* `Coroutine` the coroutine engine used by `Async` and `Generator`.
6
7Your code doesn't need to do anything special to be a coroutine, and only standard, or commonly available libraries are needed.
8
9## Prerequisites
10
11The goal was to make a system which can be used 'out of the box'. These libraries rely on as much as possible on C's cross-platform comfort zone. C's standard libraries are used as far as possible, but, as `threads.h` is not usually supported, `pthread.h` has been used instead.
12
13You will need to build & link the code as part of your system - `coroutine/async.c`, `coroutine/generator.c` and `coroutine/coroutine.c` - ensure the headers, `include/*`, are available on your include path.
14
15## Quick Start
16
17### Async
18
19To run an Async program:
20
21 #include "async.h"
22 main(){
23 Async_StartSystem();
24 void *res = NULL;
25 bool canceled = Async_Run(asyncmain, &param, &res);
26 Async_StopSystem();
27 }
28
29Async runs tasks, switching between them when the current task waits on an `Async_Future`. `asyncmain()` is run as a task. The start function for any task looks like this:
30
31 bool mytask(void *param, void **res){
32
33 // do your thing here
34
35 return canceled;
36 }
37
38When `Task` returns from its start function, it returns whether it was canceled. Canceled `Task`s are assumed to have not finished what they were doing.
39
40Within your async task, create `Async_Task`s and `Async_Task_Await()` them when you want to wait for their result:
41
42 Async_Task task1;
43 Async_Task_ctor(&task1, adifferenttask, &task1param);
44
45 void *result;
46 bool canceled = Async_Task_Await(&task1, &result);
47
48 Async_Task_dtor(&task1);
49
50 // use the result
51
52When a task needs to wait for something, and wants to allow other tasks to run, it should use a `Future`:
53
54 Async_Future future;
55 Async_Future_ctor(&future);
56
57 // pass the future to the background-thing-which-might-take-a-while
58
59 void *res;
60 bool canceled = Async_Future_Await(&future, &res);
61
62 Async_Future_dtor(&future);
63
64When the background-thing-which-might-take-a-while has a result:
65
66 Async_Future_SetResult(future, false, result);
67
68### Generators
69
70The coroutine system needs to be started, either through `Async_StartSystem()`, or directly with `Coroutine_StartSystem()` if you don't want to do async things.
71
72You will need a generator function:
73
74 void *yield_my_things(void *param){
75 bool domore = true;
76
77 // loop/call functions to find more values to yield, and when you have one:
78 domore = Generator_Yield(thing);
79 // .. if domore is false, exit your generator - it is being destructed
80
81 // not actually used by generators, but this is a useful convention for bubbling
82 // the flag out to calling functions.
83 return (void *)domore;
84 }
85
86And to use it:
87
88 Generator gen;
89 Generator_ctor(&gen, yield_my_things, "..");
90 void *thing;
91 while(Generator_Next(&gen, &thing)){
92 // use thing - a value yielded by your generator
93 }
94 Generator_dtor(&gen);
95
96### Coroutines
97
98While you can use coroutines directly, it's designed as a system to support more useful patterns, like `Async` and `Generators`.
99
100The Coroutines system must be started:
101
102 Coroutine_StartSystem();
103 // use coroutines here
104 Coroutine_StopSystem();
105
106Your coroutine will need to have a start function:
107
108 void *start(void *param){
109 ...
110 }
111
112When there is no coroutine running, start your 'main' coroutine:
113
114 void *result = Coroutine_Run(comain, param);
115
116Create other coroutines like this:
117
118 Coroutine *cor = Coroutine_New(start);
119
120When you want a Coroutine to run, or to return from a yield:
121
122 Coroutine_Continue(cor, value, run_early);
123
124`value` will be start function's parameter, or the value returned from the yield.
125
126Within the Coroutine, to yield a value:
127
128 void *Coroutine_Yield(value, on_yield, void *me);
129
130The on_yield function is called after the coroutine has been 'wait'ed, but before the next coroutine is resumed.
131
132## How it Works
133
134The coroutine system uses the stack, divided into smaller stacks, for the coroutines. This means you may need to consider whether the coroutine stack size, set by `COROUTINE_STARTUP_STACK_SIZE`, is right for your coroutines, and whether your stack size is enough for the number of coroutines you might run concurrently.
135
136As each of your thread has its own stack - the coroutine system can be run (or not) independantly on each of your threads. For some special cases, you may want to adjust each of your thread's stack sizes depending on how it is used.
137
138## Style
139
140The style is influenced by C++. For example, where possible, a `Something *Something_New(a, b, c)` and `Something_Delete(Something *)`, where a `Something` is `malloc`ed, will have corresponding `Somthing_ctor(Somthing *, a, b, c)` and `Something_dtor(Something *)` to initialise and finalise a `Something` on the stack, or within another object. Using `.._ctor()` and `.._dtor()` will be faster as they avoid the `malloc()` and `free()`.
141
142 Something *oneofthem = Something_New();
143 // use oneofthem
144 Something_Delete(oneofthem);
145
146Can be also be done like this, and this will run faster:
147
148 Something oneofthem;
149 Something_ctor(&oneofthem);
150 // use oneofthem
151 Something_dtor(&oneofthem);
152
153The exception is `Coroutine_New()` and `Coroutine_Delete()`. The returned `Coroutine` is somewhere on your thread's stack - its memory is managed by the coroutine system, and is allocated and freed quickly.
154
155## Usage
156
157When you are using coroutines or generators:
158
159 void *myfunc(void *){
160 // your function here
161 }
162
163 Coroutine_StartSystem();
164 Coroutine_Run(myfunc, (void *)myparam);
165 Coroutine_StopSystem();
166
167If you also use async, then:
168
169 bool myfunc(void *myparam, void **res){
170 // your async function here
171 }
172
173 Async_StartSystem();
174 void *res = NULL;
175 bool canceled = Async_Run(myfunc, myparam, &res);
176 Async_StopSystem();
177
178While the system is started, you can make many calls to `Coroutine_Run()` or `Async_Run()`. A running system is thread local - each thread you want to use coroutines on will need to be `Coroutine_StartSystem()`ed or `Async_StartSystem()`ed.
179
180## Stack Overruns
181
182The C stack is divided down into smaller stacks. There's one to give some work room between `..StartSystem()` and `..Run()`, and one for each coroutine. These have guard markers which are checked to see if the stack has overrun. If there is a stack overrun, the system cannot continue - a message is output and the programe exited. There's a number of ways to avoid this issue:
183
184* Use less stack. This is, sometimes, the right advice, especially if the startup stack overrins. The expectation is that very little is done between `.._StartSystem()` and `..Run()`. If your situation needs more doing, you can...
185
186* increase the stack size. Adjust `COROUTINE_STACK_SIZE` (for startup) or `COROUTINE_STARTUP_STACK_SIZE` (for coroutines) as appropriate. If your use case is even more demanding, such as if you want 1000s of coroutines (so you need small stack chunks), /and/ some of them can recurse an unknown amount (so you need a deep stack for that coroutine), then you can...
187
188* monitor stack headroom, and add another stack chunk if you need to:
189
190In this last case you'll need to add some code at key points:
191
192 void *myfunction(void *param){
193 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
194 return Coroutine_Chain(myfunction, param);
195 }
196 // do everything normally
197 }
198
199More realistically:
200
201 struct myfunctionparams {
202 int a;
203 char *b;
204 struct dog *d;
205 }
206
207 void *mychain(void *param){
208 struct myfunctionparams *myparams = (struct myfunctionparams *)params;
209 return (void *)myfunction(myparams->a, myparams->b, *myparams->d);
210 }
211
212 int myfunction(int a, char *b, struct dog d){
213 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
214 struct myfunctionparams params = {
215 a,
216 b,
217 &d
218 };
219 return (int)Coroutine_Chain(mychain, &params);
220 }
221 }
222
223# API
224
225## Async
226
227The pattern for using async is:
228
229 void *myasyncmaintask(void *param){
230 // do your main async task things here, like starting more tasks
231 }
232
233 Async_StartSystem();
234 void *res = NULL;
235 bool canceled = Async_Run(myasyncmaintask, NULL, &res);
236 Async_StopSystem();
237
238To create and wait for an async task:
239
240 Async_Task task1;
241 Async_Task_ctor(&task1, asynctask1, &task1param);
242 void *res = NULL;
243 bool canceled = Async_Task_Await(&task1, void **res)
244 Async_Task_dtor(&task1);
245
246or, if you prefer new & delete:
247
248 Async_Task *task1 = Async_Task_New(asynctask1, &task1param);
249 void *res = NULL;
250 bool canceled = Async_Task_Await(task1, void **res)
251 Async_Task_Delete(task1);
252
253Inside your task, when there is something to wait for and you want other tasks to run while your task is waiting, you will need a future:
254
255 Async_Future future;
256 Async_Future_ctor(&future);
257
258 // keep &future to hand for when the background thing completes
259 bool canceled = Async_Future_Await(&future, NULL);
260
261 Async_Future_dtor(&future);
262
263`Async_Future_New()` and `Async_Future_Delete()` are also available if you prefer that style.
264
265Inside the callback when the background thing is complete:
266
267 // result is a void *
268 Async_Future_SetResult(future, result, false);
269
270or, if something went wrong:
271
272 // exception is a void *
273 Async_Future_SetResult(future, exception, true);
274
275Back in the task, you can respond to the future:
276
277 ... Async_Future_Await has returned
278 if (canceled){
279 // exit quickly - you've been canceled
280 // you could, for example, use the future's result as an exception, or error code here
281 }
282 // carry on - the future's result may be an actual result, that's up to you
283
284
285### void Async_Future_ctor(Async_Future *fut)
286
287Initialise a future. When you no longer need it, use `Async_Future_dtor()`.
288
289### Async_Future *Async_Future_New()
290
291Allocate and initialise a future, When you no longer need it, use `Async_Future_Delete()`.
292
293Returns the new `Async_Future`
294
295#### `void Async_Future_dtor(Async_Future *fut)`
296
297fut
298: The `Async_Future` being destructed
299
300Destruct a future previously constructed with `Async_Future_ctor()`.
301
302### void Async_Future_Delete(Async_Future *fut)
303
304Delete (finalise and free) a future previously new'ed with `Async_Future_New()`
305
306### void Async_Future_SetResult(Async_Future *fut, bool canceled, void *value)
307
308Set the result of a future. This has an effect only the first time its done, ie a completed future can't be canceled and a canceled future can;t be completed.
309
310### bool Async_Future_GetResult(Async_Future *fut, void **res)
311
312Get the result of a future. The return value is whether it was canceled. `res` may be `NULL`.
313
314### typedef void (*Future_Watcher)(void *me, Async_Future *fut)
315
316A `Future_Watcher` is a callback called when a future is done. The `me` parameter is the one passed to `Async_Future_AddWatcher()`. `fut` is the future which has just completed.
317
318### void Async_Future_AddWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
319
320Add a watcher (callback) to be called when the future is done. If the future is already complete, `watcher` is immediately called. The `me` value is passed to the watcher as its `me` parameter.
321
322### void Async_Future_RemoveWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
323
324Remove a watcher from a future.
325
326### bool Async_Future_Await(Async_Future *fut, void **res)
327
328
329
330## Generator
331
332The pattern for a `Generator` is:
333
334#### A loop which uses the `Generator`
335
336 Generator gen;
337 Generator_ctor(&gen, mygen, &param);
338
339 void *value;
340 while(Generator_Next(&gen, &value)){
341 // use value here
342 }
343 // value is now the return value from the Generator
344
345 Generator_dtor(&gen);
346
347Or:
348
349 Generator *gen = Generator_New(mygen, &param);
350
351 void *value;
352 while(Generator_Next(gen, &value)){
353 // use value here
354 }
355
356 Generator_Delete(gen);
357
358`Generator`s yield a series of `void *`s - what the `void *`s mean is up to you. `Generator_Next()` returns a `bool` to indicate whether the `Generator` has finished.
359
360#### A generator function
361
362 void *mygen(void *param){
363 bool domore = true;
364 // The parameter is a pointer to a string of chars
365 for (char *str = param; *str; ++str) {
366 // The value yielded is a pointer to a character in the string
367 domore = Generator_Yield(str);
368 if (!domore){
369 break;
370 }
371 }
372
373 return (void *)domore;
374 }
375
376The `bool` returned from `Generator_Yield()` indicates whether the generator function should yield more values. When it is `false` the `Generator` is being finalised - your generator function should close files, and release any other resources it has claimed, before exiting.
377
378### void Generator_ctor(Generator *gen, void *(*start)(void *), void *param)
379
380Initialise a `Generator`
381
382 Generator gen;
383 Generator_ctor(&gen, mystart, &params);
384
385 // Generator is used
386
387 // ... later:
388 Generator_dtor(&gen);
389
390### Generator *Generator_New(void *(*)(void *), void *)
391
392`new` a `Generator` - malloc it and initialise it.
393
394 Generator *gen = Generator_New(mystart, &params);
395
396 // Generator is used
397
398 // ... later:
399 Generator_Delete(gen);
400
401### void Generator_dtor(Generator *gen)
402
403Finalise a `Generator`. Once a `Generator` is no longer needed, it must be finalised:
404
405 // earlier...
406 Generator gen;
407 Generator_ctor(&gen, mystart, &params);
408
409 // Generator is used
410
411 // the Generator is no longer needed
412 Generator_dtor(&gen);
413
414
415### void Generator_Delete(Generator *)
416
417Finalise then `free()` a `Generator`. Once a `new`ed `Generator` is no longer needed, it must be deleted:
418
419 // earlier...
420 Generator *gen = Generator_New(mystart, &params);
421
422 // Generator is used
423
424 // the Generator is no longer needed
425 Generator_Delete(gen);
426
427
428### bool Generator_Next(Generator *, void **value)
429
430Get the next value yielded by the `Generator`.
431
432 void *value;
433 while(Generator_Next(gen, &value)){
434 // use value here
435 }
436
437When `true` is returned, `value` is the value yielded by the `Generator`. When `false` is returned, `value` is the value returned when the `Generator` returned - the `Generator` has finished.
438
439### bool Generator_Yield(void *)
440
441Yield a value from a `Generator` function.
442
443 bool domore = Generator_Yield(value);
444
445`value` is then provided by `Generator_Next()` as the next value from the generator. The `bool` returned by `Generator_Yield()` indicates whether more values should be provided by your generator function. When `true` provide more values if possible. When `false` close files, free memory, free up any other resources and `return`. `false` is returned when the `Generator` is being finalised.
446
447## Coroutine
448
449### void Coroutine_StartSystem();
450
451Start the coroutine system on this thread. When you've finished with `Coroutine` call `Coroutine_Stop()`. `Coroutine` can be started & stopped many times on one thread. The total stack allowed for all coroutines running on a thread is the size of the call stack on that thread.
452
453### void Coroutine_StopSystem();
454
455Stop the coroutine system on this thread.
456
457### Coroutine_Start
458
459 void *(*)(void *param)
460
461The entry function for a coroutine. The `param` is the value passed to `Coroutine_Continue`, and the `void *` return value can be accessed through the `Coroutine` object using `Coroutine_GetValue()`.
462
463### Coroutine *Coroutine_New(Coroutine_Start start)
464
465Create a new `Coroutine`. The `Coroutine` system must be started to create a `Coroutine`. The stack size available to the coroutine will be `COROUTINE_STACK_SIZE` defined in `coroutine.h`. When you have finished with your `Coroutine`, use `Coroutine_Delete()` to delete it.
466
467### void Coroutine_Run_Coroutine(Coroutine *cor, void *value)
468
469Run the `Coroutine` and return when it returns. This is how to start coroutines running in the coroutine system. It is an error for the run coroutine to return before all other coroutines have completed.
470
471### void *Coroutine_Run(Coroutine_Start start, void *value)
472
473Convenience wrapper for `Coroutine_Run_Coroutine` which creates the `Coroutine` and retrieves its result.
474
475### void Coroutine_Delete(Coroutine *cor)
476
477Use `Coroutine_Delete()` to delete a coroutine when it is no longer needed.
478
479### void Coroutine_Continue(Coroutine *cor, void *value, bool early)
480
481Continue the given `Coroutine`. `value` is passed to the coroutine, as `param` to the `start` function, or as the return value from `Coroutine_Yield`. `early` determines whether the continued coroutine is added to the head of tail of the list of runable coroutines.
482
483### void *Coroutine_Yield(void *value, Coroutine_YieldCallback on_yield, void *this)
484
485Yield `value` from the current coroutine; this coroutine is moved to the list of coroutines waiting to be continued. The next runable coroutine is run - either by its start routine being called with `value` as its `param`, or by `value`being returned from its `Coroutine_Yield()`.
486
487### void *Coroutine_GetValue(Coroutine *cor)
488
489Return the `Coroutine`'s value - the value last yielded, or returned by its `start` routine.
490
491### Coroutine *Coroutine_GetActive()
492
493Return whihc coroutine is currently running, ie the caller's `Coroutine`.
494
495### bool Coroutine_IsRunning(Coroutine *cor)
496
497Return whether the given coroutine is still running - it may be running, ready to run, or waiting to be continued, but won't have returned from its `start` function.
498
499### int Coroutine_GetStackHeadroom()
500
501Return the headroom available in the current coroutine's stack. This can be used to detect when your coroutine is nearing its stack limit, and then use `Coroutine_Chain()` to continue in a new chunk of coroutine stack.
502
503### bool Coroutine_HasCoroutinesInFreePool()
504
505Return whether the coroutine system has any coroutine stacks available in its coroutine stack free pool.
506
507### void *Coroutine_GetCStackTop()
508
509Return an address which is near to the top of used C stack.
510
511### void *Coroutine_Chain(Coroutine_Start start, void *value)
512
513Run `start` with `value` on a new coroutine, and return its return value. It is expected that `Coroutine_Chain()` will be used when your coroutine is running short of stack - it is not an alternative to `Coroutine_Run()`.
514