0 branches 0 tags
30
Documentation experiment again
jonroach 19 Sep 2025 08:33
2 contributors
518 lines18.3 KB
Markdown Text Raw Blame
  • Delete
1Stackful coroutines in C.
2
3* `Async` coroutines which can pause, waiting for a future.
4* `Generator` coroutines used as generators for loops.
5* `Coroutine` the coroutine engine used by `Async` and `Generator`.
6
7Your code doesn't need to do anything special to be a coroutine, and only standard, or commonly available libraries are needed.
8
9## Prerequisites
10
11The goal was to make a system which can be used 'out of the box'. These libraries rely on as much as possible on C's cross-platform comfort zone. C's standard libraries are used as far as possible, but, as `threads.h` is not usually supported, `pthread.h` has been used instead.
12
13You will need to build & link the code as part of your system - `coroutine/async.c`, `coroutine/generator.c` and `coroutine/coroutine.c` - ensure the headers, `include/*`, are available on your include path.
14
15## Quick Start
16
17### Async
18
19To run an Async program:
20
21 #include "async.h"
22 main(){
23 Async_StartSystem();
24 void *res = NULL;
25 bool canceled = Async_Run(asyncmain, &param, &res);
26 Async_StopSystem();
27 }
28
29Async runs tasks, switching between them when the current task waits on an `Async_Future`. `asyncmain()` is run as a task. The start function for any task looks like this:
30
31 bool mytask(void *param, void **res){
32
33 // do your thing here
34
35 return canceled;
36 }
37
38When `Task` returns from its start function, it returns whether it was canceled. Canceled `Task`s are assumed to have not finished what they were doing.
39
40Within your async task, create `Async_Task`s and `Async_Task_Await()` them when you want to wait for their result:
41
42 Async_Task task1;
43 Async_Task_ctor(&task1, adifferenttask, &task1param);
44
45 void *result;
46 bool canceled = Async_Task_Await(&task1, &result);
47
48 Async_Task_dtor(&task1);
49
50 // use the result
51
52When a task needs to wait for something, and wants to allow other tasks to run, it should use a `Future`:
53
54 Async_Future future;
55 Async_Future_ctor(&future);
56
57 // pass the future to the background-thing-which-might-take-a-while
58
59 void *res;
60 bool canceled = Async_Future_Await(&future, &res);
61
62 Async_Future_dtor(&future);
63
64When the background-thing-which-might-take-a-while has a result:
65
66 Async_Future_SetResult(future, false, result);
67
68### Generators
69
70The coroutine system needs to be started, either through `Async_StartSystem()`, or directly with `Coroutine_StartSystem()` if you don't want to do async things.
71
72You will need a generator function:
73
74 void *yield_my_things(void *param){
75 bool domore = true;
76
77 // loop/call functions to find more values to yield, and when you have one:
78 domore = Generator_Yield(thing);
79 // .. if domore is false, exit your generator - it is being destructed
80
81 // not actually used by generators, but this is a useful convention for bubbling
82 // the flag out to calling functions.
83 return (void *)domore;
84 }
85
86And to use it:
87
88 Generator gen;
89 Generator_ctor(&gen, yield_my_things, "..");
90 void *thing;
91 while(Generator_Next(&gen, &thing)){
92 // use thing - a value yielded by your generator
93 }
94 Generator_dtor(&gen);
95
96### Coroutines
97
98While you can use coroutines directly, it's designed as a system to support more useful patterns, like `Async` and `Generators`.
99
100The Coroutines system must be started:
101
102 Coroutine_StartSystem();
103 // use coroutines here
104 Coroutine_StopSystem();
105
106Your coroutine will need to have a start function:
107
108 void *start(void *param){
109 ...
110 }
111
112When there is no coroutine running, start your 'main' coroutine:
113
114 void *result = Coroutine_Run(comain, param);
115
116Create other coroutines like this:
117
118 Coroutine *cor = Coroutine_New(start);
119
120When you want a Coroutine to run, or to return from a yield:
121
122 Coroutine_Continue(cor, value, run_early);
123
124`value` will be start function's parameter, or the value returned from the yield.
125
126Within the Coroutine, to yield a value:
127
128 void *Coroutine_Yield(value, on_yield, void *me);
129
130The on_yield function is called after the coroutine has been 'wait'ed, but before the next coroutine is resumed.
131
132## How it Works
133
134The coroutine system uses the stack, divided into smaller stacks, for the coroutines. This means you may need to consider whether the coroutine stack size, set by `COROUTINE_STARTUP_STACK_SIZE`, is right for your coroutines, and whether your stack size is enough for the number of coroutines you might run concurrently.
135
136As each of your thread has its own stack - the coroutine system can be run (or not) independantly on each of your threads. For some special cases, you may want to adjust each of your thread's stack sizes depending on how it is used.
137
138## Style
139
140The style is influenced by C++. For example, where possible, a `Something *Something_New(a, b, c)` and `Something_Delete(Something *)`, where a `Something` is `malloc`ed, will have corresponding `Somthing_ctor(Somthing *, a, b, c)` and `Something_dtor(Something *)` to initialise and finalise a `Something` on the stack, or within another object. Using `.._ctor()` and `.._dtor()` will be faster as they avoid the `malloc()` and `free()`.
141
142 Something *oneofthem = Something_New();
143 // use oneofthem
144 Something_Delete(oneofthem);
145
146Can be also be done like this, and this will run faster:
147
148 Something oneofthem;
149 Something_ctor(&oneofthem);
150 // use oneofthem
151 Something_dtor(&oneofthem);
152
153The exception is `Coroutine_New()` and `Coroutine_Delete()`. The returned `Coroutine` is somewhere on your thread's stack - its memory is managed by the coroutine system, and is allocated and freed quickly.
154
155## Usage
156
157When you are using coroutines or generators:
158
159 void *myfunc(void *){
160 // your function here
161 }
162
163 Coroutine_StartSystem();
164 Coroutine_Run(myfunc, (void *)myparam);
165 Coroutine_StopSystem();
166
167If you also use async, then:
168
169 bool myfunc(void *myparam, void **res){
170 // your async function here
171 }
172
173 Async_StartSystem();
174 void *res = NULL;
175 bool canceled = Async_Run(myfunc, myparam, &res);
176 Async_StopSystem();
177
178While the system is started, you can make many calls to `Coroutine_Run()` or `Async_Run()`. A running system is thread local - each thread you want to use coroutines on will need to be `Coroutine_StartSystem()`ed or `Async_StartSystem()`ed.
179
180## Stack Overruns
181
182The C stack is divided down into smaller stacks. There's one to give some work room between `..StartSystem()` and `..Run()`, and one for each coroutine. These have guard markers which are checked to see if the stack has overrun. If there is a stack overrun, the system cannot continue - a message is output and the programe exited. There's a number of ways to avoid this issue:
183
184* Use less stack. This is, sometimes, the right advice, especially if the startup stack overrins. The expectation is that very little is done between `.._StartSystem()` and `..Run()`. If your situation needs more doing, you can...
185
186* increase the stack size. Adjust `COROUTINE_STACK_SIZE` (for startup) or `COROUTINE_STARTUP_STACK_SIZE` (for coroutines) as appropriate. If your use case is even more demanding, such as if you want 1000s of coroutines (so you need small stack chunks), /and/ some of them can recurse an unknown amount (so you need a deep stack for that coroutine), then you can...
187
188* monitor stack headroom, and add another stack chunk if you need to:
189
190In this last case you'll need to add some code at key points:
191
192 void *myfunction(void *param){
193 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
194 return Coroutine_Chain(myfunction, param);
195 }
196 // do everything normally
197 }
198
199More realistically:
200
201 struct myfunctionparams {
202 int a;
203 char *b;
204 struct dog *d;
205 }
206
207 void *mychain(void *param){
208 struct myfunctionparams *myparams = (struct myfunctionparams *)params;
209 return (void *)myfunction(myparams->a, myparams->b, *myparams->d);
210 }
211
212 int myfunction(int a, char *b, struct dog d){
213 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
214 struct myfunctionparams params = {
215 a,
216 b,
217 &d
218 };
219 return (int)Coroutine_Chain(mychain, &params);
220 }
221 }
222
223# API
224
225## Async
226
227The pattern for using async is:
228
229 void *myasyncmaintask(void *param){
230 // do your main async task things here, like starting more tasks
231 }
232
233 Async_StartSystem();
234 void *res = NULL;
235 bool canceled = Async_Run(myasyncmaintask, NULL, &res);
236 Async_StopSystem();
237
238To create and wait for an async task:
239
240 Async_Task task1;
241 Async_Task_ctor(&task1, asynctask1, &task1param);
242 void *res = NULL;
243 bool canceled = Async_Task_Await(&task1, void **res)
244 Async_Task_dtor(&task1);
245
246or, if you prefer new & delete:
247
248 Async_Task *task1 = Async_Task_New(asynctask1, &task1param);
249 void *res = NULL;
250 bool canceled = Async_Task_Await(task1, void **res)
251 Async_Task_Delete(task1);
252
253Inside your task, when there is something to wait for and you want other tasks to run while your task is waiting, you will need a future:
254
255 Async_Future future;
256 Async_Future_ctor(&future);
257
258 // keep &future to hand for when the background thing completes
259 bool canceled = Async_Future_Await(&future, NULL);
260
261 Async_Future_dtor(&future);
262
263`Async_Future_New()` and `Async_Future_Delete()` are also available if you prefer that style.
264
265Inside the callback when the background thing is complete:
266
267 // result is a void *
268 Async_Future_SetResult(future, result, false);
269
270or, if something went wrong:
271
272 // exception is a void *
273 Async_Future_SetResult(future, exception, true);
274
275Back in the task, you can respond to the future:
276
277 ... Async_Future_Await has returned
278 if (canceled){
279 // exit quickly - you've been canceled
280 // you could, for example, use the future's result as an exception, or error code here
281 }
282 // carry on - the future's result may be an actual result, that's up to you
283
284
285#### `void Async_Future_ctor(Async_Future *fut)`
286
287fut
288: The `Async_Future` being constructed
289
290Initialise a future. When you no longer need it, use `Async_Future_dtor()`.
291
292##### `Async_Future *Async_Future_New()`
293
294(returns)
295: The new future
296
297Allocates and initialises a future, When you no longer need it, use `Async_Future_Delete()`.
298
299###### `void Async_Future_dtor(Async_Future *fut)`
300
301fut
302: The `Async_Future` being destructed
303
304Destruct a future previously constructed with `Async_Future_ctor()`.
305
306#### `void Async_Future_Delete(Async_Future *fut)`
307
308Delete (finalise and free) a future previously new'ed with `Async_Future_New()`
309
310#### void Async_Future_SetResult(Async_Future *fut, bool canceled, void *value)
311
312Set the result of a future. This has an effect only the first time its done, ie a completed future can't be canceled and a canceled future can;t be completed.
313
314#### bool Async_Future_GetResult(Async_Future *fut, void **res)
315
316Get the result of a future. The return value is whether it was canceled. `res` may be `NULL`.
317
318#### typedef void (*Future_Watcher)(void *me, Async_Future *fut)
319
320A `Future_Watcher` is a callback called when a future is done. The `me` parameter is the one passed to `Async_Future_AddWatcher()`. `fut` is the future which has just completed.
321
322#### void Async_Future_AddWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
323
324Add a watcher (callback) to be called when the future is done. If the future is already complete, `watcher` is immediately called. The `me` value is passed to the watcher as its `me` parameter.
325
326#### void Async_Future_RemoveWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
327
328Remove a watcher from a future.
329
330#### bool Async_Future_Await(Async_Future *fut, void **res)
331
332
333
334## Generator
335
336The pattern for a `Generator` is:
337
338#### A loop which uses the `Generator`
339
340 Generator gen;
341 Generator_ctor(&gen, mygen, &param);
342
343 void *value;
344 while(Generator_Next(&gen, &value)){
345 // use value here
346 }
347 // value is now the return value from the Generator
348
349 Generator_dtor(&gen);
350
351Or:
352
353 Generator *gen = Generator_New(mygen, &param);
354
355 void *value;
356 while(Generator_Next(gen, &value)){
357 // use value here
358 }
359
360 Generator_Delete(gen);
361
362`Generator`s yield a series of `void *`s - what the `void *`s mean is up to you. `Generator_Next()` returns a `bool` to indicate whether the `Generator` has finished.
363
364#### A generator function
365
366 void *mygen(void *param){
367 bool domore = true;
368 // The parameter is a pointer to a string of chars
369 for (char *str = param; *str; ++str) {
370 // The value yielded is a pointer to a character in the string
371 domore = Generator_Yield(str);
372 if (!domore){
373 break;
374 }
375 }
376
377 return (void *)domore;
378 }
379
380The `bool` returned from `Generator_Yield()` indicates whether the generator function should yield more values. When it is `false` the `Generator` is being finalised - your generator function should close files, and release any other resources it has claimed, before exiting.
381
382#### void Generator_ctor(Generator *gen, void *(*start)(void *), void *param)
383
384Initialise a `Generator`
385
386 Generator gen;
387 Generator_ctor(&gen, mystart, &params);
388
389 // Generator is used
390
391 // ... later:
392 Generator_dtor(&gen);
393
394#### Generator *Generator_New(void *(*)(void *), void *)
395
396`new` a `Generator` - malloc it and initialise it.
397
398 Generator *gen = Generator_New(mystart, &params);
399
400 // Generator is used
401
402 // ... later:
403 Generator_Delete(gen);
404
405#### void Generator_dtor(Generator *gen)
406
407Finalise a `Generator`. Once a `Generator` is no longer needed, it must be finalised:
408
409 // earlier...
410 Generator gen;
411 Generator_ctor(&gen, mystart, &params);
412
413 // Generator is used
414
415 // the Generator is no longer needed
416 Generator_dtor(&gen);
417
418
419#### void Generator_Delete(Generator *)
420
421Finalise then `free()` a `Generator`. Once a `new`ed `Generator` is no longer needed, it must be deleted:
422
423 // earlier...
424 Generator *gen = Generator_New(mystart, &params);
425
426 // Generator is used
427
428 // the Generator is no longer needed
429 Generator_Delete(gen);
430
431
432#### bool Generator_Next(Generator *, void **value)
433
434Get the next value yielded by the `Generator`.
435
436 void *value;
437 while(Generator_Next(gen, &value)){
438 // use value here
439 }
440
441When `true` is returned, `value` is the value yielded by the `Generator`. When `false` is returned, `value` is the value returned when the `Generator` returned - the `Generator` has finished.
442
443#### bool Generator_Yield(void *)
444
445Yield a value from a `Generator` function.
446
447 bool domore = Generator_Yield(value);
448
449`value` is then provided by `Generator_Next()` as the next value from the generator. The `bool` returned by `Generator_Yield()` indicates whether more values should be provided by your generator function. When `true` provide more values if possible. When `false` close files, free memory, free up any other resources and `return`. `false` is returned when the `Generator` is being finalised.
450
451## Coroutine
452
453#### void Coroutine_StartSystem();
454
455Start the coroutine system on this thread. When you've finished with `Coroutine` call `Coroutine_Stop()`. `Coroutine` can be started & stopped many times on one thread. The total stack allowed for all coroutines running on a thread is the size of the call stack on that thread.
456
457#### void Coroutine_StopSystem();
458
459Stop the coroutine system on this thread.
460
461#### Coroutine_Start
462
463 void *(*)(void *param)
464
465The entry function for a coroutine. The `param` is the value passed to `Coroutine_Continue`, and the `void *` return value can be accessed through the `Coroutine` object using `Coroutine_GetValue()`.
466
467#### Coroutine *Coroutine_New(Coroutine_Start start)
468
469Create a new `Coroutine`. The `Coroutine` system must be started to create a `Coroutine`. The stack size available to the coroutine will be `COROUTINE_STACK_SIZE` defined in `coroutine.h`. When you have finished with your `Coroutine`, use `Coroutine_Delete()` to delete it.
470
471#### void Coroutine_Run_Coroutine(Coroutine *cor, void *value)
472
473Run the `Coroutine` and return when it returns. This is how to start coroutines running in the coroutine system. It is an error for the run coroutine to return before all other coroutines have completed.
474
475#### void *Coroutine_Run(Coroutine_Start start, void *value)
476
477Convenience wrapper for `Coroutine_Run_Coroutine` which creates the `Coroutine` and retrieves its result.
478
479#### void Coroutine_Delete(Coroutine *cor)
480
481Use `Coroutine_Delete()` to delete a coroutine when it is no longer needed.
482
483#### void Coroutine_Continue(Coroutine *cor, void *value, bool early)
484
485Continue the given `Coroutine`. `value` is passed to the coroutine, as `param` to the `start` function, or as the return value from `Coroutine_Yield`. `early` determines whether the continued coroutine is added to the head of tail of the list of runable coroutines.
486
487#### void *Coroutine_Yield(void *value, Coroutine_YieldCallback on_yield, void *this)
488
489Yield `value` from the current coroutine; this coroutine is moved to the list of coroutines waiting to be continued. The next runable coroutine is run - either by its start routine being called with `value` as its `param`, or by `value`being returned from its `Coroutine_Yield()`.
490
491#### void *Coroutine_GetValue(Coroutine *cor)
492
493Return the `Coroutine`'s value - the value last yielded, or returned by its `start` routine.
494
495#### Coroutine *Coroutine_GetActive()
496
497Return whihc coroutine is currently running, ie the caller's `Coroutine`.
498
499#### bool Coroutine_IsRunning(Coroutine *cor)
500
501Return whether the given coroutine is still running - it may be running, ready to run, or waiting to be continued, but won't have returned from its `start` function.
502
503#### int Coroutine_GetStackHeadroom()
504
505Return the headroom available in the current coroutine's stack. This can be used to detect when your coroutine is nearing its stack limit, and then use `Coroutine_Chain()` to continue in a new chunk of coroutine stack.
506
507#### bool Coroutine_HasCoroutinesInFreePool()
508
509Return whether the coroutine system has any coroutine stacks available in its coroutine stack free pool.
510
511#### void *Coroutine_GetCStackTop()
512
513Return an address which is near to the top of used C stack.
514
515### void *Coroutine_Chain(Coroutine_Start start, void *value)
516
517Run `start` with `value` on a new coroutine, and return its return value. It is expected that `Coroutine_Chain()` will be used when your coroutine is running short of stack - it is not an alternative to `Coroutine_Run()`.
518