0 branches 0 tags
27
Documentation experiment
jonroach 19 Sep 2025 08:11
2 contributors
514 lines18.2 KB
Newer
Older
Markdown Text Raw Blame
  • Delete
-
+
commited
{line.log.rev}
 on
9 months ago
1
 Stackful coroutines in C.
9 months ago
17
2
9 months ago
3
 * `Async` coroutines which can pause, waiting for a future.
4
 * `Generator` coroutines used as generators for loops.
5
 * `Coroutine` the coroutine engine used by `Async` and `Generator`.
9 months ago
17
6
9 months ago
7
 Your code doesn't need to do anything special to be a coroutine, and only standard, or commonly available libraries are needed.
9 months ago
17
8
9 months ago
9
 ## Prerequisites
9 months ago
17
10
9 months ago
11
 The goal was to make a system which can be used 'out of the box'. These libraries rely on as much as possible on C's cross-platform comfort zone. C's standard libraries are used as far as possible, but, as `threads.h` is not usually supported, `pthread.h` has been used instead.
9 months ago
17
12
9 months ago
13
 You will need to build & link the code as part of your system - `coroutine/async.c`, `coroutine/generator.c` and `coroutine/coroutine.c` - ensure the headers, `include/*`, are available on your include path.
9 months ago
17
14
9 months ago
15
 ## Quick Start
9 months ago
17
16
9 months ago
17
 ### Async
9 months ago
17
18
9 months ago
19
 To run an Async program:
9 months ago
17
20
9 months ago
21
 #include "async.h"
22
 main(){
23
 Async_StartSystem();
24
 void *res = NULL;
25
 bool canceled = Async_Run(asyncmain, &param, &res);
26
 Async_StopSystem();
27
 }
28
29
 Async runs tasks, switching between them when the current task waits on an `Async_Future`. `asyncmain()` is run as a task. The start function for any task looks like this:
30
31
 bool mytask(void *param, void **res){
32
33
 // do your thing here
34
35
 return canceled;
36
 }
37
38
 When `Task` returns from its start function, it returns whether it was canceled. Canceled `Task`s are assumed to have not finished what they were doing.
39
40
 Within your async task, create `Async_Task`s and `Async_Task_Await()` them when you want to wait for their result:
41
42
 Async_Task task1;
43
 Async_Task_ctor(&task1, adifferenttask, &task1param);
44
45
 void *result;
46
 bool canceled = Async_Task_Await(&task1, &result);
47
48
 Async_Task_dtor(&task1);
49
50
 // use the result
51
52
 When a task needs to wait for something, and wants to allow other tasks to run, it should use a `Future`:
53
54
 Async_Future future;
55
 Async_Future_ctor(&future);
56
57
 // pass the future to the background-thing-which-might-take-a-while
58
9 months ago
59
 void *res;
60
 bool canceled = Async_Future_Await(&future, &res);
9 months ago
61
62
 Async_Future_dtor(&future);
63
64
 When the background-thing-which-might-take-a-while has a result:
65
66
 Async_Future_SetResult(future, false, result);
67
68
 ### Generators
69
70
 The coroutine system needs to be started, either through `Async_StartSystem()`, or directly with `Coroutine_StartSystem()` if you don't want to do async things.
71
72
 You will need a generator function:
73
74
 void *yield_my_things(void *param){
75
 bool domore = true;
76
77
 // loop/call functions to find more values to yield, and when you have one:
78
 domore = Generator_Yield(thing);
79
 // .. if domore is false, exit your generator - it is being destructed
80
81
 // not actually used by generators, but this is a useful convention for bubbling
82
 // the flag out to calling functions.
83
 return (void *)domore;
84
 }
85
86
 And to use it:
87
88
 Generator gen;
89
 Generator_ctor(&gen, yield_my_things, "..");
90
 void *thing;
91
 while(Generator_Next(&gen, &thing)){
92
 // use thing - a value yielded by your generator
93
 }
94
 Generator_dtor(&gen);
95
96
 ### Coroutines
97
98
 While you can use coroutines directly, it's designed as a system to support more useful patterns, like `Async` and `Generators`.
99
100
 The Coroutines system must be started:
101
102
 Coroutine_StartSystem();
103
 // use coroutines here
104
 Coroutine_StopSystem();
105
106
 Your coroutine will need to have a start function:
107
108
 void *start(void *param){
109
 ...
110
 }
111
112
 When there is no coroutine running, start your 'main' coroutine:
113
114
 void *result = Coroutine_Run(comain, param);
115
116
 Create other coroutines like this:
117
118
 Coroutine *cor = Coroutine_New(start);
119
120
 When you want a Coroutine to run, or to return from a yield:
121
122
 Coroutine_Continue(cor, value, run_early);
123
124
 `value` will be start function's parameter, or the value returned from the yield.
125
126
 Within the Coroutine, to yield a value:
127
128
 void *Coroutine_Yield(value, on_yield, void *me);
129
130
 The on_yield function is called after the coroutine has been 'wait'ed, but before the next coroutine is resumed.
131
132
 ## How it Works
133
134
 The coroutine system uses the stack, divided into smaller stacks, for the coroutines. This means you may need to consider whether the coroutine stack size, set by `COROUTINE_STARTUP_STACK_SIZE`, is right for your coroutines, and whether your stack size is enough for the number of coroutines you might run concurrently.
135
136
 As each of your thread has its own stack - the coroutine system can be run (or not) independantly on each of your threads. For some special cases, you may want to adjust each of your thread's stack sizes depending on how it is used.
137
138
 ## Style
139
9 months ago
17
140
 The style is influenced by C++. For example, where possible, a `Something *Something_New(a, b, c)` and `Something_Delete(Something *)`, where a `Something` is `malloc`ed, will have corresponding `Somthing_ctor(Somthing *, a, b, c)` and `Something_dtor(Something *)` to initialise and finalise a `Something` on the stack, or within another object. Using `.._ctor()` and `.._dtor()` will be faster as they avoid the `malloc()` and `free()`.
141
142
 Something *oneofthem = Something_New();
143
 // use oneofthem
144
 Something_Delete(oneofthem);
145
146
 Can be also be done like this, and this will run faster:
147
148
 Something oneofthem;
149
 Something_ctor(&oneofthem);
150
 // use oneofthem
151
 Something_dtor(&oneofthem);
152
9 months ago
153
 The exception is `Coroutine_New()` and `Coroutine_Delete()`. The returned `Coroutine` is somewhere on your thread's stack - its memory is managed by the coroutine system, and is allocated and freed quickly.
154
9 months ago
17
155
 ## Usage
156
157
 When you are using coroutines or generators:
158
159
 void *myfunc(void *){
160
 // your function here
161
 }
162
163
 Coroutine_StartSystem();
164
 Coroutine_Run(myfunc, (void *)myparam);
165
 Coroutine_StopSystem();
166
167
 If you also use async, then:
168
169
 bool myfunc(void *myparam, void **res){
170
 // your async function here
171
 }
172
173
 Async_StartSystem();
174
 void *res = NULL;
175
 bool canceled = Async_Run(myfunc, myparam, &res);
176
 Async_StopSystem();
177
178
 While the system is started, you can make many calls to `Coroutine_Run()` or `Async_Run()`. A running system is thread local - each thread you want to use coroutines on will need to be `Coroutine_StartSystem()`ed or `Async_StartSystem()`ed.
179
9 months ago
180
 ## Stack Overruns
9 months ago
17
181
9 months ago
182
 The C stack is divided down into smaller stacks. There's one to give some work room between `..StartSystem()` and `..Run()`, and one for each coroutine. These have guard markers which are checked to see if the stack has overrun. If there is a stack overrun, the system cannot continue - a message is output and the programe exited. There's a number of ways to avoid this issue:
9 months ago
17
183
9 months ago
184
 * Use less stack. This is, sometimes, the right advice, especially if the startup stack overrins. The expectation is that very little is done between `.._StartSystem()` and `..Run()`. If your situation needs more doing, you can...
185
186
 * increase the stack size. Adjust `COROUTINE_STACK_SIZE` (for startup) or `COROUTINE_STARTUP_STACK_SIZE` (for coroutines) as appropriate. If your use case is even more demanding, such as if you want 1000s of coroutines (so you need small stack chunks), /and/ some of them can recurse an unknown amount (so you need a deep stack for that coroutine), then you can...
187
188
 * monitor stack headroom, and add another stack chunk if you need to:
189
190
 In this last case you'll need to add some code at key points:
191
192
 void *myfunction(void *param){
193
 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
194
 return Coroutine_Chain(myfunction, param);
195
 }
196
 // do everything normally
197
 }
198
199
 More realistically:
200
201
 struct myfunctionparams {
202
 int a;
203
 char *b;
204
 struct dog *d;
205
 }
206
207
 void *mychain(void *param){
208
 struct myfunctionparams *myparams = (struct myfunctionparams *)params;
209
 return (void *)myfunction(myparams->a, myparams->b, *myparams->d);
210
 }
211
212
 int myfunction(int a, char *b, struct dog d){
213
 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
214
 struct myfunctionparams params = {
215
 a,
216
 b,
217
 &d
218
 };
219
 return (int)Coroutine_Chain(mychain, &params);
220
 }
221
 }
222
9 months ago
223
 # API
9 months ago
17
224
9 months ago
225
 ## Async
226
227
 The pattern for using async is:
228
229
 void *myasyncmaintask(void *param){
230
 // do your main async task things here, like starting more tasks
231
 }
232
233
 Async_StartSystem();
234
 void *res = NULL;
235
 bool canceled = Async_Run(myasyncmaintask, NULL, &res);
236
 Async_StopSystem();
237
238
 To create and wait for an async task:
239
240
 Async_Task task1;
241
 Async_Task_ctor(&task1, asynctask1, &task1param);
242
 void *res = NULL;
243
 bool canceled = Async_Task_Await(&task1, void **res)
244
 Async_Task_dtor(&task1);
245
246
 or, if you prefer new & delete:
247
248
 Async_Task *task1 = Async_Task_New(asynctask1, &task1param);
249
 void *res = NULL;
250
 bool canceled = Async_Task_Await(task1, void **res)
251
 Async_Task_Delete(task1);
252
253
 Inside your task, when there is something to wait for and you want other tasks to run while your task is waiting, you will need a future:
254
255
 Async_Future future;
256
 Async_Future_ctor(&future);
257
258
 // keep &future to hand for when the background thing completes
259
 bool canceled = Async_Future_Await(&future, NULL);
260
261
 Async_Future_dtor(&future);
262
263
 `Async_Future_New()` and `Async_Future_Delete()` are also available if you prefer that style.
264
265
 Inside the callback when the background thing is complete:
266
267
 // result is a void *
268
 Async_Future_SetResult(future, result, false);
269
270
 or, if something went wrong:
271
272
 // exception is a void *
273
 Async_Future_SetResult(future, exception, true);
274
275
 Back in the task, you can respond to the future:
276
277
 ... Async_Future_Await has returned
278
 if (canceled){
279
 // exit quickly - you've been canceled
280
 // you could, for example, use the future's result as an exception, or error code here
281
 }
282
 // carry on - the future's result may be an actual result, that's up to you
283
284
285
 ### void Async_Future_ctor(Async_Future *fut)
286
287
 Initialise a future. When you no longer need it, use `Async_Future_dtor()`.
288
289
 ### Async_Future *Async_Future_New()
290
291
 Allocate and initialise a future, When you no longer need it, use `Async_Future_Delete()`.
292
293
 Returns the new `Async_Future`
294
295
 #### `void Async_Future_dtor(Async_Future *fut)`
296
297
 fut
298
 : The `Async_Future` being destructed
299
300
 Destruct a future previously constructed with `Async_Future_ctor()`.
301
302
 ### void Async_Future_Delete(Async_Future *fut)
303
304
 Delete (finalise and free) a future previously new'ed with `Async_Future_New()`
305
306
 ### void Async_Future_SetResult(Async_Future *fut, bool canceled, void *value)
307
308
 Set the result of a future. This has an effect only the first time its done, ie a completed future can't be canceled and a canceled future can;t be completed.
309
310
 ### bool Async_Future_GetResult(Async_Future *fut, void **res)
311
312
 Get the result of a future. The return value is whether it was canceled. `res` may be `NULL`.
313
314
 ### typedef void (*Future_Watcher)(void *me, Async_Future *fut)
315
316
 A `Future_Watcher` is a callback called when a future is done. The `me` parameter is the one passed to `Async_Future_AddWatcher()`. `fut` is the future which has just completed.
317
318
 ### void Async_Future_AddWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
319
320
 Add a watcher (callback) to be called when the future is done. If the future is already complete, `watcher` is immediately called. The `me` value is passed to the watcher as its `me` parameter.
321
322
 ### void Async_Future_RemoveWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
323
324
 Remove a watcher from a future.
325
326
 ### bool Async_Future_Await(Async_Future *fut, void **res)
327
328
329
9 months ago
330
 ## Generator
9 months ago
17
331
9 months ago
332
 The pattern for a `Generator` is:
333
334
 #### A loop which uses the `Generator`
335
9 months ago
17
336
 Generator gen;
9 months ago
337
 Generator_ctor(&gen, mygen, &param);
9 months ago
17
338
9 months ago
339
 void *value;
340
 while(Generator_Next(&gen, &value)){
341
 // use value here
9 months ago
17
342
 }
9 months ago
343
 // value is now the return value from the Generator
344
9 months ago
17
345
 Generator_dtor(&gen);
346
9 months ago
347
 Or:
9 months ago
17
348
9 months ago
349
 Generator *gen = Generator_New(mygen, &param);
9 months ago
17
350
9 months ago
351
 void *value;
352
 while(Generator_Next(gen, &value)){
353
 // use value here
354
 }
9 months ago
17
355
9 months ago
356
 Generator_Delete(gen);
9 months ago
17
357
9 months ago
358
 `Generator`s yield a series of `void *`s - what the `void *`s mean is up to you. `Generator_Next()` returns a `bool` to indicate whether the `Generator` has finished.
9 months ago
17
359
9 months ago
360
 #### A generator function
9 months ago
17
361
9 months ago
362
 void *mygen(void *param){
363
 bool domore = true;
364
 // The parameter is a pointer to a string of chars
365
 for (char *str = param; *str; ++str) {
366
 // The value yielded is a pointer to a character in the string
367
 domore = Generator_Yield(str);
368
 if (!domore){
369
 break;
370
 }
371
 }
372
373
 return (void *)domore;
9 months ago
17
374
 }
375
9 months ago
376
 The `bool` returned from `Generator_Yield()` indicates whether the generator function should yield more values. When it is `false` the `Generator` is being finalised - your generator function should close files, and release any other resources it has claimed, before exiting.
9 months ago
17
377
9 months ago
378
 ### void Generator_ctor(Generator *gen, void *(*start)(void *), void *param)
9 months ago
17
379
9 months ago
380
 Initialise a `Generator`
9 months ago
17
381
9 months ago
382
 Generator gen;
383
 Generator_ctor(&gen, mystart, &params);
9 months ago
17
384
9 months ago
385
 // Generator is used
9 months ago
17
386
9 months ago
387
 // ... later:
388
 Generator_dtor(&gen);
389
390
 ### Generator *Generator_New(void *(*)(void *), void *)
391
392
 `new` a `Generator` - malloc it and initialise it.
393
394
 Generator *gen = Generator_New(mystart, &params);
395
396
 // Generator is used
397
398
 // ... later:
399
 Generator_Delete(gen);
400
401
 ### void Generator_dtor(Generator *gen)
402
403
 Finalise a `Generator`. Once a `Generator` is no longer needed, it must be finalised:
404
405
 // earlier...
406
 Generator gen;
407
 Generator_ctor(&gen, mystart, &params);
408
409
 // Generator is used
410
411
 // the Generator is no longer needed
412
 Generator_dtor(&gen);
413
414
415
 ### void Generator_Delete(Generator *)
416
417
 Finalise then `free()` a `Generator`. Once a `new`ed `Generator` is no longer needed, it must be deleted:
418
419
 // earlier...
420
 Generator *gen = Generator_New(mystart, &params);
421
422
 // Generator is used
423
424
 // the Generator is no longer needed
425
 Generator_Delete(gen);
426
427
428
 ### bool Generator_Next(Generator *, void **value)
429
430
 Get the next value yielded by the `Generator`.
431
432
 void *value;
433
 while(Generator_Next(gen, &value)){
434
 // use value here
9 months ago
17
435
 }
9 months ago
436
9 months ago
437
 When `true` is returned, `value` is the value yielded by the `Generator`. When `false` is returned, `value` is the value returned when the `Generator` returned - the `Generator` has finished.
9 months ago
438
9 months ago
439
 ### bool Generator_Yield(void *)
440
441
 Yield a value from a `Generator` function.
442
443
 bool domore = Generator_Yield(value);
444
445
 `value` is then provided by `Generator_Next()` as the next value from the generator. The `bool` returned by `Generator_Yield()` indicates whether more values should be provided by your generator function. When `true` provide more values if possible. When `false` close files, free memory, free up any other resources and `return`. `false` is returned when the `Generator` is being finalised.
446
9 months ago
447
 ## Coroutine
448
449
 ### void Coroutine_StartSystem();
450
451
 Start the coroutine system on this thread. When you've finished with `Coroutine` call `Coroutine_Stop()`. `Coroutine` can be started & stopped many times on one thread. The total stack allowed for all coroutines running on a thread is the size of the call stack on that thread.
452
453
 ### void Coroutine_StopSystem();
454
455
 Stop the coroutine system on this thread.
456
457
 ### Coroutine_Start
458
459
 void *(*)(void *param)
460
9 months ago
461
 The entry function for a coroutine. The `param` is the value passed to `Coroutine_Continue`, and the `void *` return value can be accessed through the `Coroutine` object using `Coroutine_GetValue()`.
9 months ago
462
9 months ago
463
 ### Coroutine *Coroutine_New(Coroutine_Start start)
9 months ago
464
465
 Create a new `Coroutine`. The `Coroutine` system must be started to create a `Coroutine`. The stack size available to the coroutine will be `COROUTINE_STACK_SIZE` defined in `coroutine.h`. When you have finished with your `Coroutine`, use `Coroutine_Delete()` to delete it.
466
9 months ago
467
 ### void Coroutine_Run_Coroutine(Coroutine *cor, void *value)
9 months ago
468
469
 Run the `Coroutine` and return when it returns. This is how to start coroutines running in the coroutine system. It is an error for the run coroutine to return before all other coroutines have completed.
470
9 months ago
471
 ### void *Coroutine_Run(Coroutine_Start start, void *value)
9 months ago
472
473
 Convenience wrapper for `Coroutine_Run_Coroutine` which creates the `Coroutine` and retrieves its result.
474
9 months ago
475
 ### void Coroutine_Delete(Coroutine *cor)
9 months ago
476
477
 Use `Coroutine_Delete()` to delete a coroutine when it is no longer needed.
478
9 months ago
479
 ### void Coroutine_Continue(Coroutine *cor, void *value, bool early)
9 months ago
480
481
 Continue the given `Coroutine`. `value` is passed to the coroutine, as `param` to the `start` function, or as the return value from `Coroutine_Yield`. `early` determines whether the continued coroutine is added to the head of tail of the list of runable coroutines.
482
9 months ago
483
 ### void *Coroutine_Yield(void *value, Coroutine_YieldCallback on_yield, void *this)
9 months ago
484
485
 Yield `value` from the current coroutine; this coroutine is moved to the list of coroutines waiting to be continued. The next runable coroutine is run - either by its start routine being called with `value` as its `param`, or by `value`being returned from its `Coroutine_Yield()`.
486
9 months ago
487
 ### void *Coroutine_GetValue(Coroutine *cor)
9 months ago
488
489
 Return the `Coroutine`'s value - the value last yielded, or returned by its `start` routine.
490
9 months ago
491
 ### Coroutine *Coroutine_GetActive()
9 months ago
492
493
 Return whihc coroutine is currently running, ie the caller's `Coroutine`.
494
9 months ago
495
 ### bool Coroutine_IsRunning(Coroutine *cor)
9 months ago
496
497
 Return whether the given coroutine is still running - it may be running, ready to run, or waiting to be continued, but won't have returned from its `start` function.
9 months ago
498
499
 ### int Coroutine_GetStackHeadroom()
500
501
 Return the headroom available in the current coroutine's stack. This can be used to detect when your coroutine is nearing its stack limit, and then use `Coroutine_Chain()` to continue in a new chunk of coroutine stack.
502
503
 ### bool Coroutine_HasCoroutinesInFreePool()
504
505
 Return whether the coroutine system has any coroutine stacks available in its coroutine stack free pool.
506
507
 ### void *Coroutine_GetCStackTop()
508
509
 Return an address which is near to the top of used C stack.
510
511
 ### void *Coroutine_Chain(Coroutine_Start start, void *value)
512
513
 Run `start` with `value` on a new coroutine, and return its return value. It is expected that `Coroutine_Chain()` will be used when your coroutine is running short of stack - it is not an alternative to `Coroutine_Run()`.
514