0 branches 0 tags
28
Documentation experiment again
jonroach 19 Sep 2025 08:53
2 contributors
518 lines18.3 KB
Newer
Older
Markdown Text Raw Blame
  • Delete
-
+
commited
{line.log.rev}
 on
8 months ago
1
 Stackful coroutines in C.
8 months ago
17
2
8 months ago
3
 * `Async` coroutines which can pause, waiting for a future.
4
 * `Generator` coroutines used as generators for loops.
5
 * `Coroutine` the coroutine engine used by `Async` and `Generator`.
8 months ago
17
6
8 months ago
7
 Your code doesn't need to do anything special to be a coroutine, and only standard, or commonly available libraries are needed.
8 months ago
17
8
8 months ago
9
 ## Prerequisites
8 months ago
17
10
8 months ago
11
 The goal was to make a system which can be used 'out of the box'. These libraries rely on as much as possible on C's cross-platform comfort zone. C's standard libraries are used as far as possible, but, as `threads.h` is not usually supported, `pthread.h` has been used instead.
8 months ago
17
12
8 months ago
13
 You will need to build & link the code as part of your system - `coroutine/async.c`, `coroutine/generator.c` and `coroutine/coroutine.c` - ensure the headers, `include/*`, are available on your include path.
8 months ago
17
14
8 months ago
15
 ## Quick Start
8 months ago
17
16
8 months ago
17
 ### Async
8 months ago
17
18
8 months ago
19
 To run an Async program:
8 months ago
17
20
8 months ago
21
 #include "async.h"
22
 main(){
23
 Async_StartSystem();
24
 void *res = NULL;
25
 bool canceled = Async_Run(asyncmain, &param, &res);
26
 Async_StopSystem();
27
 }
28
29
 Async runs tasks, switching between them when the current task waits on an `Async_Future`. `asyncmain()` is run as a task. The start function for any task looks like this:
30
31
 bool mytask(void *param, void **res){
32
33
 // do your thing here
34
35
 return canceled;
36
 }
37
38
 When `Task` returns from its start function, it returns whether it was canceled. Canceled `Task`s are assumed to have not finished what they were doing.
39
40
 Within your async task, create `Async_Task`s and `Async_Task_Await()` them when you want to wait for their result:
41
42
 Async_Task task1;
43
 Async_Task_ctor(&task1, adifferenttask, &task1param);
44
45
 void *result;
46
 bool canceled = Async_Task_Await(&task1, &result);
47
48
 Async_Task_dtor(&task1);
49
50
 // use the result
51
52
 When a task needs to wait for something, and wants to allow other tasks to run, it should use a `Future`:
53
54
 Async_Future future;
55
 Async_Future_ctor(&future);
56
57
 // pass the future to the background-thing-which-might-take-a-while
58
8 months ago
59
 void *res;
60
 bool canceled = Async_Future_Await(&future, &res);
8 months ago
61
62
 Async_Future_dtor(&future);
63
64
 When the background-thing-which-might-take-a-while has a result:
65
66
 Async_Future_SetResult(future, false, result);
67
68
 ### Generators
69
70
 The coroutine system needs to be started, either through `Async_StartSystem()`, or directly with `Coroutine_StartSystem()` if you don't want to do async things.
71
72
 You will need a generator function:
73
74
 void *yield_my_things(void *param){
75
 bool domore = true;
76
77
 // loop/call functions to find more values to yield, and when you have one:
78
 domore = Generator_Yield(thing);
79
 // .. if domore is false, exit your generator - it is being destructed
80
81
 // not actually used by generators, but this is a useful convention for bubbling
82
 // the flag out to calling functions.
83
 return (void *)domore;
84
 }
85
86
 And to use it:
87
88
 Generator gen;
89
 Generator_ctor(&gen, yield_my_things, "..");
90
 void *thing;
91
 while(Generator_Next(&gen, &thing)){
92
 // use thing - a value yielded by your generator
93
 }
94
 Generator_dtor(&gen);
95
96
 ### Coroutines
97
98
 While you can use coroutines directly, it's designed as a system to support more useful patterns, like `Async` and `Generators`.
99
100
 The Coroutines system must be started:
101
102
 Coroutine_StartSystem();
103
 // use coroutines here
104
 Coroutine_StopSystem();
105
106
 Your coroutine will need to have a start function:
107
108
 void *start(void *param){
109
 ...
110
 }
111
112
 When there is no coroutine running, start your 'main' coroutine:
113
114
 void *result = Coroutine_Run(comain, param);
115
116
 Create other coroutines like this:
117
118
 Coroutine *cor = Coroutine_New(start);
119
120
 When you want a Coroutine to run, or to return from a yield:
121
122
 Coroutine_Continue(cor, value, run_early);
123
124
 `value` will be start function's parameter, or the value returned from the yield.
125
126
 Within the Coroutine, to yield a value:
127
128
 void *Coroutine_Yield(value, on_yield, void *me);
129
130
 The on_yield function is called after the coroutine has been 'wait'ed, but before the next coroutine is resumed.
131
132
 ## How it Works
133
134
 The coroutine system uses the stack, divided into smaller stacks, for the coroutines. This means you may need to consider whether the coroutine stack size, set by `COROUTINE_STARTUP_STACK_SIZE`, is right for your coroutines, and whether your stack size is enough for the number of coroutines you might run concurrently.
135
136
 As each of your thread has its own stack - the coroutine system can be run (or not) independantly on each of your threads. For some special cases, you may want to adjust each of your thread's stack sizes depending on how it is used.
137
138
 ## Style
139
8 months ago
17
140
 The style is influenced by C++. For example, where possible, a `Something *Something_New(a, b, c)` and `Something_Delete(Something *)`, where a `Something` is `malloc`ed, will have corresponding `Somthing_ctor(Somthing *, a, b, c)` and `Something_dtor(Something *)` to initialise and finalise a `Something` on the stack, or within another object. Using `.._ctor()` and `.._dtor()` will be faster as they avoid the `malloc()` and `free()`.
141
142
 Something *oneofthem = Something_New();
143
 // use oneofthem
144
 Something_Delete(oneofthem);
145
146
 Can be also be done like this, and this will run faster:
147
148
 Something oneofthem;
149
 Something_ctor(&oneofthem);
150
 // use oneofthem
151
 Something_dtor(&oneofthem);
152
8 months ago
153
 The exception is `Coroutine_New()` and `Coroutine_Delete()`. The returned `Coroutine` is somewhere on your thread's stack - its memory is managed by the coroutine system, and is allocated and freed quickly.
154
8 months ago
17
155
 ## Usage
156
157
 When you are using coroutines or generators:
158
159
 void *myfunc(void *){
160
 // your function here
161
 }
162
163
 Coroutine_StartSystem();
164
 Coroutine_Run(myfunc, (void *)myparam);
165
 Coroutine_StopSystem();
166
167
 If you also use async, then:
168
169
 bool myfunc(void *myparam, void **res){
170
 // your async function here
171
 }
172
173
 Async_StartSystem();
174
 void *res = NULL;
175
 bool canceled = Async_Run(myfunc, myparam, &res);
176
 Async_StopSystem();
177
178
 While the system is started, you can make many calls to `Coroutine_Run()` or `Async_Run()`. A running system is thread local - each thread you want to use coroutines on will need to be `Coroutine_StartSystem()`ed or `Async_StartSystem()`ed.
179
8 months ago
180
 ## Stack Overruns
8 months ago
17
181
8 months ago
182
 The C stack is divided down into smaller stacks. There's one to give some work room between `..StartSystem()` and `..Run()`, and one for each coroutine. These have guard markers which are checked to see if the stack has overrun. If there is a stack overrun, the system cannot continue - a message is output and the programe exited. There's a number of ways to avoid this issue:
8 months ago
17
183
8 months ago
184
 * Use less stack. This is, sometimes, the right advice, especially if the startup stack overrins. The expectation is that very little is done between `.._StartSystem()` and `..Run()`. If your situation needs more doing, you can...
185
186
 * increase the stack size. Adjust `COROUTINE_STACK_SIZE` (for startup) or `COROUTINE_STARTUP_STACK_SIZE` (for coroutines) as appropriate. If your use case is even more demanding, such as if you want 1000s of coroutines (so you need small stack chunks), /and/ some of them can recurse an unknown amount (so you need a deep stack for that coroutine), then you can...
187
188
 * monitor stack headroom, and add another stack chunk if you need to:
189
190
 In this last case you'll need to add some code at key points:
191
192
 void *myfunction(void *param){
193
 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
194
 return Coroutine_Chain(myfunction, param);
195
 }
196
 // do everything normally
197
 }
198
199
 More realistically:
200
201
 struct myfunctionparams {
202
 int a;
203
 char *b;
204
 struct dog *d;
205
 }
206
207
 void *mychain(void *param){
208
 struct myfunctionparams *myparams = (struct myfunctionparams *)params;
209
 return (void *)myfunction(myparams->a, myparams->b, *myparams->d);
210
 }
211
212
 int myfunction(int a, char *b, struct dog d){
213
 if (Coroutine_GetStackHeadroom() < MIN_ALLOWED_STACK){
214
 struct myfunctionparams params = {
215
 a,
216
 b,
217
 &d
218
 };
219
 return (int)Coroutine_Chain(mychain, &params);
220
 }
221
 }
222
8 months ago
223
 # API
8 months ago
17
224
8 months ago
225
 ## Async
226
227
 The pattern for using async is:
228
229
 void *myasyncmaintask(void *param){
230
 // do your main async task things here, like starting more tasks
231
 }
232
233
 Async_StartSystem();
234
 void *res = NULL;
235
 bool canceled = Async_Run(myasyncmaintask, NULL, &res);
236
 Async_StopSystem();
237
238
 To create and wait for an async task:
239
240
 Async_Task task1;
241
 Async_Task_ctor(&task1, asynctask1, &task1param);
242
 void *res = NULL;
243
 bool canceled = Async_Task_Await(&task1, void **res)
244
 Async_Task_dtor(&task1);
245
246
 or, if you prefer new & delete:
247
248
 Async_Task *task1 = Async_Task_New(asynctask1, &task1param);
249
 void *res = NULL;
250
 bool canceled = Async_Task_Await(task1, void **res)
251
 Async_Task_Delete(task1);
252
253
 Inside your task, when there is something to wait for and you want other tasks to run while your task is waiting, you will need a future:
254
255
 Async_Future future;
256
 Async_Future_ctor(&future);
257
258
 // keep &future to hand for when the background thing completes
259
 bool canceled = Async_Future_Await(&future, NULL);
260
261
 Async_Future_dtor(&future);
262
263
 `Async_Future_New()` and `Async_Future_Delete()` are also available if you prefer that style.
264
265
 Inside the callback when the background thing is complete:
266
267
 // result is a void *
268
 Async_Future_SetResult(future, result, false);
269
270
 or, if something went wrong:
271
272
 // exception is a void *
273
 Async_Future_SetResult(future, exception, true);
274
275
 Back in the task, you can respond to the future:
276
277
 ... Async_Future_Await has returned
278
 if (canceled){
279
 // exit quickly - you've been canceled
280
 // you could, for example, use the future's result as an exception, or error code here
281
 }
282
 // carry on - the future's result may be an actual result, that's up to you
283
284
8 months ago
285
 #### `void Async_Future_ctor(Async_Future *fut)`
8 months ago
286
8 months ago
287
 fut
288
 : The `Async_Future` being constructed
289
8 months ago
290
 Initialise a future. When you no longer need it, use `Async_Future_dtor()`.
291
8 months ago
292
 #### `Async_Future *Async_Future_New()`
8 months ago
293
8 months ago
294
 ~returns~
295
 : The new future
8 months ago
296
8 months ago
297
 Allocates and initialises a future, When you no longer need it, use `Async_Future_Delete()`.
8 months ago
298
299
 #### `void Async_Future_dtor(Async_Future *fut)`
300
301
 fut
302
 : The `Async_Future` being destructed
303
304
 Destruct a future previously constructed with `Async_Future_ctor()`.
305
306
 ### void Async_Future_Delete(Async_Future *fut)
307
308
 Delete (finalise and free) a future previously new'ed with `Async_Future_New()`
309
310
 ### void Async_Future_SetResult(Async_Future *fut, bool canceled, void *value)
311
312
 Set the result of a future. This has an effect only the first time its done, ie a completed future can't be canceled and a canceled future can;t be completed.
313
314
 ### bool Async_Future_GetResult(Async_Future *fut, void **res)
315
316
 Get the result of a future. The return value is whether it was canceled. `res` may be `NULL`.
317
318
 ### typedef void (*Future_Watcher)(void *me, Async_Future *fut)
319
320
 A `Future_Watcher` is a callback called when a future is done. The `me` parameter is the one passed to `Async_Future_AddWatcher()`. `fut` is the future which has just completed.
321
322
 ### void Async_Future_AddWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
323
324
 Add a watcher (callback) to be called when the future is done. If the future is already complete, `watcher` is immediately called. The `me` value is passed to the watcher as its `me` parameter.
325
326
 ### void Async_Future_RemoveWatcher(Async_Future *fut, Future_Watcher watcher, void *me)
327
328
 Remove a watcher from a future.
329
330
 ### bool Async_Future_Await(Async_Future *fut, void **res)
331
332
333
8 months ago
334
 ## Generator
8 months ago
17
335
8 months ago
336
 The pattern for a `Generator` is:
337
338
 #### A loop which uses the `Generator`
339
8 months ago
17
340
 Generator gen;
8 months ago
341
 Generator_ctor(&gen, mygen, &param);
8 months ago
17
342
8 months ago
343
 void *value;
344
 while(Generator_Next(&gen, &value)){
345
 // use value here
8 months ago
17
346
 }
8 months ago
347
 // value is now the return value from the Generator
348
8 months ago
17
349
 Generator_dtor(&gen);
350
8 months ago
351
 Or:
8 months ago
17
352
8 months ago
353
 Generator *gen = Generator_New(mygen, &param);
8 months ago
17
354
8 months ago
355
 void *value;
356
 while(Generator_Next(gen, &value)){
357
 // use value here
358
 }
8 months ago
17
359
8 months ago
360
 Generator_Delete(gen);
8 months ago
17
361
8 months ago
362
 `Generator`s yield a series of `void *`s - what the `void *`s mean is up to you. `Generator_Next()` returns a `bool` to indicate whether the `Generator` has finished.
8 months ago
17
363
8 months ago
364
 #### A generator function
8 months ago
17
365
8 months ago
366
 void *mygen(void *param){
367
 bool domore = true;
368
 // The parameter is a pointer to a string of chars
369
 for (char *str = param; *str; ++str) {
370
 // The value yielded is a pointer to a character in the string
371
 domore = Generator_Yield(str);
372
 if (!domore){
373
 break;
374
 }
375
 }
376
377
 return (void *)domore;
8 months ago
17
378
 }
379
8 months ago
380
 The `bool` returned from `Generator_Yield()` indicates whether the generator function should yield more values. When it is `false` the `Generator` is being finalised - your generator function should close files, and release any other resources it has claimed, before exiting.
8 months ago
17
381
8 months ago
382
 ### void Generator_ctor(Generator *gen, void *(*start)(void *), void *param)
8 months ago
17
383
8 months ago
384
 Initialise a `Generator`
8 months ago
17
385
8 months ago
386
 Generator gen;
387
 Generator_ctor(&gen, mystart, &params);
8 months ago
17
388
8 months ago
389
 // Generator is used
8 months ago
17
390
8 months ago
391
 // ... later:
392
 Generator_dtor(&gen);
393
394
 ### Generator *Generator_New(void *(*)(void *), void *)
395
396
 `new` a `Generator` - malloc it and initialise it.
397
398
 Generator *gen = Generator_New(mystart, &params);
399
400
 // Generator is used
401
402
 // ... later:
403
 Generator_Delete(gen);
404
405
 ### void Generator_dtor(Generator *gen)
406
407
 Finalise a `Generator`. Once a `Generator` is no longer needed, it must be finalised:
408
409
 // earlier...
410
 Generator gen;
411
 Generator_ctor(&gen, mystart, &params);
412
413
 // Generator is used
414
415
 // the Generator is no longer needed
416
 Generator_dtor(&gen);
417
418
419
 ### void Generator_Delete(Generator *)
420
421
 Finalise then `free()` a `Generator`. Once a `new`ed `Generator` is no longer needed, it must be deleted:
422
423
 // earlier...
424
 Generator *gen = Generator_New(mystart, &params);
425
426
 // Generator is used
427
428
 // the Generator is no longer needed
429
 Generator_Delete(gen);
430
431
432
 ### bool Generator_Next(Generator *, void **value)
433
434
 Get the next value yielded by the `Generator`.
435
436
 void *value;
437
 while(Generator_Next(gen, &value)){
438
 // use value here
8 months ago
17
439
 }
8 months ago
440
8 months ago
441
 When `true` is returned, `value` is the value yielded by the `Generator`. When `false` is returned, `value` is the value returned when the `Generator` returned - the `Generator` has finished.
8 months ago
442
8 months ago
443
 ### bool Generator_Yield(void *)
444
445
 Yield a value from a `Generator` function.
446
447
 bool domore = Generator_Yield(value);
448
449
 `value` is then provided by `Generator_Next()` as the next value from the generator. The `bool` returned by `Generator_Yield()` indicates whether more values should be provided by your generator function. When `true` provide more values if possible. When `false` close files, free memory, free up any other resources and `return`. `false` is returned when the `Generator` is being finalised.
450
8 months ago
451
 ## Coroutine
452
453
 ### void Coroutine_StartSystem();
454
455
 Start the coroutine system on this thread. When you've finished with `Coroutine` call `Coroutine_Stop()`. `Coroutine` can be started & stopped many times on one thread. The total stack allowed for all coroutines running on a thread is the size of the call stack on that thread.
456
457
 ### void Coroutine_StopSystem();
458
459
 Stop the coroutine system on this thread.
460
461
 ### Coroutine_Start
462
463
 void *(*)(void *param)
464
8 months ago
465
 The entry function for a coroutine. The `param` is the value passed to `Coroutine_Continue`, and the `void *` return value can be accessed through the `Coroutine` object using `Coroutine_GetValue()`.
8 months ago
466
8 months ago
467
 ### Coroutine *Coroutine_New(Coroutine_Start start)
8 months ago
468
469
 Create a new `Coroutine`. The `Coroutine` system must be started to create a `Coroutine`. The stack size available to the coroutine will be `COROUTINE_STACK_SIZE` defined in `coroutine.h`. When you have finished with your `Coroutine`, use `Coroutine_Delete()` to delete it.
470
8 months ago
471
 ### void Coroutine_Run_Coroutine(Coroutine *cor, void *value)
8 months ago
472
473
 Run the `Coroutine` and return when it returns. This is how to start coroutines running in the coroutine system. It is an error for the run coroutine to return before all other coroutines have completed.
474
8 months ago
475
 ### void *Coroutine_Run(Coroutine_Start start, void *value)
8 months ago
476
477
 Convenience wrapper for `Coroutine_Run_Coroutine` which creates the `Coroutine` and retrieves its result.
478
8 months ago
479
 ### void Coroutine_Delete(Coroutine *cor)
8 months ago
480
481
 Use `Coroutine_Delete()` to delete a coroutine when it is no longer needed.
482
8 months ago
483
 ### void Coroutine_Continue(Coroutine *cor, void *value, bool early)
8 months ago
484
485
 Continue the given `Coroutine`. `value` is passed to the coroutine, as `param` to the `start` function, or as the return value from `Coroutine_Yield`. `early` determines whether the continued coroutine is added to the head of tail of the list of runable coroutines.
486
8 months ago
487
 ### void *Coroutine_Yield(void *value, Coroutine_YieldCallback on_yield, void *this)
8 months ago
488
489
 Yield `value` from the current coroutine; this coroutine is moved to the list of coroutines waiting to be continued. The next runable coroutine is run - either by its start routine being called with `value` as its `param`, or by `value`being returned from its `Coroutine_Yield()`.
490
8 months ago
491
 ### void *Coroutine_GetValue(Coroutine *cor)
8 months ago
492
493
 Return the `Coroutine`'s value - the value last yielded, or returned by its `start` routine.
494
8 months ago
495
 ### Coroutine *Coroutine_GetActive()
8 months ago
496
497
 Return whihc coroutine is currently running, ie the caller's `Coroutine`.
498
8 months ago
499
 ### bool Coroutine_IsRunning(Coroutine *cor)
8 months ago
500
501
 Return whether the given coroutine is still running - it may be running, ready to run, or waiting to be continued, but won't have returned from its `start` function.
8 months ago
502
503
 ### int Coroutine_GetStackHeadroom()
504
505
 Return the headroom available in the current coroutine's stack. This can be used to detect when your coroutine is nearing its stack limit, and then use `Coroutine_Chain()` to continue in a new chunk of coroutine stack.
506
507
 ### bool Coroutine_HasCoroutinesInFreePool()
508
509
 Return whether the coroutine system has any coroutine stacks available in its coroutine stack free pool.
510
511
 ### void *Coroutine_GetCStackTop()
512
513
 Return an address which is near to the top of used C stack.
514
515
 ### void *Coroutine_Chain(Coroutine_Start start, void *value)
516
517
 Run `start` with `value` on a new coroutine, and return its return value. It is expected that `Coroutine_Chain()` will be used when your coroutine is running short of stack - it is not an alternative to `Coroutine_Run()`.
518