Having been playing with ucontext lately, and I would like share my experience on it. The spec I have for it is the one from open group, makecontext, and the Linux manual page, setcontext, which probably doesn’t apply for Mac.

§Minimal stack size

Firstly, let try the example from open group. The relevant change I did is to replace the stack size with one macro so that we don’t need to change two places if we want a different stack size.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
static void
f1 (void)
{
    puts("start f1");
    swapcontext(&ctx[1], &ctx[2]);
    puts("finish f1");
}

static void
f2 (void)
{
    puts("start f2");
    swapcontext(&ctx[2], &ctx[1]);
    puts("finish f2");
}

static void
test_f1f2_main()
{
    char st1[SSIZE];
    char st2[SSIZE];

    printf("%p\n", &st1);
    getcontext(&ctx[1]);
    ctx[1].uc_stack.ss_sp = st1;
    ctx[1].uc_stack.ss_size = sizeof st1;
    ctx[1].uc_link = &ctx[0];
    makecontext(&ctx[1], f1, 0);


    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st2;
    ctx[2].uc_stack.ss_size = sizeof st2;
    ctx[2].uc_link = &ctx[1];
    makecontext(&ctx[2], f2, 0);

    swapcontext(&ctx[0], &ctx[2]);
}

After reasoning, we expect the output to be:

start f2
start f1
finish f2
finish f1

Fortunately, that’s exactly I obtained on Linux. However, it hangs on Mac, which is due to the small stack size. The minimal size on Mac is defined in signal.h as MINSIGSTKSZ, so let just do that in the beginning.

#define SSIZE MINSIGSTKSZ // minimal 32K for Mac; 2K for Linux

After this, it works like a charm on both platforms. One SO question was asked due to this stack size problem.

I shall just quote the relevant paragraph from open group:

The uc_link member is used to determine the context that shall be resumed when
the context being modified by makecontext() returns.

The first example we saw has proved that when it’s used with makecontext, it indicates the resuming context. Then, does uc_link also work with contexts that are not modified by makecontext? If we understand the spec correctly, the answer should be no.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
static void
test_uc_link (void)
{
    flag = 1;
    puts("start test_setcontext");
    if(swapcontext(&ctx[1], &ctx[3])) {
        puts("err");
    };
    puts("finish test_setcontext");
}

static void
test_uc_link_main()
{
    char st[SSIZE];

    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st;
    ctx[2].uc_stack.ss_size = sizeof st;
    ctx[2].uc_link = &ctx[0];
    makecontext(&ctx[2], test_uc_link, 0);

    getcontext(&ctx[3]);
    puts("ctx3");
    if(flag) {
        ctx[1].uc_link = &ctx[3];
        if(swapcontext(&ctx[4], &ctx[1])) {
            puts("err");
        };
        puts("ctx4");
    } else {
        if(swapcontext(&ctx[0], &ctx[2])) {
            puts("err");
        };
    }
    puts("over");
}

The output for Linux:

1
2
3
4
5
ctx3
start test_setcontext
ctx3
finish test_setcontext
over

and the output for Mac:

1
2
3
4
5
6
ctx3
start test_setcontext
ctx3
finish test_setcontext
ctx4
over

The output is different, but they remain the same independent of whether we set uc_link or not. Therefore, we could say uc_link only works with contexts modified by makecontext. The different output from two platforms is caused by the fact that it’s undefined behavior for contexts that created by setcontext or swapcontext reaches the end.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
static void
test_uc_link_2 (void)
{
    flag = 1;
    puts("start test_setcontext");
    if(swapcontext(&ctx[2], &ctx[3])) {
        puts("err");
    };
    puts("finish test_setcontext");
}


static void
test_uc_link_2_main()
{
    char st[SSIZE];

    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st;
    ctx[2].uc_stack.ss_size = sizeof st;
    ctx[2].uc_link = &ctx[0];
    makecontext(&ctx[2], test_uc_link_2, 0);

    getcontext(&ctx[3]);
    puts("ctx3");
    if(!flag) {
        if(swapcontext(&ctx[0], &ctx[2])) {
            puts("err");
        };
    } else {
        ctx[2].uc_link = &ctx[4];
        if(swapcontext(&ctx[4], &ctx[2])) {
            puts("err");
        };
        puts("ctx4");
    }
    puts("over");
}

The output for Linux:

1
2
3
4
5
ctx3
start test_setcontext
ctx3
finish test_setcontext
over

and the output for Mac:

1
2
3
4
5
6
ctx3
start test_setcontext
ctx3
finish test_setcontext
ctx4
over

Therefore, Mac is more flexible on this topic. The doc doesn’t say anything about this, so I had to go to the source code to understand the behavior. makecontext for Linux uses uc_link info when the makecontext is called; it’s stored in the stack already. While, in Mac, makecontext registers callback to use uc_link when context is finished, so we could change uc_link after the makecontext call.

§Reusing saved context multiple times

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
static void
test_reusing (void)
{
    puts("start test_setcontext");
    if(swapcontext(&ctx[1], &ctx[0])) {
        puts("err");
    };
    puts("finish test_setcontext");
}


static void
test_reusing_main()
{
    char st[SSIZE];
    int counter = 0;

    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st;
    ctx[2].uc_stack.ss_size = sizeof st;
    ctx[2].uc_link = &ctx[0];
    makecontext(&ctx[2], test_reusing, 0);

    if(swapcontext(&ctx[0], &ctx[2])) {
        puts("err");
    };
    if (counter > 0) {
        return;
    }
    puts("btw");
    counter++;
    setcontext(&ctx[0]);
    puts("over");
}

ctx[0] is entered twice, first, the swapcontext inside test_reusing, second, the setcontext in main.

The output for Linux:

1
2
start test_setcontext
btw

and the output for Mac:

1
2
3
start test_setcontext
btw
over

The Linux output is the one I am expecting. I was so surprised to see ‘over’ on my stdout.

The source of swapcontext looks quite innocent, and I was stuck here for a very long time. Having no idea where the problem lies, I decided to re-implement swapcontext after the source code from Apple.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
static int myswapcontext(ucontext_t *old, ucontext_t *new)
{
    volatile int flag = 1;
    int ret;
    ret = getcontext(old);
    if (flag) {
        flag = 0;
        ret = setcontext(new);
    }
    return ret;
}

static void
test_wrapper_main()
{
    char st[SSIZE];
    int counter = 0;

    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st;
    ctx[2].uc_stack.ss_size = sizeof st;
    ctx[2].uc_link = &ctx[0];
    makecontext(&ctx[2], test_reusing, 0);

    if(myswapcontext(&ctx[0], &ctx[2])) {
        puts("err");
    };
    if (counter > 0) {
        return;
    }
    puts("btw");
    counter++;
    setcontext(&ctx[0]);
    puts("over");
}

This time the output for both platforms “shifted” a bit; for Linux:

1
2
3
start test_setcontext
btw
over

for Mac:

1
2
3
start test_setcontext
btw
zsh: segmentation fault  ./a.out

Apparently, this naive implement of swapcontext is wrong. I kindly of understood why swapcontext on Linux is implemented like this. (I don’t know assembly; hope I guessed it right.)

§What I am trying to do

I just want some basic cooperative scheduling for coroutine. The original problem could be simplified as the following snippets:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
static void
test_coroutine (void)
{
    puts("start test_setcontext");
    if(swapcontext(&ctx[1], &ctx[0])) {
        puts("err");
    };
    puts("finish test_setcontext");
    // setcontext(&ctx[3]);
}


static void
test_coroutine_main()
{
    char st[SSIZE];
    int counter = 0;

    getcontext(&ctx[2]);
    ctx[2].uc_stack.ss_sp = st;
    ctx[2].uc_stack.ss_size = sizeof st;
    ctx[2].uc_link = &ctx[0];
    makecontext(&ctx[2], test_coroutine, 0);

    if(swapcontext(&ctx[0], &ctx[2])) {
        puts("err");
    };
    puts("btw");
    if(swapcontext(&ctx[3], &ctx[1])) {
        puts("err");
    };
    puts("over");
}

For Linux the output is the following, with exit status 23.

1
2
3
4
5
start test_setcontext
btw
finish test_setcontext
btw
finish test_setcontext

For Mac, due to the buggy implementation of swapcontext, it actually works:

1
2
3
4
start test_setcontext
btw
finish test_setcontext
over

If you understand everything so far, it’s quite obvious that the solution is just to add one explicit context switch in the end of test_coroutine, uncommenting the commented statement.