This post compares the stack used by the primordial thread, also called the main thread, with the stack used by a thread created through the pthread API. On Linux, the two types of stacks use different memory mappings.

Any mapping has three related but largely independent metrics:

Term What it means Where to read it
Reserve (virtual) Address space the kernel has set aside Size: in /proc/<pid>/smaps
Committed Memory the kernel has accounted against the overcommit limit Committed_AS globally, or ac in a mapping’s VmFlags
Resident (RSS) Physical pages currently resident in RAM Rss: in smaps

Memory is usually reserved first, then committed, and finally made resident when written to.

§Main thread vs. pthread stacks

The main thread’s stack is the kernel’s [stack] mapping. It is not a fixed-size mmap. The kernel extends the mapping toward lower virtual addresses on demand as alloca grows the stack and the program writes to new pages. Its reservation and commit charge grow only as far as the stack is used, so Reserve == Committed, and both increase after the alloca call.

A pthread stack is a fixed-size mmap that glibc creates with the thread. In this experiment, glibc reserves and commits the full 8 MiB immediately, in contrast to the main thread’s lazy reservation and commit.

For both types of threads, resident memory is usually much smaller than the committed size because threads often do not touch all the memory they request.

§The experiment

The program below reports the main thread’s stack mapping, grows the stack with alloca, writes to the allocated memory to extend the [stack] mapping, and then reports the mapping again. It repeats the process in a newly created thread. To find a thread’s stack in smaps, the program takes the address of a local variable and selects the region that contains it.

§The program

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#include <alloca.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define PAGE 4096

/* Locate this thread's stack region in smaps. An "ac" flag marks the VMA as
   accountable, so its full Size counts toward committed memory. */
static void dump_stack(const char* label) {
  int marker;
  void* sp = &marker;
  char path[64];
  snprintf(path, sizeof path, "/proc/%d/smaps", getpid());

  FILE* fp = fopen(path, "r");
  if (!fp) {
    perror("smaps");
    return;
  }

  char line[512];
  int in = 0, accountable = 0;
  unsigned long sz = 0, rss = 0, start, end;

  while (fgets(line, sizeof line, fp)) {
    if (sscanf(line, "%lx-%lx", &start, &end) == 2) {
      if (in)
        break; /* entered the next region */
      in = (unsigned long)sp >= start && (unsigned long)sp < end;
    } else if (in) {
      unsigned long v;
      if (sscanf(line, "Size: %lu", &v) == 1)
        sz = v;
      else if (sscanf(line, "Rss: %lu", &v) == 1)
        rss = v;
      if (strstr(line, "VmFlags:") && strstr(line, " ac"))
        accountable = 1;
      if (line[0] == '\n')
        break;
    }
  }
  fclose(fp);
  printf("%-28s : Reserve %lu kB | Committed %ld kB | Resident %lu kB\n", label,
         sz, accountable ? (long)sz : 0L, rss);
}

static void expand_stack(size_t bytes) {
  volatile char* buf = alloca(bytes);
  for (size_t i = 0; i < bytes; i += PAGE)
    buf[i] = (char)(i / PAGE);
  buf[bytes - 1] = 1;
}

/* Shared by both threads: dump, grow the stack, dump again. */
static void run_demo(const char* fresh, const char* after) {
  const size_t expand = 1 * 1024 * 1024; /* growth applied to both threads */
  dump_stack(fresh);
  expand_stack(expand);
  dump_stack(after);
}

static void* thread_fn(void* arg) {
  (void)arg;
  run_demo("pthread stack (fresh)", "pthread stack (after touch)");
  return NULL;
}

int main(void) {
  run_demo("main stack (fresh)", "main stack (after expand)");

  pthread_t t;
  pthread_create(&t, NULL, thread_fn, NULL);
  pthread_join(t, NULL);
  return 0;
}

§Observed output

1
2
3
4
main stack (fresh)           : Reserve 136 kB | Committed 136 kB | Resident 16 kB
main stack (after expand)    : Reserve 1036 kB | Committed 1036 kB | Resident 1036 kB
pthread stack (fresh)        : Reserve 8192 kB | Committed 8192 kB | Resident 8 kB
pthread stack (after touch)  : Reserve 8192 kB | Committed 8192 kB | Resident 1032 kB