What happens if I keep calling malloc() forever? 

Today is not a mood-brightening weather in Providence. So I decided to make fun of my computer. Exactly how I did it was by putting malloc within a gigantic loop. Each iteration it would ask OS to allocate a sizable amount of memory. I wondered what would happen. Good question, right?

Here is how the program looks like:

int main(int argc, char** argv) {
    const size_t NUM_MALLOCS = LONG_MAX; // 2^31-1, at least.
    long pageSize = sysconf(_SC_PAGE_SIZE);  // Number of bytes to allocate for each malloc() call.
    vector<void*> mallocPtrs;

    for (size_t i = 0; i < NUM_MALLOCS; ++i) {
        void* p = malloc(static_cast<size_t>(pageSize));
        if (p == NULL) {
            // Malloc failed at total size: i * pageSize.
            cleanUp(mallocPtrs);  // Free the allocated memory.
            return 0;
        }  else {
            mallocPtrs.push_back(p);
        }
    }

    cleanUp(mallocPtrs);
    return 0;
}

In theory there’s one outcome I can think of. What the malloc() function does essentially is simply to add some rooms for the process’ data segment to grow, and this increment happens in the heap portion of the virtual memory space. So if we malloc too many times, we might end up having the process’ data segment trespass into the stack portion of virtual memory space (because heap grows low-to-high, and stack grows high-to-low address-wise). This does strike the chord of the notorious segmentation fault, aye?

In fact, when the program runs, it doesn’t sour up to 100% memory usage and seg-fault immediately afterwards. The memory utilization goes slower and slower towards 80%, and stays at around 81%, and I notice that the swap space starts to utilize some of its free space. 

It seems that the OS starts to heavily swap cold physical memory pages, both dirty and clean, to the swap space located in my disk. Incredible slow down on my machine as well, as I try to switch terminal screen sessions or to Chrome.

I kill the processes after a while, before the swap space runs out (which I shouldn’t, because then I’d have been able to see the legendary OOM-killer (out-of-memory killer) come out and kill random processes!).

Now I notice a huge vacuum in the free memory space:

sam@butler$ /usr/bin/procinfo
Memory:        Total        Used        Free     Buffers
RAM:         4060612      833360     3227252        1016
Swap:        8388604     2880484     5508120

And switching to Chrome and terminal is still taking ages… Probably because the pages are still in swap space.

I start browsing using Chrome, despite the slowness. And I see it slowly regains the speed among switching tabs, because the swapped pages have returned to the physical memory:

sam@butler$ /usr/bin/procinfo
Memory:        Total        Used        Free     Buffers
RAM:         4060612     1395148     2665464        2568
Swap:        8388604     2625552     5763052