Comments on: mmap()

By: Aistis

Aistis — Thu, 21 Aug 2008 15:42:52 +0000

visiskai nerealus thread’as lkml’e. aciu ;))

By: Domas Mituzas

Domas Mituzas — Tue, 19 Aug 2008 08:12:49 +0000

Indeed, there’re multiple cases when mmap() can be helpful, if used correctly – but there’re many ways to make mistakes too.

Software has to be really helping the OS with madvise() – though the interface is quite limited (it is just extreme flushing of cache, not reducing priorities).

By: Anonymouse

Anonymouse — Mon, 18 Aug 2008 15:43:28 +0000

Just a clarification: With the default options, mmap() does not allocate the pages until you access them, so the memory pressure is really more related to the working set size, not the file size.

mmap() really wins when you can use the data directly, without processing it, and due to the secondary effects on the file cache (if you read() data and put that into anonymous pages, it may force pages out of the file cache, which does not happen with mmap).

In some cases it also eases development, since mmap()’ing data is almost zero-cost when the data is already in the cache, so you can save on initialization time.

By: Mark R

Mark R — Mon, 18 Aug 2008 09:06:56 +0000

The case where mmap() is really good is where we can reduce the number of copies of data made with it. The MyISAM key buffer is an example where I tried – not very successfully – to use it.

If we can get/put data directly from a mmap()’d location, instead of copying it from an OS buffer (as with pread / pwrite) then we save a copy operation.

If, of course, the code is written so that it assumes it has a private copy of the page to mess around with (I’m pretty sure MyISAM DOES do this), then you have to copy it anyway (e.g. with memcpy) which pretty much negates the advantage (I expect).

My performance data from mmap()ing MyISAM keys suggested that the benefits were modest anyway.

Mark