C++ FAQ Celebrating Twenty-One Years of the C++ FAQ!!!
(Click here for a personal note from Marshall Cline.)
Section 11:
[11.14] Is there a way to force new to allocate memory from a specific memory area?

Yes. The good news is that these "memory pools" are useful in a number of situations. The bad news is that I'll have to drag you through the mire of how it works before we discuss all the uses. But if you don't know about memory pools, it might be worthwhile to slog through this FAQ — you might learn something useful!

First of all, recall that a memory allocator is simply supposed to return uninitialized bits of memory; it is not supposed to produce "objects." In particular, the memory allocator is not supposed to set the virtual-pointer or any other part of the object, as that is the job of the constructor which runs after the memory allocator. Starting with a simple memory allocator function, allocate(), you would use placement new to construct an object in that memory. In other words, the following is morally equivalent to new Foo():

void* raw = allocate(sizeof(Foo));  // line 1
Foo* p = new(raw) Foo();            // line 2
Okay, assuming you've used placement new and have survived the above two lines of code, the next step is to turn your memory allocator into an object. This kind of object is called a "memory pool" or a "memory arena." This lets your users have more than one "pool" or "arena" from which memory will be allocated. Each of these memory pool objects will allocate a big chunk of memory using some specific system call (e.g., shared memory, persistent memory, stack memory, etc.; see below), and will dole it out in little chunks as needed. Your memory-pool class might look something like this:
class Pool {
public:
  void* alloc(size_t nbytes);
  void dealloc(void* p);
private:
  ...data members used in your pool object...
};

void* Pool::alloc(size_t nbytes)
{
  ...your algorithm goes here...
}

void Pool::dealloc(void* p)
{
  ...your algorithm goes here...
}
Now one of your users might have a Pool called pool, from which they could allocate objects like this:
Pool pool;
...
void* raw = pool.alloc(sizeof(Foo));
Foo* p = new(raw) Foo();
Or simply:
Foo* p = new(pool.alloc(sizeof(Foo))) Foo();
The reason it's good to turn Pool into a class is because it lets users create N different pools of memory rather than having one massive pool shared by all users. That allows users to do lots of funky things. For example, if they have a chunk of the system that allocates memory like crazy then goes away, they could allocate all their memory from a Pool, then not even bother doing any deletes on the little pieces: just deallocate the entire pool at once. Or they could set up a "shared memory" area (where the operating system specifically provides memory that is shared between multiple processes) and have the pool dole out chunks of shared memory rather than process-local memory. Another angle: many systems support a non-standard function often called alloca() which allocates a block of memory from the stack rather than the heap. Naturally this block of memory automatically goes away when the function returns, eliminating the need for explicit deletes. Someone could use alloca() to give the Pool its big chunk of memory, then all the little pieces allocated from that Pool act like they're local: they automatically vanish when the function returns. Of course the destructors don't get called in some of these cases, and if the destructors do something nontrivial you won't be able to use these techniques, but in cases where the destructor merely deallocates memory, these sorts of techniques can be useful.

Okay, assuming you survived the 6 or 8 lines of code needed to wrap your allocate function as a method of a Pool class, the next step is to change the syntax for allocating objects. The goal is to change from the rather clunky syntax new(pool.alloc(sizeof(Foo))) Foo() to the simpler syntax new(pool) Foo(). To make this happen, you need to add the following two lines of code just below the definition of your Pool class:

inline void* operator new(size_t nbytes, Pool& pool)
{
  return pool.alloc(nbytes);
}
Now when the compiler sees new(pool) Foo(), it calls the above operator new and passes sizeof(Foo) and pool as parameters, and the only function that ends up using the funky pool.alloc(nbytes) method is your own operator new.

Now to the issue of how to destruct/deallocate the Foo objects. Recall that the brute force approach sometimes used with placement new is to explicitly call the destructor then explicitly deallocate the memory:

void sample(Pool& pool)
{
  Foo* p = new(pool) Foo();
  ...
  p->~Foo();        // explicitly call dtor
  pool.dealloc(p);  // explicitly release the memory
}
This has several problems, all of which are fixable:
  1. The memory will leak if Foo::Foo() throws an exception.
  2. The destruction/deallocation syntax is different from what most programmers are used to, so they'll probably screw it up.
  3. Users must somehow remember which pool goes with which object. Since the code that allocates is often in a different function from the code that deallocates, programmers will have to pass around two pointers (a Foo* and a Pool*), which gets ugly fast (example, what if they had an array of Foos each of which potentially came from a different Pool; ugh).

We will fix them in the above order.

Problem #1: plugging the memory leak. When you use the "normal" new operator, e.g., Foo* p = new Foo(), the compiler generates some special code to handle the case when the constructor throws an exception. The actual code generated by the compiler is functionally similar to this:

// This is functionally what happens with Foo* p = new Foo()

Foo* p;

// don't catch exceptions thrown by the allocator itself
void* raw = operator new(sizeof(Foo));

// catch any exceptions thrown by the ctor
try {
  p = new(raw) Foo();  // call the ctor with raw as this
}
catch (...) {
  // oops, ctor threw an exception
  operator delete(raw);
  throw;  // rethrow the ctor's exception
}
The point is that the compiler deallocates the memory if the ctor throws an exception. But in the case of the "new with parameter" syntax (commonly called "placement new"), the compiler won't know what to do if the exception occurs so by default it does nothing:
// This is functionally what happens with Foo* p = new(pool) Foo():

void* raw = operator new(sizeof(Foo), pool);
// the above function simply returns "pool.alloc(sizeof(Foo))"

Foo* p = new(raw) Foo();
// if the above line "throws", pool.dealloc(raw) is NOT called
So the goal is to force the compiler to do something similar to what it does with the global new operator. Fortunately it's simple: when the compiler sees new(pool) Foo(), it looks for a corresponding operator delete. If it finds one, it does the equivalent of wrapping the ctor call in a try block as shown above. So we would simply provide an operator delete with the following signature (be careful to get this right; if the second parameter has a different type from the second parameter of the operator new(size_t, Pool&), the compiler doesn't complain; it simply bypasses the try block when your users say new(pool) Foo()):
void operator delete(void* p, Pool& pool)
{
  pool.dealloc(p);
}
After this, the compiler will automatically wrap the ctor calls of your new expressions in a try block:
// This is functionally what happens with Foo* p = new(pool) Foo()

Foo* p;

// don't catch exceptions thrown by the allocator itself
void* raw = operator new(sizeof(Foo), pool);
// the above simply returns "pool.alloc(sizeof(Foo))"

// catch any exceptions thrown by the ctor
try {
  p = new(raw) Foo();  // call the ctor with raw as this
}
catch (...) {
  // oops, ctor threw an exception
  operator delete(raw, pool);  // that's the magical line!!
  throw;  // rethrow the ctor's exception
}
In other words, the one-liner function operator delete(void* p, Pool& pool) causes the compiler to automagically plug the memory leak. Of course that function can be, but doesn't have to be, inline.

Problems #2 ("ugly therefore error prone") and #3 ("users must manually associate pool-pointers with the object that allocated them, which is error prone") are solved simultaneously with an additional 10-20 lines of code in one place. In other words, we add 10-20 lines of code in one place (your Pool header file) and simplify an arbitrarily large number of other places (every piece of code that uses your Pool class).

The idea is to implicitly associate a Pool* with every allocation. The Pool* associated with the global allocator would be NULL, but at least conceptually you could say every allocation has an associated Pool*. Then you replace the global operator delete so it looks up the associated Pool*, and if non-NULL, calls that Pool's deallocate function. For example, if(!) the normal deallocator used free(), the replacment for the global operator delete would look something like this:

void operator delete(void* p)
{
  if (p != NULL) {
    Pool* pool = /* somehow get the associated 'Pool*' */;
    if (pool == NULL)
      free(p);
    else
      pool->dealloc(p);
  }
}
If you're not sure if the normal deallocator was free(), the easiest approach is also replace the global operator new with something that uses malloc(). The replacement for the global operator new would look something like this (note: this definition ignores a few details such as the new_handler loop and the throw std::bad_alloc() that happens if we run out of memory):
void* operator new(size_t nbytes)
{
  if (nbytes == 0)
    nbytes = 1;  // so all alloc's get a distinct address
  void* raw = malloc(nbytes);
  ...somehow associate the NULL 'Pool*' with 'raw'...
  return raw;
}
The only remaining problem is to associate a Pool* with an allocation. One approach, used in at least one commercial product, is to use a std::map<void*,Pool*>. In other words, build a look-up table whose keys are the allocation-pointer and whose values are the associated Pool*. For reasons I'll describe in a moment, it is essential that you insert a key/value pair into the map only in operator new(size_t,Pool&). In particular, you must not insert a key/value pair from the global operator new (e.g., you must not say, poolMap[p] = NULL in the global operator new). Reason: doing that would create a nasty chicken-and-egg problem — since std::map probably uses the global operator new, it ends up inserting a new entry every time inserts a new entry, leading to infinite recursion — bang you're dead.

Even though this technique requires a std::map look-up for each deallocation, it seems to have acceptable performance, at least in many cases.

Another approach that is faster but might use more memory and is a little trickier is to prepend a Pool* just before all allocations. For example, if nbytes was 24, meaning the caller was asking to allocate 24 bytes, we would allocate 28 (or 32 if you think the machine requires 8-byte alignment for things like doubles and/or long longs), stuff the Pool* into the first 4 bytes, and return the pointer 4 (or 8) bytes from the beginning of what you allocated. Then your global operator delete backs off the 4 (or 8) bytes, finds the Pool*, and if NULL, uses free() otherwise calls pool->dealloc(). The parameter passed to free() and pool->dealloc() would be the pointer 4 (or 8) bytes to the left of the original parameter, p. If(!) you decide on 4 byte alignment, your code would look something like this (although as before, the following operator new code elides the usual out-of-memory handlers):

void* operator new(size_t nbytes)
{
  if (nbytes == 0)
    nbytes = 1;                    // so all alloc's get a distinct address
  void* ans = malloc(nbytes + 4);  // overallocate by 4 bytes
  *(Pool**)ans = NULL;             // use NULL in the global new
  return (char*)ans + 4;           // don't let users see the Pool*
}

void* operator new(size_t nbytes, Pool& pool)
{
  if (nbytes == 0)
    nbytes = 1;                    // so all alloc's get a distinct address
  void* ans = pool.alloc(nbytes + 4); // overallocate by 4 bytes
  *(Pool**)ans = &pool;            // put the Pool* here
  return (char*)ans + 4;           // don't let users see the Pool*
}

void operator delete(void* p)
{
  if (p != NULL) {
    p = (char*)p - 4;              // back off to the Pool*
    Pool* pool = *(Pool**)p;
    if (pool == NULL)
      free(p);                     // note: 4 bytes left of the original p
    else
      pool->dealloc(p);            // note: 4 bytes left of the original p
  }
}
Naturally the last few paragraphs of this FAQ are viable only when you are allowed to change the global operator new and operator delete. If you are not allowed to change these global functions, the first three quarters of this FAQ is still applicable.