Mar 9

When the April 2003 issue of the C/C++ Users Journal arrived in my mailbox, I quickly noticed that this would be yet another issue dedicated to one platform.

It was nice to see that this issue did not contain a majority of articles specific to the Windows environment. But I don't think it is any better that this issue is almost completely dedicated to a Linux environment either.

One of the "featured" articles on the cover that did catch my eye was "STL Containers in Shared Memory". It sounded interesting, but not enough to make me turn directly to it. Instead, the magazine got thrown on my desk waiting for some free time to be read.

It was not until I had a short conversation with Isaac that I decided to pick it up. He mentioned that the concept of shared STL containers was interesting, and that after he read the article, it might be fun to implement something similar on Windows.

I then decided to take a few minutes and read the article. Not only did I read it once, but over the next few days I read it over and over. I was sure that I was missing something. I even downloaded the source code from the CUJ website and tried to compile and understand it.

Out of all the CUJ articles that I have read, this has to be the worst one ever. I was even more surprised when I read the author's mini-bio. It states that Grum Ketema has been using C++ since 1988, has worked at AT&T Bell Labs, TASC, MIT, SWIFT, BEA and Northrop. I can only hope that none of these companies are using this "shared memory allocator".

For those who have not had the chance to read the article, it is best summarized by this excerpt:

"Imagine placing STL containers, such as maps, vectors, lists, etc., in shared memory. Placing such powerful generic data structures in shared memory equips processes using shared memory for IPC with a powerful tool."

The article goes on to explain that a special purpose allocator can be written so that STL containers can be shared between processes. This special purpose allocator can then be used as a template argument to any of the STL containers. An example is given:

vector<int, myAlloc<int> >;

Where myAlloc<> is the special purpose allocator that will allocate memory from a shared memory pool.

This article has so many problems that there is no way I could cover them all. Specifically, I won't cover the fact that the code listings have numerous syntax errors, or that the source code includes files such as fstream.h, or even that all the standard containers are assumed to be in the global namespace. You should also watch out for statements in the article such as the one saying that the new operator allocates memory from the heap.

The two major problems that I see with the article are:

1. this code is very dangerous in the real world, and
2. I don't think that sharing containers is a very good idea

As I quoted above, the author states that this special purpose allocator will work with all STL containers. This could not be further from the truth. The fact is, std::list and all the associative containers (std::map, std::multimap, std::set, std::multiset) will not even use the allocator, but instead use the new operator. More on this can be found in "Effective STL" by Scott Meyers.

So, if you use this special purpose allocator with any container other than std::vector or std::deque, the container will allocate memory from the free store, not shared memory. The problems with that are obvious.

In addition, you would have to be very careful about what you place in your std::vector or std::deque that is using this special purpose allocator. For example, can you see the problem with this example:

std::vector<std::string, myAlloc<std::string> > v;

The std::string objects would be created in shared memory, and placed inside the vector, but the memory that std::string would need to allocate in order store its character array would be taken from the free store. This could be worked around, but it would require that you do the following:

typedef std::basic_string<char, std::char_traits<char>, myAlloc<char> > shared_string;
std::vector<shared_string, myAlloc<shared_string> > v;

This work around does not work for most other user defined types, however. Don't even try to create a shared vector for classes that use the pimpl idiom, or in any other way allocate memory from the free store.

Even if you do consider using the author's special purpose allocator, you must be careful how you use it. It should be clear from the above text that you can use this allocator with std::vector<int>. Looks pretty safe doesn't it?

Be warned, the implementation that they author used for the article does not allow the memory pool to grow. If you try to push_back() too many int's, well, you get the idea.

With all those problems aside, why would you want to share containers using shared memory anyways? You might be accustomed to thinking of encapsulation on the class level, but I think it equally applies to the library and application level.

If you share any part of your memory, especially a standard container, with another process, you are losing a lot of abstraction and incurring a lot of coupling.

It would be far better, although possibly a little slower, if you implemented a protocol to use over a local domain socket or a pipe. At first, you may only see the abstraction benefit, but rather quickly you see how this solution scales better too.

What happens when you want yet another application be able to access some internal state information from the host application? With shared memory you have many limitations, like the requirement that the two processes be on the same physical machine. Not to mention an explosion of coupling. Imagine what would happen if you wanted to change from a std::vector to a std::deque!

If you are using a protocol over a local domain socket, it would be trivial to change the host application to use full TCP/IP. Now applications running anywhere on the network can get the information from the host application.

This offers much better scalability, and you can even change from a std::vector to a std::deque without effecting any code except that in the host application.

I am not only shocked that such a poor article made it into CUJ, but that such a bad idea made it in to CUJ, especially from someone with so much software engineering experience.

Let this be a lesson to anyone who is in a position to hire a software engineer. Someone with 17 years of experience, multiple Masters degrees and an impressive work history does not guarantee much at all, if anything.