GCC vs. VS2013 Compile Speed

Testing has shown that VS2013 (the MSVC compiler) appears to be much, much faster than GCC (actually g++). Compiling a set of programs with VS2013 takes 8 seconds. The same programs require 167 seconds to compile with g++. In both cases, pre-compiled headers are being used. Has anyone seen this before? Does anyone have any idea of why g++ would be so much slower? Am i missing some flag that would produce more comparable results?

Note that a debug build was done using VS2013 and -O0 was specified using g++. In some respects this comparison is biased against VS2013. The VS2013 build appears to be sequential (one file at a time). By contrast, the g++ build was done using -j 4. Removing -j 4 slowed the g++ build a lot (to 354 seconds).

The C++ source code is pretty vanilla. The code does use templates to some degree. However, the code is not crazy about them.