How to reduce the memory consumption of unordered_multiset?

I used unordered_multiset in my code by the following two reasons,

1. Should be easy to find or look up the data.
2. Should supports to load duplicate values.

unordered_multiset are typically much faster than multisets & vector, both for insertion and for lookup, and sometimes even for deletion.

But the bad thing is,it takes too much of memory.

I have stored unsigned __int64 (8 bytes)values in the unordered_multiset and properly clear the values from the unordered_multiset. can you anyone explain,why its taking the memory & how to solve this memory consumption?