C++ / Linux: Temporarily Storing Decrypted Data in Memory

Background

I am working on a simple application in native C++11 which uses Microsoft's Cassablanca REST framework to communicate with various remote endpoints. To do this, I have to cryptographically sign the various requests with some secret provided by whichever API is being targeted. By linking Cassablanca I already have OpenSSL so the actual crypto is reasonably straight-forward.

Now, the tricky part is how to store the secrets themselves. My plan is to AES encrypt a JSON file with all of the goodies using a symmetric key from the user's brain (read: password) so that it does not have to be stored on disk.

The Question

What is a secure way to store the decrypted secrets in memory during the execution of the application? This is not another "how do I store secrets on disk so the user doesn't have to do anything" question.

My big concern is that someone will be able to examine my application's memory externally and find the appropriate regions containing the goodies. I know Windows provides credstores and protected memory however I am on Linux and would prefer a platform-agnostic solution if possible. There is also a significant chance that this will be deployed to a VM cluster so protected memory may only buy limited assurances (though, still better than nothing).

Finally, I know that any cryptographic system can be overcome with sufficiently large amount of money+time or a large enough rubber hose. That being said, I would still like to be as secure as possible as an exercise in Good™ coding practice.

My constraints are:

• Must be able to access secrets for signing the various requests quickly. Milliseconds count here but I am willing to accept a slowdown if security is truly improved.

• I would prefer a minimum of external libraries and to use native C++11 where possible. I feel like this should be doable with what I already have (Cassablanca which requires OpenSSL, Boost, libcrypt).

• Finally, it should be able to accommodate a reasonably large number of secrets / requests. My application domain is basically a giant parallelization problem and scalability is key.

What I already know to do:

• Never ever store sensitive material on disk unencrypted. This includes usage of disk-backed containers and whatnot.

• An old but useful trick: attach a debugger instance to prevent remote debugging from external process.

• Build without debug symbols and strip final executable.

• Minimize in-application access to data to only the parts which truly need it.

• Destroy sensitive information as soon as it is no longer needed (Ex. NULL out user password as soon as decryption is finished, etc).

• Do not trust anybody to sanitize memory for me. Manually overwrite areas with NULL / garbage.

Similar Questions

• Storing sensitive data in memory

PHP specific and advises writing something not in PHP to store the credentials...which is exactly the problem I am trying to solve.

• Store password temporarily in memory

Android application lifecycle specific. I already know the storage duration for my secrets will be the lifetime of the application.