2

I'm writing some code that handles cryptographic secrets, and I've created a custom ZeroedMemory implementation of std::pmr::memory_resource which handles sanitizes memory on deallocation and encapsulates using the magic you have to use to prevent optimizing compilers from eliding away the operation. The idea was to avoid specializing std::array, because the lack of a virtual destructor means that destruction after type erasure would cause memory to be freed without being sanitized.

Unfortunately, I came to realize afterwards that std::array isn't an AllocatorAwareContainer. My std::pmr::polymorphic_allocator approach was a bit misguided, since obviously there's no room in an std::array to store a pointer to a specific allocator instance. Still, I can't fathom why allocators for which std::allocator_traits<A>::is_always_equal::value == true wouldn't be allowed, and I could easily re-implement my solution as a generic Allocator instead of the easier-to-use std::pmr::memory_resource...

Now, I could normally just use an std::pmr::vector instead, but one of the nice features of std::array is that the length of the array is part of the type. If I'm dealing with a 32-byte key, for example, I don't have to do runtime checks to be sure that the std::array<uint8_t, 32> parameter someone passed to my function is, in fact, the right length. In fact, those cast down nicely to a const std::span<uint8_t, 32>, which vastly simplifies writing functions that need to interoperate with C code because they enable me to handle arbitrary memory blocks from any source basically for free.

Ironically, std::tuple takes allocators... but I shudder to imagine the typedef needed to handle a 32-byte std::tuple<uint8_t, uint8_t, uint8_t, uint8_t, ...>.

So: is there any standard-ish type that holds a fixed number of homogenously-typed items, a la std::array, but is allocator aware (and preferably stores the items in a continguous region, so it can be down-cast to an std::span)?

Reid Rankin
  • 1,078
  • 8
  • 26
  • 4
    `std::array` doesn't allocate any memory on the heap, and so has no need for an allocator. It also needs to be an aggregate, and so cannot have user-defined constructors that could take an allocator parameter. `std::tuple` takes an allocator only to pass it to constructors of its components, if they would take one; the allocator is not used for anything else. `std::tuple` won't use its allocator for anything. – Igor Tandetnik Aug 14 '19 at 03:07
  • 1
    @IgorTandetnik ...that makes sense. I guess I'd need to promote it to a heap-based object, meaning something like `std::shared_ptr>`, at which point I could use a custom deleter. I've always returned `std::unique_ptr` instead so the caller could choose their own pointer flavor, and I was worried because that type requires the (internal) deleter as a parameter, but it looks like `std::shared_ptr` doesn't have that problem. – Reid Rankin Aug 14 '19 at 03:16
  • Well, if you are concerned with misuse by clients, `shared_ptr` can be `release`d. – Igor Tandetnik Aug 14 '19 at 03:26
  • @IgorTandetnik Footguns are OK, as long as they have a decent safety. It's not my job to prevent a client from unsafely deallocating memory any more than it's my job to prevent them from, say, emailing the contents of the buffer to the Chinese; it's just my job to make sure they can't do it by accident. – Reid Rankin Aug 14 '19 at 03:32
  • 1
    @IgorTandetnik Also, `std::shared_ptr` doesn't have a `release()`; it's got a `reset()` but that still calls the deleter. – Reid Rankin Aug 14 '19 at 03:33
  • From my understanding of your question, the best thing you can do seems to be to write a `Key` class which holds an `std::array` and zeros the contents in the destructor. – L. F. Aug 14 '19 at 06:44

2 Answers2

3

This sounds like a XY problem. You seem to be misusing allocators. Allocators are used to handle runtime memory allocation and deallocation, not to hook stack memory. What you are trying to do — zeroing the memory after using — should really be done with a destructor. You may want to write a class Key for this:

class Key {
public:
    // ...
    ~Key()
    {
        secure_clear(*this); // for illustration
    }
    // ...
private:
    std::array<std::uint8_t, 32> key;
};

You can easily implement iterator and span support. And you don't need to play with allocators.

If you want to reduce boilerplate code and make the new class automatically iterator / span friendly, use inheritance:

class Key :public std::array<std::uint8_t, 32> {
public:
    // ...
    ~Key()
    {
        secure_clear(*this); // for illustration
    }
    // ...
};
L. F.
  • 19,445
  • 8
  • 48
  • 82
  • I'm not a fan of this approach, for a number of reasons. First, it requires a whole ton of boilerplate just to get a behavior change consisting of a single line of code, which seems wrong. Second, it requires clients to either be aware of and support my custom container (reducing portability) or be templated themselves in a manner that accepts any stl-style container (requiring significant extra complexity). Third, and most importantly, any time you return an instance of this class the compiler is allowed to move-optimize the process and *not run the destructor on the old copy*. – Reid Rankin Aug 14 '19 at 22:17
  • 1
    The way I see it, you **have** to use a heap allocation to store the data if you want to be sure you control its deallocation. Furthermore, because you are explicitly concerned with the fate of memory that will never be visible to the program again, you have to hook in at the runtime memory management level--which is appropriate, given that you are, in fact, concerned with the management of memory rather than of higher-level constructs like objects or values. – Reid Rankin Aug 14 '19 at 22:23
  • @ReidRankin The first point makes sense, but other approaches would require even more boilerplate code, I guess. If you goal is less LOC, you can derive publicly from array and inherit iterator / span support. The second point is a more serious problem if you use an allocator, right? And the third point doesn’t make much sense to me. Can you provide an example? – L. F. Aug 15 '19 at 07:00
  • As to the second point, while some STL containers use the allocator type as a template parameter (thereby complicating client code), not all do. For example, you can pass an allocator to the vector constructor when you're making a `vector`, and when you return it the client will be none the wiser. – Reid Rankin Aug 15 '19 at 07:44
  • To understand the third point, look at what happens when you call `Foo bar() { Foo retval; foo.doThings(); return retval; }` by doing `Foo x = bar();`. First, the function `bar()` creates a `Foo` in its stack frame; second, it returns the `Foo`, which needs to end up stored as `x` in the caller's stack frame. The compiler is allowed to move the return value instead of, say, calling a copy constructor to do this. Normally moves result in the moved-from object being blanked out, but since the life of `retval` is over the compiler doesn't have to due to the as-if rule. – Reid Rankin Aug 15 '19 at 08:01
  • Essentially, the compiler pretends that nothing happened: no new objects got created and nothing got destroyed, the returned object's address just magically changed. Because any pointers to objects on the returned-from function's stack frame are obviously invalid, there's no references hanging around to the old copy, so this is perfectly acceptable from a standards point of view. Still, in reality it leaves a copy of the old data in the just-abandoned stack frame. This is why an allocator is required: because we're actually interested in the disposition of the memory, not the container. – Reid Rankin Aug 15 '19 at 08:07
  • 1
    @ReidRankin No, the destructor is always called. Moving / copy elision doesn’t change this. There is no such thing as “destruction elision.” If it were to exist, then any class with a nontrivial destructor will get screwed up. – L. F. Aug 15 '19 at 09:05
  • 1
    @ReidRankin Incidentally, your explanation about the second point seems to suggest that the no-allocator approach is actually superior. – L. F. Aug 15 '19 at 09:07
  • Quoting from C++11: "When copy elision occurs, the implementation treats the source and target of the omitted copy/move operation as simply two different ways of referring to the same object, and destruction of that object occurs at the later of the times when the two objects would have been destroyed without the optimization." – Reid Rankin Aug 15 '19 at 18:27
  • @ReidRankin Yeah, in this case there is *exactly one* object, and the destructor is called exactly once on this object. There is no “copy of the old data in the just-abandoned stack frame.” That copy doesn’t exist at all. That’s how copy elision works. – L. F. Aug 15 '19 at 20:00
  • Well, I'm not sure you're right, but I'm not sure I'm right anymore either! I've opened a [separate question](https://stackoverflow.com/questions/57515813/is-there-any-situation-in-which-an-objects-storage-might-change-during-its-life) for this specific issue since we've strayed somewhat from this particular question's topic but this is important in its own right (and should be googlable for posterity). – Reid Rankin Aug 15 '19 at 20:29
3

You need cooperation from both the compiler and the OS in order for such a scheme to work. P1315 is a proposal to address the compiler/language side of things. As for the OS, you have to make sure that the memory was never paged out to disk, etc. in order for this to truly zero memory.

Nevin
  • 4,595
  • 18
  • 24
  • I didn't know about that proposal; thanks for bringing it to my attention! Still, even though it's harder to read and potentially less efficient, `std::fill((volatile uint8_t*)data, (volatile uint8_t*)data + length, 0)` will do the job on the compiler side. I'm tempted to declare the swapping problem out-of-scope, though I could probably make my allocator only use memory for pages that are pinned to RAM without too much work... but it doesn't matter in my application anyway because the target device doesn't have any swap space. – Reid Rankin Aug 14 '19 at 22:42
  • That won't work either. Roughly speaking, if it is stack memory, the compiler can still optimize it out. Look at https://wg21.link/p1152R0 for a more detailed description on what the woefully underspecified volatile keyword means. (Note: the later versions of the paper leave out some of these details.) – Nevin Aug 15 '19 at 18:27
  • Very interesting! I think I like that proposal. However, I don't see how it breaks the use of std::fill? The parameters aren't volatile themselves but pointers-to-volatile. – Reid Rankin Aug 15 '19 at 18:42
  • 1
    If the compiler determines that you are pointing to stack memory (and they are pretty good at this), the compiler can assume it isn't memory-mapped IO and can optimize it away. – Nevin Aug 15 '19 at 20:32