Is interpreting a pointer to first member as the class itself well defined?

I have some code that look like this:

template<typename T>
struct memory_block {
    memory_block(memory_block const&) = delete;
    memory_block(memory_block const&&) = delete;
    memory_block(memory_block&) = delete;
    memory_block(memory_block&&) = delete;
    memory_block& operator=(memory_block const&) = delete;
    memory_block& operator=(memory_block&&) = delete;

    template<typename... Args>
    explicit memory_block(Args&&... args) noexcept :
        data{std::forward<Args>(args)...} {}

    T data;
};

template<typename T>
struct special_block : memory_block<T> {
    using memory_block<T>::memory_block;
    std::vector<double> special_data;
};

// There is no other inheritance. The hierarchy ends here.

Now I have to store these types into type erased storage. I chose a vector of void* as my container. Before inserting into the container, I cast all pointers to memory_block:

struct NonTrivial { virtual ~NonTrivial() {} };

// exposed to other code
std::vector<void*> vec;

// My code use dynamic memory instead of static
// Data, but it's simpler to show it that way.
static memory_block<int> data0;
static special_block<NonTrivial> data1;

void add_stuff_into_vec() {
    // Add pointer to `data` member to the vector.
    vec.emplace_back(&(data0->data));
    vec.emplace_back(&(data1->data));
}

Then later in the code, I access the data:

// Yay everything is fine, I cast the void* to it original type
int* data1 = static_cast<int*>(vec[0]);
NonTrivial* data1 = static_cast<NonTrivial*>(vec[1]);

The problem is that I want to access special_data in the non-trivial case:

// Pretty sure this cast is valid! (famous last words)
std::vector<double>* special = static_cast<special_block<NonTrivial>*>(
    static_cast<memory_block<NonTrivial>*>(vec[1]) // (1)
);

---

So now, the question

The problem arise at line (1): I have a pointer to data (of type NonTrivial), which is a member of memory_block<NonTrivial>. I know that the void* will always point to the first data member of a memory_block<T>.

So is casting a void* to the first member of a class into the class safe? If not, is there another way to do it? If it can make things simpler, I can get rid of the inheritance.

I hoped standard layout would help me in this case, but my static assert seem to fail.