How to safely cast array of char to type

I got some data coming in over an RS485 line. I store that data in a vector<char> and now I need to cast that to several types. Currently that is done by the following function:

template<typename To>
static std::size_t parseBinaryToType(const char* input, To& destination) {
    destination = *(reinterpret_cast<const To*>(input)); //FIXME 
    return sizeof(destination);
}
//EXAMPLE
enum class SomeEnum: uint16_t {
  BAR,
  BAZ
}

class Foo {
  SomeEnum m_Enum;
  uint8_t m_Int;
  bool m_Bool;
public: 
  static Foo deserializeFrom(const char* const buffer) {
    Foo result;
    offset = 0;
    offset+= parseBinaryToType(buffer + offset, result.m_Enum);
    offset+= parseBinaryToType(buffer + offset, result.m_Int);
    offset+= parseBinaryToType(buffer + offset, result.m_Bool);
    return result;
  }
}

But I find this very unsafe. Especially due to the reinterpret_cast and the fact that it doesn't check for endianness.. Is there a safe way to do this in cpp?