How to convert a cv::Mat to protobuf and back in C++?

I am trying to get the basic functionality of conversion from cv::Mat into a protobuf message and then back, between the gRPC Server and Client. I tried to do it on my own, but the char array always seems to be empty when I try to convert it from cv::Mat to std::unique_ptr char_img.

The following is my protobuf for image message :

message PbfImage {
    bool color = 1;
    bytes data = 2;
    int32 width = 3;
    int32 height = 4;
}

The following is the failing cv::Mat to unsigned char [] conversion code :

#include <iostream>
#include <memory>
#include <opencv2/core.hpp>
#include <image.grpc.pb.h>

// Reverse conversion from protobuf to cv::Mat
cv::Mat get_cvImage(const PbfImage &img_msg) {
    std::unique_ptr<unsigned char> img (new unsigned char [img_msg.data().size()]);
    std::memcpy(&*img, img_msg.data().c_str(), img_msg.data().size());
    return cv::Mat(img_msg.height(), img_msg.width(), img_msg.color(), &img);
}

// Forward conversion from cv::Mat to protobuf
PbfImage get_PbfImage(const cv::Mat &cv_img) {
    std::unique_ptr<unsigned char> img_msg (new unsigned char [cv_img.total()]);
    std::memcpy(&*img_msg, cv_img.data, sizeof(cv_img.at<uchar>(0, 0)) * cv_img.total());
    return PbfImage();
}

int main(int argc, char *argv[]) {
    std::string file_name = "../images/costa_rica.jpg";
    cv::Mat image = cv::imread(file_name);
    if (image.empty()) {
        std::cout << "Couldn't load the image \n";
    }
    get_PbfImage(image); // The char array is completely empty in this function
    return 0;
}

After this copy operation above, the uchar/ byte array is completely empty. After this we have to convert the byte array to std::string and then pass it as a message to the gRPC server ?