Fastest way to convert 3-Channel Image to single vector of type T

I want to flatten 3-channel RGB image into a single vector of type T. In my case I am focused on float datatype. I wrote a function which is working fine. But I want to do this operation in less than 1MilliSecond. Here time cost is really a issue for me. Here is my code.

template<typename T>
inline std::vector<T> flatten_temp(cv::Mat frame_image)
{
    assert(!frame_image.empty());
    const int image_depth = frame_image.channels();
    frame_image.convertTo(frame_image, CV_32FC3);
    cv::Mat *planes =  new cv::Mat[image_depth];        
    
    // /** Split into BGR */
    cv::split(frame_image, planes);
    std::vector<T> flattened_image;
    
    flattened_image.assign(planes[0].begin<T>(), planes[0].end<T>());

    for (int i = 1; i < image_depth; ++i) {
        flattened_image.insert(flattened_image.end(), planes[i].begin<T>(), planes[i].end<T>());
    }
    frame_image.release();
    delete [] planes;
    return flattened_image;
}

Any help would be appreciated.