Load image array from GPU memory into keras/tensorflow without numpy

I have trained model in keras for image classification. Now, I am using my own encoder-decoder (not using OpenCV) for image processing and streaming using GPU. This encoder-decoder code is written in c++. I am able to transfer the image array generated using encoder-decoder to python using numpy and ctypes, and using that numpy array for image classification. This process induces huge overhead as it involves copying data from GPU to numpy array which uses CPU and again copying numpy array to GPU for image classification.

How can I directly use the GPU image array into the keras for inference? I have the address of the GPU image array through c++ code generated by encoder-decoder.