Inserting a Smart Pointer in a Unordered Map calls destructor

I'm making an engine and to handle materials stuff I have a static renderer class storing a static std::unordered_map<uint, Ref<Material>> m_Materials; (a material and its ID), being Ref the a shared pointer using the next methods:

template<typename T>
using Ref = std::shared_ptr<T>;

template<typename T, typename ... Args>
constexpr Ref<T> CreateRef(Args&& ... args) { return std::make_shared<T>(std::forward<Args>(args)...); }

template<typename T>
constexpr Ref<T> CreateRef(T* t) { return std::shared_ptr<T>(t); }

So a Material is a simple class constructed with an unsigned int (uint) to store its ID and a string to store its name, and to create them and store them, I have the next static function in the Renderer:

Ref<Material> Renderer::CreateMaterialWithID(uint material_id, const std::string& name)
{
    Ref<Material> mat = CreateRef<Material>(new Material(material_id, name));
    std::pair<uint, Ref<Material>> mat_pair(material_id, mat);

    m_Materials.insert(mat_pair);
    return mat;
}

The problem is that the insertion into the map, for some reason, calls the material destructor, resulting in the insertion of an empty smart pointer (though the ID is correctly inserted) [material_id, empty].

Now, the material and the pair created seems to properly store the data of the material, so I don't know why it is the map failing. I also tried to insert as initializer list ({material_id, mat}), to create the material as CreateRef<Material>(material_id, name) (without the new), and also to use emplace instead of insert but I have always the same result, and I don't understand why the map calls the destructor.

Anyone sees something wrong?