How to avoid const cast for map access?

I have the following issue:

std::map<A*,double> map;

void getColor( A const * obj){
    doubel d = map[obj]; // does not compile wihtout const_cast<A*>(obj)
    // do something
}

I have a map std::map (somewhere) which stores pointers to objects A. I have a function getColor which does not manipulate objects A and so takes a pointer to a const A as input.

The function getColor will not compile without using a const_cast.

Const cast is a design issue, but I dont know how to circumenvent it, if I dont want to make the keys in map const.

Any help appreciated.