Dynamic attributes type

I'm writing a program that reads a text file and then instantiate a few objects. The problem is, it needs to decide what kind of attributes those objects will get based upon what has been parsed from the text file.

To simplify the problem, let's suppose in the text file you should write only "int" or "double" and the object needs to be instantiated with a member variable of type respectively int or double.

The solution I've come up with is, at its core, the following:

1. Implement an abstract class, NumericalType, with no attribute.
2. Write one child class for every admitted type. In this case, int and double, so intType and doubleType, with a member variable of the proper type and proper implementations to deal with it.
3. Give every instatiated object a NumericalType * const (const because once read from the text file, it stays the same) and assign to it the proper type.

However I don't like it that much. There's a lot indirection; for example, variables can be accessed only through getters or a dynamic_cast (which seems a bit too much for what should be a simple architecture).

Can this be done in a more simple and elegant way?