Concurrent reads on non-atomic variable

I encounter this question while trying to implement a shared pointer. Let's focus on the managed data pointer. Its lifetime can be divided into three stages:

1. Construction where there is no concurrent access.
2. Concurrent reads on it.
3. Destruction where there is no concurrent access. This is guaranteed by reference counting.

My question is, given this situation, is it necessary for the pointer to be atomic? I think it's equivalent to: will stage 2 lead to undefined behavior if the pointer is not atomic? Ideally, I want to hear an answer discussing from both a theoretical (language-lawyer) point of view and a practical point of view. For example, if not atomic, stage 2 may be undefined behavior theoretically, but is practically OK on actual platforms. For implementing shared pointer, if non-atomic is OK, the managed pointer can be unique_ptr<T>, otherwise it has to be atomic<T*>.