Should I declare a method noexcept if it never throws when used correctly?

I'm implementing a queue, and I was wondering, what should I do when a user misuses the container?

For example I have two methods, Front and Pop, which never throw (I static_assert that the destructor for the contained element is noexcept) so long as they're not called on an empty queue. I could add a check to these that throws if they're called on empty queues, but then I couldn't define them noexcept.

I was thinking that it makes sense to declare these noexcept, and then say that the behavior is undefined when called on an empty queue (I provide Size and Empty methods for users to check). I could then add the check only on debug builds, so it calls terminate in debug when misused and attempts to destruct or dereference missing elements on release. I was wondering what the better approach would be.