Why does the C++ standard specifically grant leeway regarding memory layout of class data members with different access specifiers?

The C++11 standard mandates an ordering in memory for the non-static data members of a class but then specifically carves out an exemption for members with different access specifiers.

Why though?

§ 9.2.13

Nonstatic data members of a (non-union) class with the same access control (Clause 11) are allocated so that later members have higher addresses within a class object. The order of allocation of non-static data members with different access control is unspecified (Clause 11). Implementation alignment requirements might cause two adjacent members not to be allocated immediately after each other; so might requirements for space for managing virtual functions (10.3) and virtual base classes (10.1).

This part of the standard has come up on stackoverflow before but I don't think it has ever been explained.