How to modify values of reference std::pair?

So my question is pretty simple, tho I haven't been able to find an answer already so I'm asking here.

I curious to know whether I can return an std:: pair reference from a function, and have the calling function modify its values. Here's an example of what I mean:

struct PairStruct {
    using PairType = std::pair<size_t, size_t>;

    PairStruct() : m_pair(std::make_pair(0, 0)) {}

    void modifyInternal() {
        auto pair = getPair();

        std::cout << "start - first: " << pair.first << ", second: " << pair.second << "\n";
        pair.first++;
        pair.second++;
        std::cout << "end - first: " << pair.first << ", second: " << pair.second << "\n";
    }

    void modifyPtrInternal() {
        auto pair = getPairPtr();

        std::cout << "start - first: " << pair->first << ", second: " << pair->second << "\n";
        pair->first++;
        pair->second++;
        std::cout << "end - first: " << pair->first << ", second: " << pair->second << "\n";
    }

    PairType &getPair() {
        return m_pair;
    }

    PairType *getPairPtr() {
        return &m_pair;
    }

    PairType m_pair;
};

int main(int argc, char ** args)
{
    PairStruct *pairInst = new PairStruct;

    // Test with reference
    std::cout << "Reference test.\n";
    pairInst->modifyInternal();
    std::cout << "\n";
    pairInst->modifyInternal();

    std::cout << "\n";

    // Test with ptr
    std::cout << "Ptr test.\n";
    pairInst->modifyPtrInternal();
    std::cout << "\n";
    pairInst->modifyPtrInternal();

    delete pairInst;
    return 0;
}

As expected when I use a pointer it correctly modyfies the values, this is not the case when returning a reference. Here's the output of this program:

Reference test.
start - first: 0, second: 0
end - first: 1, second: 1

start - first: 0, second: 0
end - first: 1, second: 1

Ptr test.
start - first: 0, second: 0
end - first: 1, second: 1

start - first: 1, second: 1
end - first: 2, second: 2

This is going to seem very trivial, however, I'd like to know why I can't use the referenced pair in this case. Thanks!