Matrix determinant implementation in template matrix class

I'm working on a project which requires me to write my own matrix class implementation. I decided to implement the matrix class as a template to provide compile-time error-checking in the following way:

template<size_t N, size_t M> // N × M matrix
class Matrix
{
    // implementation...
};

I managed to implement basic operations such as addition/subtraction, transpose and multiplication. However, I'm having trouble implementing the determinant. I was thinking of implementing it recursively using the Laplace expansion, so I must first implement a way to calculate the i,j minor of a matrix. The problem is, the minor of an N × N matrix is an (N-1) × (N-1) matrix. The following does not compile:

template<size_t N>
Matrix<N-1, N-1> Minor(const Matrix<N, N>& mat, size_t i, size_t j)
{
    Matrix<N-1, N-1> minor;
    // calculate i,j minor
    return minor
}

How could I go around this and calculate the minor, while keeping the templated form of the class?