let compiler optimize generalized class for spezific sizes (like a template)

I have programmed a class revolving around arbitrary sized Matrices. My question now is, though I could hardly articulate it in a title, whether there is some way to let the compiler create an extra version of lets say 3X3 or 4X4 matrices where for example all the loops would be optimized away. My problem is very time critical so it would be amazing if there was some way to let the compiler degeneralize my matrix functions for those popular cases without having to maintain 3 Versions of the same algorithms in a fashion similiar to a template approach.

I would be really glad if someone could give me recommendations regarding this problem though I dont think it can be solved quite as a template.