How to set library flags after source file in cmake?

I'm trying to compile a project using hwloc with CMake. However, I get a ton of undefined reference errors when linking:

undefined reference to `hwloc_get_type_depth'
undefined reference to `hwloc_bitmap_zero'
[...]

According to this answer to a similar question the order of flags is important.

So, how can I generate a command like this in CMake? :

g++ -Wall -std=c++11 source.cpp-lhwloc

Excerpt from my CMakeLists.txt:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -std=c++11 -lhwloc")

set(SOURCE_FILES source.cpp)
add_executable(source ${SOURCE_FILES})

Any help is greatly appreciated!