jeudi 25 août 2016

What's the proper way of calling a Win32/64 function from LLVM?

I'm attempting to call a method from LLVM IR back to C++ code. I'm working in 64-bit Visual C++, or as LLVM describes it:

Machine CPU:      skylake
Machine info:     x86_64-pc-windows-msvc

For integer types and pointer types my code works fine as-is. However, floating point numbers seem to be handled a bit strange.

Basically the call looks like this:

struct SomeStruct 
{
    static double Callback(uint8_t* ptr, double foo) { return foo * 2; }
};

and LLVM IR looks like this:

define i32 @main(i32, i8**) {
varinit:
  // omitted here: initialize %ptr from i8**. This IR code works. 

  %1 = call double @"SomeStruct::Callback"(i8* %ptr, double 2.700000e+01)
  ret i32 0
}

declare double @"SomeStruct::Callback"(i8*, double)

I figured that the problem is probably in the way the calling conventions work. So I've attempted to make some adjustments to correct for that:

// during initialization of the function
auto function = llvm::Function::Create(functionType, llvm::Function::ExternalLinkage, name, module);
function->setCallingConv(llvm::CallingConv::X86_64_Win64);
...

// during calling of the function
call->setCallingConv(llvm::CallingConv::X86_64_Win64);

Unfortunately no matter what I try, I end up with 'invalid instruction' errors, which this user reports to be an issue with calling conventions: Clang producing executable with illegal instruction .

I've read up on http://ift.tt/1S47Frk in an attempt to figure out what's going on. Then I looked at the assembly output and found:

    movq    (%rdx), %rcx
    vmovss  __real@41d80000(%rip), %xmm1
    callq   "SomeStruct::Callback"

According to MSDN, argument 2 should be in %xmm1 so that also seems correct. However, when checking if everything works in the debugger, Visual Studio reports a lot of question marks (e.g. 'illegal instruction').

Any feedback is appreciated.

Aucun commentaire:

Enregistrer un commentaire