To properly call atan2 push two double precision floating points values onto the regular stack and then call atan2. The answer in double precision is returned to the top of the floating point stack (st0). Below is an example code that takes two single precision floating point values (imaginary and magnitude) and calls atan2 and prints out the result. Use the -lm option with gcc to link your .o file. fld dword[imaginary] sub esp ,16 fstp qword[esp] fld dword[magnitude] fstp qword[esp+8] call atan2 Make_Local_Str flst_len1, flst_str1, 'atan2 is -> %.6e', 10, 0 sub esp, 8 fstp qword [esp] push dword flst_str call printf add esp, 12