flexlink produces an invalid dll when building lablgtk-2.18.3 on mingw64

Hi all,

I'm recently building lablgtk (a GTK2 wrapper for OCaml) using mingw64 toolchains provided by msys2. The package uses ocamlmklib (and thus flexlink) to create a dll library called dlllablgtk2.dll. Here are the version of the tools in my environment:

flexdll    0.34 (from http://alain.frisch.fr/flexdll.html; built from source)
ocaml    4.02.1 (built from source)

Flexlink generates the library without error, but the library is considered invalid by LoadLibraryEx:

Error: Error on dynamically loaded library: .\dlllablgtk2.dll: %1 is not a valid win32 application

The following toy program gives the same result.

$ cat testdll.c
#include <flexdll.h>
#include <stdio.h>
#include <windows.h>

int main(int argc, char *argv[]) {
    void *handle;
    printf("Try open: %s\n", argv[1]);
    handle = flexdll_dlopen(argv[1], FLEXDLL_RTLD_GLOBAL);
    printf("Handle: %p\n", handle);
    if (handle == NULL) {
            printf("Error code: %d\n", GetLastError());
            printf("Error message: %s\n", flexdll_dlerror());
    }
    return 0;
}

$ flexlink -chain mingw64 -exe -o testdll testdll.c
$ testdll.exe dlllablgtk2.dll
Try open: dlllablgtk2.dll
Handle: 0000000000000000
Error code: 193
Error message: %1 is not a valid win32 application

The library is created using 24 object files in addition to some system libraries. The command is:

flexlink -v -v -chain mingw64 -LD:/msys64/mingw64/x86_64-w64-mingw32/lib \
-o dlllablgtk2.dll -lpthread -LD:/msys64/mingw64/lib -lgtk-win32-2.0 \
-limm32 -lshell32 -lole32 -lpangocairo-1.0 -lpangoft2-1.0 -lpangowin32-1.0 -lgdi32 \
-lpango-1.0 -lm -latk-1.0 -lcairo -lpixman-1 -lfontconfig -lexpat -lfreetype -lexpat -lfreetype \
-lbz2 -lharfbuzz -lgdk_pixbuf-2.0 -lpng16 -lgio-2.0 -lz -lgmodule-2.0 -lgobject-2.0 -lffi \
-lglib-2.0 -lws2_32 -lole32 -lwinmm -lshlwapi -lintl \
ml_gobject.o ml_gpointer.o ml_gtk.o ml_gtkaction.o ml_gtkbin.o ml_gtkbroken.o ml_gtkbutton.o \
ml_gtkassistant.o ml_gtkedit.o ml_gtkfile.o ml_gtklist.o ml_gtkmenu.o ml_gtkmisc.o ml_gtkpack.o \
ml_gtkrange.o ml_gtkstock.o ml_gtktext.o ml_gtktree.o ml_gdkpixbuf.o ml_gdk.o ml_glib.o \
ml_pango.o ml_gvaluecaml.o wrappers.o

When I remove some of the objects (e.g. ml_gtktree.o), the generated library becomes valid.

$ testdll.exe dlllablgtk2.dll        # ml_gtktree.o removed from the command
Try open: dlllablgtk2.dll
Handle: 0000000000000000
Error code: 1114
Error message: Cannot resolve caml_failwith

It seems the issue is not raised by a single object. The library built without ml_gtktext.o (but with ml_gtktree.o) is also valid.

The binaries from https://github.com/shadinger/flexdll-win64 (version 0.26) does not suffer from this issue.

Here is the verbose log during linking.

** Use cygpath: true
** Search path:
D:/msys64/mingw64/lib
D:/msys64/mingw64/x86_64-w64-mingw32/lib
D:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2
/mingw/lib
/mingw64/x86_64-w64-mingw32/lib
** Default libraries:
dllcrt2.o
-lmingw32
-lgcc
-lmoldname
-lmingwex
-lmsvcrt
-luser32
-lkernel32
-ladvapi32
-lshell32
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\dllcrt2.o
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libmingw32.a
** open: D:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2\libgcc.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libmoldname.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libmingwex.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libmsvcrt.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libuser32.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libkernel32.a
** open: D:/msys64/mingw64/x86_64-w64-mingw32/lib\libadvapi32.a
+ x86_64-w64-mingw32-gcc -mconsole -shared -Wl,-eFlexDLLiniter  -L. -I"D:/msys64/mingw64/lib" -I"D:/msys64/mingw64/x86_64-w64-mingw32/lib" -L"D:/msys64/mingw64/lib" -L"D:/msys64/mingw64/x86_64-w64-mingw32/lib" -o "test.dll" "D:\msys64\tmp\dyndll3ef3ef.o" "D:\msys64\mingw64\bin\flexdll_initer_mingw64.o" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libpthread.dll.a" "D:/msys64/mingw64/lib\libgtk-win32-2.0.dll.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libimm32.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libshell32.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libole32.a" "D:/msys64/mingw64/lib\libpangocairo-1.0.dll.a" "D:/msys64/mingw64/lib\libpangoft2-1.0.dll.a" "D:/msys64/mingw64/lib\libpangowin32-1.0.dll.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libgdi32.a" "D:/msys64/mingw64/lib\libpango-1.0.dll.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libm.a" "D:/msys64/mingw64/lib\libatk-1.0.dll.a" "D:/msys64/mingw64/lib\libcairo.dll.a" "D:/msys64/mingw64/lib\libpixman-1.dll.a" "D:/msys64/mingw64/lib\libfontconfig.dll.a" "D:/msys64/mingw64/lib\libexpat.dll.a" "D:/msys64/mingw64/lib\libfreetype.dll.a" "D:/msys64/mingw64/lib\libbz2.dll.a" "D:/msys64/mingw64/lib\libharfbuzz.dll.a" "D:/msys64/mingw64/lib\libgdk_pixbuf-2.0.dll.a" "D:/msys64/mingw64/lib\libpng16.dll.a" "D:/msys64/mingw64/lib\libgio-2.0.dll.a" "D:/msys64/mingw64/lib\libz.dll.a" "D:/msys64/mingw64/lib\libgmodule-2.0.dll.a" "D:/msys64/mingw64/lib\libgobject-2.0.dll.a" "D:/msys64/mingw64/lib\libffi.dll.a" "D:/msys64/mingw64/lib\libglib-2.0.dll.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libws2_32.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libwinmm.a" "D:/msys64/mingw64/x86_64-w64-mingw32/lib\libshlwapi.a" "D:/msys64/mingw64/lib\libintl.dll.a" "D:\msys64\tmp\dyndll00be4c.o" "D:\msys64\tmp\dyndlle902c0.o" "D:\msys64\tmp\dyndll54d32d.o" "D:\msys64\tmp\dyndll2e0163.o" "ml_gtkbin.o" "D:\msys64\tmp\dyndll7ac0f6.o" "D:\msys64\tmp\dyndll3f46a1.o" "D:\msys64\tmp\dyndll6e7d00.o" "D:\msys64\tmp\dyndll709dae.o" "D:\msys64\tmp\dyndll4b5dee.o" "D:\msys64\tmp\dyndll027612.o" "D:\msys64\tmp\dyndll478b19.o" "D:\msys64\tmp\dyndll0fdffc.o" "D:\msys64\tmp\dyndll533488.o" "D:\msys64\tmp\dyndllc5412c.o" "D:\msys64\tmp\dyndllb81a8b.o" "D:\msys64\tmp\dyndll5f1731.o" "D:\msys64\tmp\dyndll4bc469.o" "D:\msys64\tmp\dyndlleed2db.o" "D:\msys64\tmp\dyndlla2929b.o" "D:\msys64\tmp\dyndll56c73c.o" "D:\msys64\tmp\dyndllde988f.o" "ml_gvaluecaml.o" "D:\msys64\tmp\dyndll049034.o" "D:\msys64\tmp\flexlink250fe6.def"
(call with bash: D:\msys64\tmp\longcmd233aa5)

Start with the flexlink -chain mingw64 entry point and reproduce the issue using the shown testdll.c command and the full object list. Compare the generated DLL with the valid reduced-object builds and the flexdll-win64 0.26 behavior; done means the full dlllablgtk2.dll loads through LoadLibraryEx without the invalid Win32 application error.