Verify Msys2 build on Windows

Start with the reported winsup test suite summary and the individual failures, including mingw/cygload, and review how MSYS2 validates a source build on Windows x86. Done means identifying the recommended validation procedure and documenting whether the 130 failures are expected or indicate a build problem.

Hi Developers

Environment

• Platform: Windows x86
• Build: MSYS compiled from source
• Test suite: winsup test suite

Background

I built MSYS from source on a Windows x86 machine and want to validate my build.

Questions

1. How can I validate my build? Is there a recommended way to verify that a source build is correct and complete on Windows x86?
2. Is there a built-in test suite I can run to validate the build? I found the winsup test suite and ran it — is this the right one to use, or are there others I should be running?
3. Are the observed failures expected? I'm seeing 130 failures out of 282 tests. Are these known/expected on Windows x86, or do they indicate a problem with my build?

Full Test Log

PASS: winsup.api/checksignal.exe
PASS: winsup.api/crlf.exe
FAIL: winsup.api/devdsp.exe
PASS: winsup.api/devzero.exe
PASS: winsup.api/iospeed.exe
PASS: winsup.api/mmaptest01.exe
PASS: winsup.api/mmaptest02.exe
PASS: winsup.api/mmaptest03.exe
PASS: winsup.api/mmaptest04.exe
FAIL: winsup.api/msgtest.exe
PASS: winsup.api/nullgetcwd.exe
PASS: winsup.api/resethand.exe
FAIL: winsup.api/semtest.exe
FAIL: winsup.api/shmtest.exe
PASS: winsup.api/sigchld.exe
PASS: winsup.api/signal-into-win32-api.exe
FAIL: winsup.api/systemcall.exe
PASS: winsup.api/user_malloc.exe
PASS: winsup.api/waitpid.exe
FAIL: winsup.api/ltp/access01.exe
FAIL: winsup.api/ltp/access03.exe
FAIL: winsup.api/ltp/access04.exe
FAIL: winsup.api/ltp/access05.exe
PASS: winsup.api/ltp/alarm01.exe
PASS: winsup.api/ltp/alarm02.exe
PASS: winsup.api/ltp/alarm03.exe
PASS: winsup.api/ltp/alarm07.exe
FAIL: winsup.api/ltp/asyncio02.exe
PASS: winsup.api/ltp/chdir02.exe
FAIL: winsup.api/ltp/chdir04.exe
FAIL: winsup.api/ltp/chmod01.exe
FAIL: winsup.api/ltp/chmod02.exe
FAIL: winsup.api/ltp/chown01.exe
FAIL: winsup.api/ltp/close01.exe
PASS: winsup.api/ltp/close02.exe
FAIL: winsup.api/ltp/close08.exe
FAIL: winsup.api/ltp/creat01.exe
FAIL: winsup.api/ltp/creat03.exe
FAIL: winsup.api/ltp/creat09.exe
FAIL: winsup.api/ltp/dup01.exe
FAIL: winsup.api/ltp/dup02.exe
FAIL: winsup.api/ltp/dup03.exe
FAIL: winsup.api/ltp/dup04.exe
FAIL: winsup.api/ltp/dup05.exe
FAIL: winsup.api/ltp/execl01.exe
FAIL: winsup.api/ltp/execle01.exe
FAIL: winsup.api/ltp/execlp01.exe
FAIL: winsup.api/ltp/execv01.exe
FAIL: winsup.api/ltp/execve01.exe
FAIL: winsup.api/ltp/execvp01.exe
PASS: winsup.api/ltp/exit01.exe
FAIL: winsup.api/ltp/exit02.exe
FAIL: winsup.api/ltp/fchdir01.exe
FAIL: winsup.api/ltp/fchdir02.exe
FAIL: winsup.api/ltp/fchmod01.exe
FAIL: winsup.api/ltp/fchown01.exe
FAIL: winsup.api/ltp/fcntl02.exe
FAIL: winsup.api/ltp/fcntl03.exe
FAIL: winsup.api/ltp/fcntl04.exe
FAIL: winsup.api/ltp/fcntl05.exe
FAIL: winsup.api/ltp/fcntl07.exe
FAIL: winsup.api/ltp/fcntl07B.exe
FAIL: winsup.api/ltp/fcntl08.exe
PASS: winsup.api/ltp/fcntl09.exe
FAIL: winsup.api/ltp/fcntl10.exe
FAIL: winsup.api/ltp/fork01.exe
PASS: winsup.api/ltp/fork02.exe
PASS: winsup.api/ltp/fork03.exe
FAIL: winsup.api/ltp/fork04.exe
PASS: winsup.api/ltp/fork06.exe
FAIL: winsup.api/ltp/fork07.exe
FAIL: winsup.api/ltp/fork09.exe
FAIL: winsup.api/ltp/fork10.exe
PASS: winsup.api/ltp/fork11.exe
FAIL: winsup.api/ltp/fpathconf01.exe
FAIL: winsup.api/ltp/fstat01.exe
FAIL: winsup.api/ltp/fstat02.exe
FAIL: winsup.api/ltp/fstat03.exe
FAIL: winsup.api/ltp/fstat04.exe
FAIL: winsup.api/ltp/fsync01.exe
FAIL: winsup.api/ltp/ftruncate01.exe
FAIL: winsup.api/ltp/ftruncate02.exe
FAIL: winsup.api/ltp/ftruncate03.exe
PASS: winsup.api/ltp/getegid01.exe
PASS: winsup.api/ltp/geteuid01.exe
PASS: winsup.api/ltp/getgid01.exe
PASS: winsup.api/ltp/getgid02.exe
PASS: winsup.api/ltp/getgid03.exe
PASS: winsup.api/ltp/getgroups01.exe
PASS: winsup.api/ltp/getgroups02.exe
PASS: winsup.api/ltp/gethostid01.exe
PASS: winsup.api/ltp/gethostname01.exe
PASS: winsup.api/ltp/getpgid01.exe
PASS: winsup.api/ltp/getpgid02.exe
PASS: winsup.api/ltp/getpgrp01.exe
PASS: winsup.api/ltp/getpid01.exe
PASS: winsup.api/ltp/getpid02.exe
PASS: winsup.api/ltp/getppid01.exe
PASS: winsup.api/ltp/getppid02.exe
PASS: winsup.api/ltp/getuid01.exe
PASS: winsup.api/ltp/getuid02.exe
PASS: winsup.api/ltp/getuid03.exe
PASS: winsup.api/ltp/kill01.exe
PASS: winsup.api/ltp/kill02.exe
PASS: winsup.api/ltp/kill03.exe
PASS: winsup.api/ltp/kill04.exe
PASS: winsup.api/ltp/kill09.exe
FAIL: winsup.api/ltp/link02.exe
FAIL: winsup.api/ltp/link03.exe
FAIL: winsup.api/ltp/link04.exe
FAIL: winsup.api/ltp/link05.exe
FAIL: winsup.api/ltp/lseek01.exe
FAIL: winsup.api/ltp/lseek02.exe
FAIL: winsup.api/ltp/lseek03.exe
FAIL: winsup.api/ltp/lseek04.exe
FAIL: winsup.api/ltp/lseek05.exe
FAIL: winsup.api/ltp/lseek06.exe
FAIL: winsup.api/ltp/lseek07.exe
FAIL: winsup.api/ltp/lseek08.exe
FAIL: winsup.api/ltp/lseek09.exe
FAIL: winsup.api/ltp/lseek10.exe
FAIL: winsup.api/ltp/lstat02.exe
FAIL: winsup.api/ltp/mkdir01.exe
FAIL: winsup.api/ltp/mkdir08.exe
FAIL: winsup.api/ltp/mknod01.exe
FAIL: winsup.api/ltp/mmap001.exe
FAIL: winsup.api/ltp/mmap02.exe
FAIL: winsup.api/ltp/mmap03.exe
FAIL: winsup.api/ltp/mmap04.exe
FAIL: winsup.api/ltp/mmap05.exe
FAIL: winsup.api/ltp/mmap06.exe
FAIL: winsup.api/ltp/mmap07.exe
FAIL: winsup.api/ltp/mmap08.exe
FAIL: winsup.api/ltp/munmap01.exe
FAIL: winsup.api/ltp/munmap02.exe
PASS: winsup.api/ltp/nice05.exe
FAIL: winsup.api/ltp/open02.exe
FAIL: winsup.api/ltp/open03.exe
FAIL: winsup.api/ltp/pathconf01.exe
PASS: winsup.api/ltp/pause01.exe
PASS: winsup.api/ltp/pipe01.exe
PASS: winsup.api/ltp/pipe08.exe
PASS: winsup.api/ltp/pipe09.exe
PASS: winsup.api/ltp/pipe10.exe
PASS: winsup.api/ltp/pipe11.exe
PASS: winsup.api/ltp/poll01.exe
FAIL: winsup.api/ltp/read01.exe
FAIL: winsup.api/ltp/read04.exe
FAIL: winsup.api/ltp/readdir01.exe
FAIL: winsup.api/ltp/readlink01.exe
FAIL: winsup.api/ltp/readlink02.exe
FAIL: winsup.api/ltp/readlink03.exe
FAIL: winsup.api/ltp/rename01.exe
FAIL: winsup.api/ltp/rename02.exe
FAIL: winsup.api/ltp/rename08.exe
FAIL: winsup.api/ltp/rename10.exe
FAIL: winsup.api/ltp/rmdir01.exe
FAIL: winsup.api/ltp/rmdir04.exe
FAIL: winsup.api/ltp/rmdir05.exe
FAIL: winsup.api/ltp/sbrk01.exe
FAIL: winsup.api/ltp/select01.exe
FAIL: winsup.api/ltp/select02.exe
FAIL: winsup.api/ltp/select03.exe
PASS: winsup.api/ltp/setgid01.exe
XFAIL: winsup.api/ltp/setgroups01.exe
PASS: winsup.api/ltp/setpgid01.exe
FAIL: winsup.api/ltp/setregid01.exe
FAIL: winsup.api/ltp/setreuid01.exe
PASS: winsup.api/ltp/setuid01.exe
XFAIL: winsup.api/ltp/setuid02.exe
FAIL: winsup.api/ltp/signal03.exe
FAIL: winsup.api/ltp/stat01.exe
FAIL: winsup.api/ltp/stat02.exe
FAIL: winsup.api/ltp/stat03.exe
FAIL: winsup.api/ltp/stat05.exe
FAIL: winsup.api/ltp/stat06.exe
FAIL: winsup.api/ltp/symlink01.exe
FAIL: winsup.api/ltp/symlink02.exe
FAIL: winsup.api/ltp/symlink03.exe
FAIL: winsup.api/ltp/symlink04.exe
FAIL: winsup.api/ltp/symlink05.exe
PASS: winsup.api/ltp/sync01.exe
FAIL: winsup.api/ltp/sync02.exe
PASS: winsup.api/ltp/time01.exe
PASS: winsup.api/ltp/time02.exe
PASS: winsup.api/ltp/times01.exe
PASS: winsup.api/ltp/times02.exe
PASS: winsup.api/ltp/times03.exe
FAIL: winsup.api/ltp/truncate01.exe
FAIL: winsup.api/ltp/truncate02.exe
PASS: winsup.api/ltp/umask01.exe
PASS: winsup.api/ltp/umask02.exe
FAIL: winsup.api/ltp/umask03.exe
PASS: winsup.api/ltp/uname01.exe
FAIL: winsup.api/ltp/unlink05.exe
FAIL: winsup.api/ltp/unlink06.exe
FAIL: winsup.api/ltp/unlink07.exe
XFAIL: winsup.api/ltp/unlink08.exe
PASS: winsup.api/ltp/vfork01.exe
PASS: winsup.api/ltp/wait02.exe
PASS: winsup.api/ltp/wait401.exe
PASS: winsup.api/ltp/wait402.exe
FAIL: winsup.api/ltp/write01.exe
FAIL: winsup.api/ltp/write02.exe
FAIL: winsup.api/ltp/write03.exe
PASS: winsup.api/pthread/cancel1.exe
PASS: winsup.api/pthread/cancel10.exe
FAIL: winsup.api/pthread/cancel11.exe
PASS: winsup.api/pthread/cancel12.exe
PASS: winsup.api/pthread/cancel2.exe
PASS: winsup.api/pthread/cancel3.exe
PASS: winsup.api/pthread/cancel4.exe
PASS: winsup.api/pthread/cancel5.exe
PASS: winsup.api/pthread/cancel6.exe
PASS: winsup.api/pthread/cancel7.exe
PASS: winsup.api/pthread/cancel8.exe
PASS: winsup.api/pthread/cancel9.exe
PASS: winsup.api/pthread/cleanup2.exe
PASS: winsup.api/pthread/cleanup3.exe
PASS: winsup.api/pthread/condvar1.exe
PASS: winsup.api/pthread/condvar2.exe
PASS: winsup.api/pthread/condvar2_1.exe
PASS: winsup.api/pthread/condvar3.exe
PASS: winsup.api/pthread/condvar3_1.exe
PASS: winsup.api/pthread/condvar3_2.exe
PASS: winsup.api/pthread/condvar3_3.exe
PASS: winsup.api/pthread/condvar4.exe
PASS: winsup.api/pthread/condvar5.exe
PASS: winsup.api/pthread/condvar6.exe
PASS: winsup.api/pthread/condvar7.exe
PASS: winsup.api/pthread/condvar8.exe
PASS: winsup.api/pthread/condvar9.exe
PASS: winsup.api/pthread/count1.exe
PASS: winsup.api/pthread/create1.exe
PASS: winsup.api/pthread/create2.exe
PASS: winsup.api/pthread/equal1.exe
PASS: winsup.api/pthread/exit1.exe
PASS: winsup.api/pthread/exit2.exe
PASS: winsup.api/pthread/exit3.exe
PASS: winsup.api/pthread/inherit1.exe
PASS: winsup.api/pthread/join0.exe
PASS: winsup.api/pthread/join1.exe
PASS: winsup.api/pthread/join2.exe
PASS: winsup.api/pthread/mainthreadexits.exe
PASS: winsup.api/pthread/mutex1.exe
PASS: winsup.api/pthread/mutex1d.exe
PASS: winsup.api/pthread/mutex1e.exe
PASS: winsup.api/pthread/mutex1n.exe
PASS: winsup.api/pthread/mutex1r.exe
PASS: winsup.api/pthread/mutex2.exe
PASS: winsup.api/pthread/mutex3.exe
PASS: winsup.api/pthread/mutex4.exe
PASS: winsup.api/pthread/mutex5.exe
PASS: winsup.api/pthread/mutex6d.exe
PASS: winsup.api/pthread/mutex6e.exe
PASS: winsup.api/pthread/mutex6n.exe
PASS: winsup.api/pthread/mutex6r.exe
PASS: winsup.api/pthread/mutex7.exe
PASS: winsup.api/pthread/mutex7d.exe
PASS: winsup.api/pthread/mutex7e.exe
PASS: winsup.api/pthread/mutex7n.exe
PASS: winsup.api/pthread/mutex7r.exe
PASS: winsup.api/pthread/mutex8e.exe
PASS: winsup.api/pthread/mutex8n.exe
PASS: winsup.api/pthread/mutex8r.exe
PASS: winsup.api/pthread/once1.exe
PASS: winsup.api/pthread/priority1.exe
PASS: winsup.api/pthread/priority2.exe
PASS: winsup.api/pthread/rwlock1.exe
PASS: winsup.api/pthread/rwlock2.exe
PASS: winsup.api/pthread/rwlock3.exe
PASS: winsup.api/pthread/rwlock4.exe
PASS: winsup.api/pthread/rwlock5.exe
PASS: winsup.api/pthread/rwlock6.exe
PASS: winsup.api/pthread/rwlock7.exe
PASS: winsup.api/pthread/self1.exe
PASS: winsup.api/pthread/self2.exe
PASS: winsup.api/pthread/threadidafterfork.exe
PASS: winsup.api/pthread/tsd1.exe
XFAIL: winsup.api/samples/sample-fail.exe
PASS: winsup.api/samples/sample-pass.exe
FAIL: mingw/cygload
============================================================================
Testsuite summary for Cygwin 0
============================================================================
# TOTAL: 282
# PASS:  148
# SKIP:  0
# XFAIL: 4
# FAIL:  130
# XPASS: 0
# ERROR: 0