Merging my work

[illwieckz]

Things can be merged in this order:

• Restore ld_bfd finding libgcc to link with it, fix bootstrap build for linux-armhf #33
• Cleanup DEPS, BUILD.gn, keep build/ available #27
• Fix incompatible pointer type in Windows code (nacl_host_dir.c) #29
• Fix incompatible pointer type in Windows code (nacl_host_desc.c) #30
• Make the loader runs on 16k page size kernel with SCons (2) #16
• Make the loader code compatible with MinGW (2) #17
• CPU detection for MinGW #22
• Prepare for the CMake plumbing to build the loader with MinGW (2) #18
• CMake plumbing to build the loader (2) #14
• Make the loader runs on 16k page size kernel with CMake (2) #15

[illwieckz]

As mentioned there:

• Rework external_deps for Saigo and CMake-based NaCL rebuild Daemon#1986

The whole can be tested at once with my illwieckz/staging branch:

• https://github.com/DaemonEngine/native_client/tree/illwieckz/staging

[slipher]

I've been trying to get the Windows build running using the Chromium toolchains for a couple days, so that I can test whether there are any regressions. It's a bit slow going since there some annoying issues. For example the non-Linux toolchain lists were deleted, so I have to restore them by reverting that commit.

I also tried to set it up with your Saigo toolchain build, but the Windows executables didn't seem to work (reported on the other repo).

[illwieckz]

I also tried to set it up with your Saigo toolchain build, but the Windows executables didn't seem to work

I suspect the Windows build went wrong: the tarball is way smaller than some of my previous builds. I'll retry them.

(reported on the other repo).

Link?

[slipher]

DaemonEngine/saigo-release-scripts#2

[slipher]

Finally got Windows builds going good with Scons so I can run the tests. There were a lot of build system mysteries to unravel.

Why are there two build systems? The GN build system is/was used to build NaCl for Chromium. But it mostly
doesn't have any tests. So the Scons build continued being maintained for CI and testing purposes. (The reliability of testing with a different build is questionable though, as this software is extremely sensitive to compiler/linker/etc. flag changes!)

How are you even supposed to build with MSVC? There were a few layers of bit rot:

• --no-clang must always be specified or it will use a nonexistent Clang path.
• In 2023 non-Linux builds were shut down. So there are a few changes afterward that add MSVC-incompatible flags that need to be fixed.
• Before that, there was a commit that modifies MSVC builds so that they only work with a special file on Google's build bot.
• The vendored Scons must not be used since it is too old to recognize recent MSVC toolchains.

I learned from commit 406521b that as of 2020, cl.exe (regular MSVC compiler) uses were mostly removed from Chromium builds. They are using clang-cl instead. The Scons file can also be made to work with clang-cl with very minor modifications.

[slipher]

I've run across a potential issue with using MinGW. A crash handling routine in src/trusted/service_runtime/win/exception_patch/intercept.S assumes the "native" ABI for thread-local storage. Most current MinGW toolchains are using a different one, "emutls". Apparently the option to use the native ABI was recently added with GCC 16, but it wouldn't be available in many distros yet. It sounds like this is something that must be configured, so just because a MinGW toolchain updated to GCC 16 doesn't necessarily mean it's using the new ABI.

This is the kind of stuff I'm talking about when I say running the tests with a different build would provide a lot less value for this software in particular. I think my external crash test should be able to catch this one, however. Could you try that with a MinGW build?

[slipher]

Note that clang-cl is said to already use the native TLS ABI, so that could be alternative to MinGW if cross-compiling is desired, though there is the annoying bit of needing to download a non-libre Microsoft SDK to get the headers. As mentioned above Chromium built everything with clang-cl; the MSVC build which is working now on the slipher/fixed-upstream branch can be modified to use clang-cl with a patch of less than 10 lines. (Cross-compiling with clang-cl would be somewhat more involved though.)

[illwieckz]

Thanks for those investigations! That's very helpful! ❤️

It looks like we will have good quality Native Client… 🙂️

[illwieckz]

I think my external DaemonEngine/Daemon#1785 should be able to catch this one, however. Could you try that with a MinGW build?

I'll do, but probably not before some days.