We might be able to make it work by building Mono itself with
ASLR protection too, but there might still be issues when loading
e.g. GDNative DLLs built without ASLR protection.
In the short term this is not a goal, but we can reconsider later
what is actually needed for ASLR protection to work and keep things
user-friendly.
This commit changes the way Thread::caller_id works. By moving caller_id
to the .cpp file we make sure that the TLS variable doesn't get
relocated twice causing a crash. Since we build with LTO for release
builds (and everyone should be doing that anyway) there is no extra
overhead from the non-static method. We do do an extra bool check now
there but I don't think this will add much in the way of overhead.
This check cannot be avoided if we still want to be able to cache the
thread ID hash, as we had to move the setter because of limitations of
the WinRT platform. The original workaround for this was in #46813 but
this has some unintended consequences. Specifically; threads that never
create a Thread object will always return 0 in Thread::get_caller_id()
which caused a regression. For instance the editor now freezes when
importing large textures. This PR also addresses that.
Additionally we now enable ASLR support when building with MingW, this
includes a workaround for MingW. MingW refuses to create an appropriate
relocation table if no symbols are exported. So we just export the
various main() functions in godot_windows.cpp.
While ASLR support isn't criticial for Godot, previous versions of Godot
just happened to work with a dynamic base 'by accident' and some users
run Godot this way. After the thread change the .tls section now needs
relocations to make this work. By enabling ASLR at build-time we create
these relocations and people who forced ALSR on previously will now get
a working Godot again.
This fixes#47256 and fixes#47219
This is the 3.x version of this PR. For master a different approach is
possible which I will make in the coming days.
Exposes AddressSanitizer support in MSVC compiler. Can be installed via individual
components in the Visual Studio 2019 Installer.
Disabled by default. Compile the engine with `scons use_asan=yes`.
(cherry picked from commit 53901d870b)
This is meant for users making custom builds to match the options used on
optimized, official builds.
This enables, on the platforms which support them:
- `use_static_cpp=yes` (portable binaries for Linux and Windows)
- `use_lto=yes` (link time optimizations - note: requires a lot of RAM!)
- `debug_symbols=no` (no debug symbols, smaller binaries)
Also abort when using MSVC with `production=yes`, as:
- It cannot optimize the GDScript VM like GCC or Clang do, leading to
significant performance drops.
- Its LTO support is unreliable, at least used to trigger crashes last
we tried it extensively.
All options can still be overridden if specified, and the `dev=yes` option
was changed to also support overrides.
(cherry picked from commit db26871210)
Otherwise we can get situations where platform-specific opts with the same name
can override each other depending on the order at which platforms are parsed,
as was the case with `use_static_cpp` in Linux/Windows.
Fixes#44304.
This also has the added benefit that the `scons --help` output will now only
include the options which are relevant for the selected (or detected) platform.
(cherry picked from commit 0f84d8dc49)
`debug_symbols=yes` will now behave like `debug_symbols=full` did
before. The difference in compressed file sizes is not that large,
which means there isn't much point in having two different values.
This helps make the buildsystem easier to understand.
(cherry picked from commit ff1f0d2cb5)
Its last use was removed in Godot 3.0, so it no longer makes sense to define.
Also removed `D3D_DEBUG_INFO` for Windows as it's likely a left over from a
long time ago pre-opensourcing when Godot had some form of Direct3D 9 support?
(cherry picked from commit dcf902df85)
Configured for a max line length of 120 characters.
psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:
- Manually wrapped strings will be reflowed, so by using a line length
of 120 for the sake of preserving readability for our long command
calls, it also means that some manually wrapped strings are back on
the same line and should be manually merged again.
- Code generators using string concatenation extensively look awful,
since black puts each operand on a single line. We need to refactor
these generators to use more pythonic string formatting, for which
many options are available (`%`, `format` or f-strings).
- CI checks and a pre-commit hook will be added to ensure that future
buildsystem changes are well-formatted.
(cherry picked from commit cd4e46ee65)
The new 'split_libmodules=yes' option is useful to work around linker
command line size limitations when linking a huge number of objects.
We're currently over 64k chars when linking libmodules.a on Windows
with MinGW, which triggers issues as seen in #30892.
Even on Linux, we can also reach linker command line size limitations
by adding more custom modules.
We force this option to True for MinGW on Windows, which fixes#30892.
Additional changes to lib splitting:
- Fix linking of the split module libs with interdependent symbols,
hacking our way into LINKCOM and SHLINKCOM to set the `--start-group`
and `--end-group` flags.
- Fix Python 3 compatibility in `methods.split_lib()`.
- Drop seemingly obsolete condition for 'msys' on 'posix'.
- Drop the unnecessary 'split_drivers' as the drivers lib is no longer
too big since we moved all thirdparty builds to modules.
Co-authored-by: Hein-Pieter van Braam-Stewart <hp@tmm.cx>
It's the recommended way to set those, and is more portable
(automatically prepends -D for GCC/Clang and /D for MSVC).
We still use CPPFLAGS for some pre-processor flags which are not
defines.
Include paths are processed from left to right, so we use Prepend to
ensure that paths to bundled thirdparty files will have precedence over
system paths (e.g. `/usr/include` should have lowest priority).
Many contributors (me included) did not fully understand what CCFLAGS,
CXXFLAGS and CPPFLAGS refer to exactly, and were thus not using them
in the way they are intended to be.
As per the SCons manual: https://www.scons.org/doc/HTML/scons-user/apa.html
- CCFLAGS: General options that are passed to the C and C++ compilers.
- CFLAGS: General options that are passed to the C compiler (C only;
not C++).
- CXXFLAGS: General options that are passed to the C++ compiler. By
default, this includes the value of $CCFLAGS, so that setting
$CCFLAGS affects both C and C++ compilation.
- CPPFLAGS: User-specified C preprocessor options. These will be
included in any command that uses the C preprocessor, including not
just compilation of C and C++ source files [...], but also [...]
Fortran [...] and [...] assembly language source file[s].
TL;DR: Compiler options go to CCFLAGS, unless they must be restricted
to either C (CFLAGS) or C++ (CXXFLAGS). Preprocessor defines go to
CPPFLAGS.
On Windows, when "Language for non-Unicode programs" were set to "Japanese (Japan)", MSVC would by default use Shift JIS (code page 932) to interpret source files, which would result in test_string failing to compile because of characters in `test_34()`. Forcing utf-8 for MSVC fixes the issue
We've been defaulting to WASAPI since 3.0 and it's superior to RtAudio
in all aspects.
Obsoletes and closes#25503.
Also enable WINMIDI on MinGW, this had been missed initially.
Fix os_windows.cpp and crash_handler_windows.cpp which had weird
dependencies on RtAudio.h's includes (ugh).
While looking into a different issue, I've noticed that Visual Studio Intellisense does not work well for Godot project when using Windows Vista+ APIs (e.g. CreateThreadpool), i.e. it does not recognise the APIs because they are defined in Windows header files for Vista+ only.
This is because the WINVER and _WIN32_WINNT symbols don't have their values set in the generated Godot project file. This fixes the problem by setting the values when generating the project file.
Also increase AppVeyor cache size to 1024,
should match what is available for us in the free plan:
https://www.appveyor.com/docs/build-cache/#cache-size-beta
And drop obsolete debug_release option for Windows, superseded
by target=release and debug_symbols=yes.
SCons has good compiler detection logic for MSVC compilers. Up to now,
Godot hasn't used it; it depends on passed-in OS environment vars from
a specific Visual Studio cmd.exe windows. This makes it harder to
build from a msys or cygwin shell.
This change allows SCons to autodetect Visual Studio unless it sees
VCINSTALLDIR in the os.environ. It also adds a 'msvc_version' arg for
manual specification of compiler version, and uses the existing 'bits'
arg to specify the target architecture. More detail could be added as
desired. It also adds 'use_mingw' to always use mingw, even if Visual
Studio is installed. That uses the existing mingw setup logic.
If people are used to building Godot in a Visual Studio cmd window,
this should not change the behavior in that case, since VCINSTALLDIR
will be set in those windows. (However, note that you could now unset
that var and build with any other MSVC version or target arch, even in
that window.)
I refactored much of platform/windows/detect.py during this, to
simplify and clarify the logic. I also cleaned up a bunch of env var
settings in windows/detect.py and SConstruct to use modern SCons
idioms and simplify things.
I suspect this will also enable using the Intel compiler on Windows,
though that hasn't been tested.
Found via `codespell -q 3 --skip="./thirdparty,./editor/translations" -I ../godot-word-whitelist.txt`
Whitelist consists of:
```
ang
doubleclick
lod
nd
que
te
unselect
```
Rémi Verschelde
Hein-Pieter van Braam-Stewart
Rafał Mikrut
Andrii Doroshenko (Xrayez)
Bastiaan Olij
Marcel Admiraal
Lyubomir Vasilev
Hugo Locurcio
Rafał Mikrut
Pedro J. Estébanez
TerminalJack
Yeongho Kim
bruvzg
Kenneth Lee
Windy Darian
Hendrikto
Marcelo Fernandez
Marcin Zawiejski
Juan Linietsky
Fabio Alessandrelli
elasota
Max Hilbrunner
Gary Oberbrunner
luz.paz