godotengine
/
godot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
godot/editor/SCsub

121 lines
4.7 KiB
Plaintext
Raw Normal View History

SCsub: Add python shebang as a hint for syntax highlighting Also switch existing shebangs to "better" /usr/bin/env python.
2016-10-17 06:50:25 +00:00
#!/usr/bin/env python
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
Import("env")
SCons: Build thirdparty code in own env, disable warnings Also remove unnecessary `Export('env')` in other SCsubs, Export should only be used when exporting *new* objects.
2018-09-28 11:29:52 +00:00
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
env.editor_sources = []
GODOT IS OPEN SOURCE
2014-02-10 01:10:30 +00:00
Implements modules documents Editor can generate documents for modules in thier own xml files
2016-12-16 11:12:22 +00:00
import os
Add `custom_modules` build option to compile external user modules This patch adds ability to include external, user-defined C++ modules to be compiled as part of Godot via `custom_modules` build option which can be passed to `scons`. ``` scons platform=x11 tools=yes custom_modules="../project/modules" ``` Features: - detects all available modules under `custom_modules` directory the same way as it does for built-in modules (not recursive); - works with both relative and absolute paths on the filesystem; - multiple search paths can be specified as a comma-separated list. Module custom documentation and editor icons collection and generation process is adapted to work with absolute paths needed by such modules. Also fixed doctool bug mixing absolute and relative paths respectively. Implementation details: - `env.module_list` is a dictionary now, which holds both module name as key and either a relative or absolute path to a module as a value. - `methods.detect_modules` is run twice: once for built-in modules, and second for external modules, all combined later. - `methods.detect_modules` was not doing what it says on the tin. It is split into `detect_modules` which collects a list of available modules and `write_modules` which generates `register_types` sources for each. - whether a module is built-in or external is distinguished by relative or absolute paths respectively. `custom_modules` scons converter ensures that the path is absolute even if relative path is supplied, including expanding user paths and symbolic links. - treats the parent directory as if it was Godot's base directory, so that there's no need to change include paths in cases where custom modules are included as dependencies in other modules. (cherry picked from commit a96f0e98d74839fecfe6ac553aa5a5521e69ddfd)
2020-04-04 11:46:15 +00:00
import glob
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
from platform_methods import run_in_subprocess
from compat import open_utf8
import editor_builders
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
Changed the doc class generation to individual files per class. It is also possible to save module files in module directories and the build system will recognize them.
2017-09-12 20:42:36 +00:00
def _make_doc_data_class_path(to_path):
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
# NOTE: It is safe to generate this file here, since this is still executed serially
g = open_utf8(os.path.join(to_path, "doc_data_class_path.gen.h"), "w")
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
g.write("static const int _doc_data_class_path_count = " + str(len(env.doc_class_path)) + ";\n")
g.write("struct _DocDataClassPath { const char* name; const char* path; };\n")
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
g.write("static const _DocDataClassPath _doc_data_class_paths[" + str(len(env.doc_class_path) + 1) + "] = {\n")
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
for c in sorted(env.doc_class_path):
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
g.write('\t{"' + c + '", "' + env.doc_class_path[c] + '"},\n')
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
g.write("\t{NULL, NULL}\n")
g.write("};\n")
Changed the doc class generation to individual files per class. It is also possible to save module files in module directories and the build system will recognize them.
2017-09-12 20:42:36 +00:00
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
g.close()
Changed the doc class generation to individual files per class. It is also possible to save module files in module directories and the build system will recognize them.
2017-09-12 20:42:36 +00:00
GODOT IS OPEN SOURCE
2014-02-10 01:10:30 +00:00
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
if env["tools"]:
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
# Register exporters
style: Fix PEP8 whitespace issues in Python files Done with `autopep8 --select=E2,W2`, fixes: - E201 - Remove extraneous whitespace. - E202 - Remove extraneous whitespace. - E203 - Remove extraneous whitespace. - E211 - Remove extraneous whitespace. - E221 - Fix extraneous whitespace around keywords. - E222 - Fix extraneous whitespace around keywords. - E223 - Fix extraneous whitespace around keywords. - E224 - Remove extraneous whitespace around operator. - E225 - Fix missing whitespace around operator. - E226 - Fix missing whitespace around operator. - E227 - Fix missing whitespace around operator. - E228 - Fix missing whitespace around operator. - E231 - Add missing whitespace. - E231 - Fix various deprecated code (via lib2to3). - E241 - Fix extraneous whitespace around keywords. - E242 - Remove extraneous whitespace around operator. - E251 - Remove whitespace around parameter '=' sign. - E261 - Fix spacing after comment hash. - E262 - Fix spacing after comment hash. - E265 - Format block comments. - E271 - Fix extraneous whitespace around keywords. - E272 - Fix extraneous whitespace around keywords. - E273 - Fix extraneous whitespace around keywords. - E274 - Fix extraneous whitespace around keywords. - W291 - Remove trailing whitespace. - W293 - Remove trailing whitespace.
2016-10-30 17:57:40 +00:00
reg_exporters_inc = '#include "register_exporters.h"\n'
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
reg_exporters = "void register_exporters() {\n"
style: Start applying PEP8 to Python files, indentation issues Done with `autopep8 --select=E1`, fixes: - E101 - Reindent all lines. - E112 - Fix under-indented comments. - E113 - Fix over-indented comments. - E115 - Fix under-indented comments. - E116 - Fix over-indented comments. - E121 - Fix a badly indented line. - E122 - Fix a badly indented line. - E123 - Fix a badly indented line. - E124 - Fix a badly indented line. - E125 - Fix indentation undistinguish from the next logical line. - E126 - Fix a badly indented line. - E127 - Fix a badly indented line. - E128 - Fix a badly indented line. - E129 - Fix a badly indented line.
2016-10-30 17:44:57 +00:00
for e in env.platform_exporters:
Refactor Android platform export structure.
2021-09-09 16:43:49 +00:00
# Glob all .cpp files in export folder
Single Compilation Unit build. Adds support for simple SCU build. This speeds up compilation by compiling multiple cpp files within a single translation unit.
2022-11-27 08:34:07 +00:00
if not env["scu_build"]:
files = Glob("#platform/" + e + "/export/" + "*.cpp")
env.add_source_files(env.editor_sources, files)
Refactor Android platform export structure.
2021-09-09 16:43:49 +00:00
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
reg_exporters += "\tregister_" + e + "_exporter();\n"
style: Fix PEP8 whitespace issues in Python files Done with `autopep8 --select=E2,W2`, fixes: - E201 - Remove extraneous whitespace. - E202 - Remove extraneous whitespace. - E203 - Remove extraneous whitespace. - E211 - Remove extraneous whitespace. - E221 - Fix extraneous whitespace around keywords. - E222 - Fix extraneous whitespace around keywords. - E223 - Fix extraneous whitespace around keywords. - E224 - Remove extraneous whitespace around operator. - E225 - Fix missing whitespace around operator. - E226 - Fix missing whitespace around operator. - E227 - Fix missing whitespace around operator. - E228 - Fix missing whitespace around operator. - E231 - Add missing whitespace. - E231 - Fix various deprecated code (via lib2to3). - E241 - Fix extraneous whitespace around keywords. - E242 - Remove extraneous whitespace around operator. - E251 - Remove whitespace around parameter '=' sign. - E261 - Fix spacing after comment hash. - E262 - Fix spacing after comment hash. - E265 - Format block comments. - E271 - Fix extraneous whitespace around keywords. - E272 - Fix extraneous whitespace around keywords. - E273 - Fix extraneous whitespace around keywords. - E274 - Fix extraneous whitespace around keywords. - W291 - Remove trailing whitespace. - W293 - Remove trailing whitespace.
2016-10-30 17:57:40 +00:00
reg_exporters_inc += '#include "platform/' + e + '/export/export.h"\n'
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
reg_exporters += "}\n"
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
Single Compilation Unit build. Adds support for simple SCU build. This speeds up compilation by compiling multiple cpp files within a single translation unit.
2022-11-27 08:34:07 +00:00
# If we are in a SCU build, we add all the platform exporters in one SCU file.
if env["scu_build"]:
env.add_source_files(env.editor_sources, "../platform/*.cpp")
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
# NOTE: It is safe to generate this file here, since this is still executed serially
Properly closing all files in Python code
2018-03-10 17:37:33 +00:00
with open_utf8("register_exporters.gen.cpp", "w") as f:
f.write(reg_exporters_inc)
f.write(reg_exporters)
style: Start applying PEP8 to Python files, indentation issues Done with `autopep8 --select=E1`, fixes: - E101 - Reindent all lines. - E112 - Fix under-indented comments. - E113 - Fix over-indented comments. - E115 - Fix under-indented comments. - E116 - Fix over-indented comments. - E121 - Fix a badly indented line. - E122 - Fix a badly indented line. - E123 - Fix a badly indented line. - E124 - Fix a badly indented line. - E125 - Fix indentation undistinguish from the next logical line. - E126 - Fix a badly indented line. - E127 - Fix a badly indented line. - E128 - Fix a badly indented line. - E129 - Fix a badly indented line.
2016-10-30 17:44:57 +00:00
Add `custom_modules` build option to compile external user modules This patch adds ability to include external, user-defined C++ modules to be compiled as part of Godot via `custom_modules` build option which can be passed to `scons`. ``` scons platform=x11 tools=yes custom_modules="../project/modules" ``` Features: - detects all available modules under `custom_modules` directory the same way as it does for built-in modules (not recursive); - works with both relative and absolute paths on the filesystem; - multiple search paths can be specified as a comma-separated list. Module custom documentation and editor icons collection and generation process is adapted to work with absolute paths needed by such modules. Also fixed doctool bug mixing absolute and relative paths respectively. Implementation details: - `env.module_list` is a dictionary now, which holds both module name as key and either a relative or absolute path to a module as a value. - `methods.detect_modules` is run twice: once for built-in modules, and second for external modules, all combined later. - `methods.detect_modules` was not doing what it says on the tin. It is split into `detect_modules` which collects a list of available modules and `write_modules` which generates `register_types` sources for each. - whether a module is built-in or external is distinguished by relative or absolute paths respectively. `custom_modules` scons converter ensures that the path is absolute even if relative path is supplied, including expanding user paths and symbolic links. - treats the parent directory as if it was Godot's base directory, so that there's no need to change include paths in cases where custom modules are included as dependencies in other modules. (cherry picked from commit a96f0e98d74839fecfe6ac553aa5a5521e69ddfd)
2020-04-04 11:46:15 +00:00
# Core API documentation.
Cleanup unnecessary debug prints
2017-10-21 17:31:23 +00:00
docs = []
Add `custom_modules` build option to compile external user modules This patch adds ability to include external, user-defined C++ modules to be compiled as part of Godot via `custom_modules` build option which can be passed to `scons`. ``` scons platform=x11 tools=yes custom_modules="../project/modules" ``` Features: - detects all available modules under `custom_modules` directory the same way as it does for built-in modules (not recursive); - works with both relative and absolute paths on the filesystem; - multiple search paths can be specified as a comma-separated list. Module custom documentation and editor icons collection and generation process is adapted to work with absolute paths needed by such modules. Also fixed doctool bug mixing absolute and relative paths respectively. Implementation details: - `env.module_list` is a dictionary now, which holds both module name as key and either a relative or absolute path to a module as a value. - `methods.detect_modules` is run twice: once for built-in modules, and second for external modules, all combined later. - `methods.detect_modules` was not doing what it says on the tin. It is split into `detect_modules` which collects a list of available modules and `write_modules` which generates `register_types` sources for each. - whether a module is built-in or external is distinguished by relative or absolute paths respectively. `custom_modules` scons converter ensures that the path is absolute even if relative path is supplied, including expanding user paths and symbolic links. - treats the parent directory as if it was Godot's base directory, so that there's no need to change include paths in cases where custom modules are included as dependencies in other modules. (cherry picked from commit a96f0e98d74839fecfe6ac553aa5a5521e69ddfd)
2020-04-04 11:46:15 +00:00
docs += Glob("#doc/classes/*.xml")
Added modules' doc files to make_doc_header They were somehow never included. This fixes that.
2017-11-16 17:53:54 +00:00
Add `custom_modules` build option to compile external user modules This patch adds ability to include external, user-defined C++ modules to be compiled as part of Godot via `custom_modules` build option which can be passed to `scons`. ``` scons platform=x11 tools=yes custom_modules="../project/modules" ``` Features: - detects all available modules under `custom_modules` directory the same way as it does for built-in modules (not recursive); - works with both relative and absolute paths on the filesystem; - multiple search paths can be specified as a comma-separated list. Module custom documentation and editor icons collection and generation process is adapted to work with absolute paths needed by such modules. Also fixed doctool bug mixing absolute and relative paths respectively. Implementation details: - `env.module_list` is a dictionary now, which holds both module name as key and either a relative or absolute path to a module as a value. - `methods.detect_modules` is run twice: once for built-in modules, and second for external modules, all combined later. - `methods.detect_modules` was not doing what it says on the tin. It is split into `detect_modules` which collects a list of available modules and `write_modules` which generates `register_types` sources for each. - whether a module is built-in or external is distinguished by relative or absolute paths respectively. `custom_modules` scons converter ensures that the path is absolute even if relative path is supplied, including expanding user paths and symbolic links. - treats the parent directory as if it was Godot's base directory, so that there's no need to change include paths in cases where custom modules are included as dependencies in other modules. (cherry picked from commit a96f0e98d74839fecfe6ac553aa5a5521e69ddfd)
2020-04-04 11:46:15 +00:00
# Module API documentation.
module_dirs = []
for d in env.doc_class_path.values():
if d not in module_dirs:
module_dirs.append(d)
Added modules' doc files to make_doc_header They were somehow never included. This fixes that.
2017-11-16 17:53:54 +00:00
Add `custom_modules` build option to compile external user modules This patch adds ability to include external, user-defined C++ modules to be compiled as part of Godot via `custom_modules` build option which can be passed to `scons`. ``` scons platform=x11 tools=yes custom_modules="../project/modules" ``` Features: - detects all available modules under `custom_modules` directory the same way as it does for built-in modules (not recursive); - works with both relative and absolute paths on the filesystem; - multiple search paths can be specified as a comma-separated list. Module custom documentation and editor icons collection and generation process is adapted to work with absolute paths needed by such modules. Also fixed doctool bug mixing absolute and relative paths respectively. Implementation details: - `env.module_list` is a dictionary now, which holds both module name as key and either a relative or absolute path to a module as a value. - `methods.detect_modules` is run twice: once for built-in modules, and second for external modules, all combined later. - `methods.detect_modules` was not doing what it says on the tin. It is split into `detect_modules` which collects a list of available modules and `write_modules` which generates `register_types` sources for each. - whether a module is built-in or external is distinguished by relative or absolute paths respectively. `custom_modules` scons converter ensures that the path is absolute even if relative path is supplied, including expanding user paths and symbolic links. - treats the parent directory as if it was Godot's base directory, so that there's no need to change include paths in cases where custom modules are included as dependencies in other modules. (cherry picked from commit a96f0e98d74839fecfe6ac553aa5a5521e69ddfd)
2020-04-04 11:46:15 +00:00
for d in module_dirs:
if not os.path.isabs(d):
docs += Glob("#" + d + "/*.xml") # Built-in.
else:
docs += Glob(d + "/*.xml") # Custom.
Changed the doc class generation to individual files per class. It is also possible to save module files in module directories and the build system will recognize them.
2017-09-12 20:42:36 +00:00
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
_make_doc_data_class_path(env.Dir("#editor/doc").abspath)
Changed the doc class generation to individual files per class. It is also possible to save module files in module directories and the build system will recognize them.
2017-09-12 20:42:36 +00:00
Improve code style of generated headers
2017-11-15 19:16:51 +00:00
docs = sorted(docs)
BuildSystem: generated files have .gen.extension
2017-06-23 15:03:41 +00:00
env.Depends("#editor/doc_data_compressed.gen.h", docs)
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
env.CommandNoCache("#editor/doc_data_compressed.gen.h", docs, run_in_subprocess(editor_builders.make_doc_header))
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
# Editor interface and class reference translations incur a significant size
# cost for the editor binary (see godot-proposals#3421).
# To limit it, we only include translations with a high enough completion
i18n: Sync editor translations with Weblate
2022-04-25 15:14:49 +00:00
# ratio (20% for the editor UI, 10% for the class reference).
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
# Generated with `make include-list` for each resource.
# Note: In 3.x, we also exclude languages that depend on complex text
# layouts to be displayed properly: ar,bn,fa,he,hi,ml,si,ta,te,ur.
style: Start applying PEP8 to Python files, indentation issues Done with `autopep8 --select=E1`, fixes: - E101 - Reindent all lines. - E112 - Fix under-indented comments. - E113 - Fix over-indented comments. - E115 - Fix under-indented comments. - E116 - Fix over-indented comments. - E121 - Fix a badly indented line. - E122 - Fix a badly indented line. - E123 - Fix a badly indented line. - E124 - Fix a badly indented line. - E125 - Fix indentation undistinguish from the next logical line. - E126 - Fix a badly indented line. - E127 - Fix a badly indented line. - E128 - Fix a badly indented line. - E129 - Fix a badly indented line.
2016-10-30 17:44:57 +00:00
i18n: Add support for translating the class reference - Parse `.po` files from `doc/translations/*.po` like already done with `editor/translations/*.po`. - Add logic to register a doc translation mapping in `TranslationServer` and `EditorSettings`. - Add `DTR()` to lookup the doc translation mapping (similar to `TTR()`). Strings are automatically dedented and stripped of whitespace to ensure that they would match the translation catalog. - Use `DTR()` to translate relevant strings in `EditorHelp`, `EditorInspector`, `CreateDialog`, `ConnectionsDialog`. - Small simplification to `TranslationLoaderPO`, the path argument was not really meaningful. (cherry picked from commit 4857648a16585bbd0fb2fbc33d3d0f768b8223b5)
2020-03-18 17:34:36 +00:00
# Editor translations
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
to_include = (
i18n: Sync editor translations with Weblate
2022-07-27 13:31:01 +00:00
"bg,ca,cs,de,el,eo,es_AR,es,fi,fr,gl,hu,id,it,ja,ko,lv,ms,nb,nl,pl,pt_BR,pt,ro,ru,sk,sv,th,tr,uk,vi,zh_CN,zh_TW"
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
).split(",")
tlist = [env.Dir("#editor/translations").abspath + "/" + f + ".po" for f in to_include]
i18n: Add support for translating the class reference - Parse `.po` files from `doc/translations/*.po` like already done with `editor/translations/*.po`. - Add logic to register a doc translation mapping in `TranslationServer` and `EditorSettings`. - Add `DTR()` to lookup the doc translation mapping (similar to `TTR()`). Strings are automatically dedented and stripped of whitespace to ensure that they would match the translation catalog. - Use `DTR()` to translate relevant strings in `EditorHelp`, `EditorInspector`, `CreateDialog`, `ConnectionsDialog`. - Small simplification to `TranslationLoaderPO`, the path argument was not really meaningful. (cherry picked from commit 4857648a16585bbd0fb2fbc33d3d0f768b8223b5)
2020-03-18 17:34:36 +00:00
env.Depends("#editor/editor_translations.gen.h", tlist)
env.CommandNoCache(
"#editor/editor_translations.gen.h", tlist, run_in_subprocess(editor_builders.make_editor_translations_header)
)
# Documentation translations
i18n: Sync classref translations with Weblate German (de) is now above the threshold for inclusion.
2021-11-03 14:00:43 +00:00
to_include = "de,es,fr,ja,zh_CN".split(",")
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
tlist = [env.Dir("#doc/translations").abspath + "/" + f + ".po" for f in to_include]
i18n: Add support for translating the class reference - Parse `.po` files from `doc/translations/*.po` like already done with `editor/translations/*.po`. - Add logic to register a doc translation mapping in `TranslationServer` and `EditorSettings`. - Add `DTR()` to lookup the doc translation mapping (similar to `TTR()`). Strings are automatically dedented and stripped of whitespace to ensure that they would match the translation catalog. - Use `DTR()` to translate relevant strings in `EditorHelp`, `EditorInspector`, `CreateDialog`, `ConnectionsDialog`. - Small simplification to `TranslationLoaderPO`, the path argument was not really meaningful. (cherry picked from commit 4857648a16585bbd0fb2fbc33d3d0f768b8223b5)
2020-03-18 17:34:36 +00:00
env.Depends("#editor/doc_translations.gen.h", tlist)
env.CommandNoCache(
"#editor/doc_translations.gen.h", tlist, run_in_subprocess(editor_builders.make_doc_translations_header)
)
style: Start applying PEP8 to Python files, indentation issues Done with `autopep8 --select=E1`, fixes: - E101 - Reindent all lines. - E112 - Fix under-indented comments. - E113 - Fix over-indented comments. - E115 - Fix under-indented comments. - E116 - Fix over-indented comments. - E121 - Fix a badly indented line. - E122 - Fix a badly indented line. - E123 - Fix a badly indented line. - E124 - Fix a badly indented line. - E125 - Fix indentation undistinguish from the next logical line. - E126 - Fix a badly indented line. - E127 - Fix a badly indented line. - E128 - Fix a badly indented line. - E129 - Fix a badly indented line.
2016-10-30 17:44:57 +00:00
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
# Fonts
i18n: Only include editor translations above a threshold This reduces the size of the editor binaries significantly, as we otherwise embed all WIP translations, including ones with very low completion ratios, and end up paying for the size of all `msgid`s for each locale. Cf. https://github.com/godotengine/godot-proposals/issues/3421 for details. The thresholds used are: - 30% for the editor interface (should already include most common strings while more obscure ones like UndoRedo action names might be untranslated). - 10% for the class reference: this is a HUGE resource and 10% is already a lot of useful content, especially if focused on the most used APIs. For 3.x, we also exclude languages that require complex text layout support to be displayed properly. This currently reduces the size of the editor binary by 17% on Linux. The list will be synced manually every now and then.
2021-10-20 11:47:50 +00:00
flist = glob.glob(env.Dir("#thirdparty").abspath + "/fonts/*.ttf")
flist.extend(glob.glob(env.Dir("#thirdparty").abspath + "/fonts/*.otf"))
Add binary MO translation file support. Add brotli decoder and WOFF2 support. Use smaller .mo files instead of .po, if gettext is available. Convert editor fonts to .woff2 format.
2022-03-25 17:31:09 +00:00
flist.extend(glob.glob(env.Dir("#thirdparty").abspath + "/fonts/*.woff"))
flist.extend(glob.glob(env.Dir("#thirdparty").abspath + "/fonts/*.woff2"))
BuildSystem: Sort input file lists so that godot package builds reproducibly in spite of indeterministic filesystem readdir order and http://bugs.python.org/issue30461 See https://reproducible-builds.org/ for why this is good. Sort font input file list, so that builtin_fonts.gen.h is created in a reproducible way Sort list of platforms, so that editor/register_exporters.gen.cpp is created in a reproducible way Sort list of source files, so that .a files and resulting godot binaries are created in a reproducible way
2018-08-28 18:40:51 +00:00
flist.sort()
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
env.Depends("#editor/builtin_fonts.gen.h", flist)
env.CommandNoCache("#editor/builtin_fonts.gen.h", flist, run_in_subprocess(editor_builders.make_fonts_header))
Running builder (content generator) functions in subprocesses on Windows - Refactored all builder (make_*) functions into separate Python modules along to the build tree - Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere - Introduced stub to use the builders module as a stand alone script and invoke a selected function There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp) on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky builds. Running all such content generators in a new subprocess instead of directly inside the build script works around the issue. Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle. Suggested workaround did not fully work either. Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of running a cross-compilation on Windows they would still be used, but likely it will not happen in practice. What counts is that the build itself is running on which platform, not the target platform. Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.
2018-03-17 22:23:55 +00:00
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
env.add_source_files(env.editor_sources, "*.cpp")
SCons: List `.gen.cpp` sources explicitly to avoid globbing errors Whenever we change the name (or remove) generated cpp files with the `.gen.cpp` extension, users run into build issues when switching between branches (i.e. switching before and after the name change/removal). This is because we glob `*.cpp` so if a now-obsolete file from a previous build is present, we'll include it too, potentially leading to bugs or compilation failure (due to missing headers or invalid code). So globbing patterns in `add_source_files` will now skip files ending with `.gen.cpp`, which should instead be passed explicitly where they're used. (cherry picked from commit c13348053129d4a356a0df1ada208809997799fe)
2021-10-15 19:59:11 +00:00
env.add_source_files(env.editor_sources, "register_exporters.gen.cpp")
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
SCons: Format buildsystem files with psf/black Configured for a max line length of 120 characters. psf/black is very opinionated and purposely doesn't leave much room for configuration. The output is mostly OK so that should be fine for us, but some things worth noting: - Manually wrapped strings will be reflowed, so by using a line length of 120 for the sake of preserving readability for our long command calls, it also means that some manually wrapped strings are back on the same line and should be manually merged again. - Code generators using string concatenation extensively look awful, since black puts each operand on a single line. We need to refactor these generators to use more pythonic string formatting, for which many options are available (`%`, `format` or f-strings). - CI checks and a pre-commit hook will be added to ensure that future buildsystem changes are well-formatted. (cherry picked from commit cd4e46ee65dab6baa6a143bf3b3f64244be36712)
2020-03-30 06:28:32 +00:00
SConscript("collada/SCsub")
SConscript("doc/SCsub")
SConscript("fileserver/SCsub")
SConscript("icons/SCsub")
SConscript("import/SCsub")
SConscript("plugins/SCsub")
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
disable caching for targets using helper functions
2017-11-28 20:27:57 +00:00
lib = env.add_library("editor", env.editor_sources)
Reorder the folders in tools to prepare moving tools/editor - `certs` and `editor_fonts` go to `thirdparty` - `dist` and `scripts` go to a new `misc` folder - `collada` and `doc` go to `tools/editor` The next step will be to rename `tools/editor` to `editor` directly, but this will be done at the right time to avoid breaking too many PRs.
2017-02-08 23:07:44 +00:00
env.Prepend(LIBS=[lib])