Commit Graph

228 Commits

Author SHA1 Message Date
vittorioromeo 71e0082ab0 Extract 'SafeBinaryMutex' to separate header
This change simply extracts 'SafeBinaryMutex' from 'mutex.h' to
    'safe_binary_mutex.h', in an effort to reduce the compilation
    speed impact of including `mutex.h`.
2024-02-04 16:59:54 +01:00
Rémi Verschelde 3dfedd69ea
Merge pull request #87711 from matthew1006/threaded-loading-progress-fix
Fix `ResourceLoader.load_threaded_get_status` returning `[0]` constantly in exported projects.
2024-01-30 17:48:11 +01:00
Rémi Verschelde b65c495d6e
Merge pull request #86845 from RandomShaper/no_load_regress
Avoid regressing in progress reporting in resource load
2024-01-29 21:33:00 +01:00
Matthew Edwards 7565d1f3ab Push p_original_path into load_paths_stack and sub_tasks instead of p_path. 2024-01-29 17:27:57 +00:00
Rémi Verschelde dc79e956b6
Merge pull request #86587 from RandomShaper/wtp_enhance
Enhance & fix `WorkerThreadPool`
2024-01-11 17:38:09 +01:00
kobewi 2e09985c0d Fix² behavior of ResourceFormatLoader CACHE_MODE_REPLACE 2024-01-09 16:01:37 +01:00
Rémi Verschelde 9ca3d3ec83
Revert "Fix behavior of ResourceFormatLoader `CACHE_MODE_REPLACE`" 2024-01-09 10:49:14 +01:00
Pedro J. Estébanez 9444d297ed WorkerThreadPool: Overhaul scheduling and synchronization
This commits rewrites the sync logic in a way that the
`use_system_threads_for_low_priority_tasks` setting, which was added due to
the lack of a cross-platform wait-for-multiple-objects functionality, can be
removed (it's as if it was effectively hardcoded to `false`).

With the new implementation, we have the best of both worlds: threads don't
have to poll, plus no bespoke threads are used.

In addition, regarding deadlock prevention, since not every possible case of
wait-deadlock could be avoided, this commits removes the current best-effort
avoidance mechanisms and keeps only a simple, pessimistic way of detection.

It turns out that the only current user of deadlock prevention, ResourceLoader,
works fine with it and so every possible situation in resource loading is now
properly handled, with no possibilities of deadlocking. There's a comment in
the code with further details.

Lastly, a potential for load tasks never being awaited/disposed is cleared.
2024-01-08 12:45:42 +01:00
Pedro J. Estébanez f5ca58d32f Avoid regressing in progress reporting in resource load 2024-01-05 19:33:58 +01:00
Yuri Sizov 2e94be2ea4 Merge pull request #84167 from SaracenOne/cache_mode_replace_fixes
Fix behavior of ResourceFormatLoader `CACHE_MODE_REPLACE`
2023-12-08 15:23:13 +01:00
Yuri Sizov bc1949d797 Correctly check scripts that must inherit `EditorPlugin`
Also updates some error messages related to this kind of check
across the codebase.
2023-11-23 16:24:43 +01:00
Saracen f392a9c4f8 Fix CACHE_MODE_REPLACE 2023-11-15 18:55:20 +00:00
Rémi Verschelde 2bcc2a3cd0
Merge pull request #84791 from akien-mga/fix-translation-remapping-check
Fix translation remapping check for imported resources
2023-11-12 23:00:08 +01:00
Rémi Verschelde 2729a78fd7
Fix translation remapping check for imported resources
Fixes #81660.
2023-11-12 13:17:12 +01:00
kobewi a3627b6e37 Assign temporary path to preloaded resources 2023-11-10 00:43:30 +01:00
Haoyu Qiu dc7e309f48 Fix heap-use-after-free when resource loaded with load_threaded_request is never fetched
Co-authored-by: Pedro J. Estébanez <pedrojrulez@gmail.com>
2023-10-23 16:21:17 +08:00
A Thousand Ships 893f889d74 [Core] Replace `ERR_FAIL_COND` with `ERR_FAIL_NULL` where applicable 2023-09-11 19:45:49 +02:00
Yuri Sizov d8ff69d53c Extract ScriptInstance to simplify includes
This allows to include script_instance.h directly in the
generated gdvirtual.gen.inc, and remove excessive includes
from the codebase.

This should also allow Resource to use GDVIRTUAL macros,
which wasn't possible previously due to a circular dependency.
2023-09-06 22:54:38 +02:00
Hugo Locurcio 5bf64255b0
Mention expected resource type in ResourceLoader load error
This helps troubleshoot issues due to incorrect import types.
2023-07-21 04:45:34 +02:00
Rémi Verschelde cdd2313ba2
Merge pull request #78977 from RandomShaper/workaround_tls_shutdown
Workaround resource loading crashes due to buggy TLS
2023-07-03 19:43:07 +02:00
Pedro J. Estébanez 41c0785636 Workaround resource loading crashes due to buggy TLS 2023-07-03 13:54:55 +02:00
Pedro J. Estébanez 30925fcfb0 Fix management of safe-for-nodes flag in ResourceLoader and WorkerThreadPool 2023-07-03 12:27:57 +02:00
Pedro J. Estébanez 2b001db76e Let user scripts disable thread safety checks 2023-06-09 18:24:01 +02:00
Pedro J. Estébanez e725b4b02b Allow threads to mark themselves as safe for nodes 2023-05-24 00:22:58 +02:00
Pedro J. Estébanez c450260e3e Fix message queue issues
- Missing flush in resource loading.
- Wrong checks about message queue instance.
2023-05-19 13:11:34 +02:00
Pedro J. Estébanez a6e43f71b6 Avoid multiple possibilites of deadlock in resource loading 2023-05-17 01:52:39 +02:00
Pedro J. Estébanez 7537a0521f Simplify ResourceLoader error callbacks 2023-05-12 17:44:48 +02:00
Pedro J. Estébanez 045401c64e Use WorkerThreadPool for threaded resource loading 2023-05-10 18:53:41 +02:00
Pedro J. Estébanez 5a4613f551 Avoid sync issues in resources with deferred updates 2023-05-10 18:53:41 +02:00
Pedro J. Estébanez 8983b20ccd Avoid interaction issues between resource loading threads 2023-05-10 18:53:41 +02:00
Rémi Verschelde cef3587427
Merge pull request #74120 from RandomShaper/res_loader_teardown
Clear resource load tasks at exit
2023-04-03 15:59:51 +02:00
Rémi Verschelde bdb3738023
Merge pull request #74166 from RandomShaper/fix_res_loader_read_freed
Fix crash in resource load
2023-03-03 11:06:15 +01:00
Rémi Verschelde 4fceabc30c
Merge pull request #73862 from RandomShaper/fix_subthreaded_res_load
Lift restriction that resource load thread requester has to be the initiator
2023-03-02 11:26:54 +01:00
Pedro J. Estébanez 047671df0f Fix crash in resource load 2023-03-01 18:30:16 +01:00
Pedro J. Estébanez b8b6a05c17 Clear resource load tasks at exit 2023-02-28 18:56:21 +01:00
Pedro J. Estébanez b60197d1c6 Fix deadlock in cyclic resource load 2023-02-26 17:19:48 +01:00
Pedro J. Estébanez c51229491d Lift restriction that resource load thread requester has to be the initiator 2023-02-24 13:19:31 +01:00
Pedro J. Estébanez b862fc8c9b Fix cases of resource load tasks not being awaitable 2023-02-20 21:20:05 +01:00
Pedro J. Estébanez 618bb173ba Fix race condition in resource loader when a load task is reused 2023-02-20 21:20:05 +01:00
Raul Santos 9e9eac4676
Use enum instead of int in virtual methods return type 2023-01-31 19:06:49 +01:00
Rémi Verschelde ebd0b40f6e
Merge pull request #71687 from reduz/support-script-class-name-in-efs
Support script global resource name in EditorFileSystem
2023-01-21 16:54:23 +01:00
Juan Linietsky dddd8d43f6 Support script global resource name in EditorFileSystem
* Works for binary and text files.
* Makes EditorQuickOpen work with custom resources again.
* Information is cached and easily accessible.

Properly fixes #66179. Supersedes #66215 and supersedes #62417

**WARNING**: This required breaking backwards binary compatibility (.res and .scn files). Files saved after this PR is merged will no longer open in any earlier versions of Godot.
2023-01-21 14:19:27 +01:00
kobewi c0083e431b Cleanup unused engine code v2 2023-01-19 13:02:18 +01:00
Rémi Verschelde d95794ec8a
One Copyright Update to rule them all
As many open source projects have started doing it, we're removing the
current year from the copyright notice, so that we don't need to bump
it every year.

It seems like only the first year of publication is technically
relevant for copyright notices, and even that seems to be something
that many companies stopped listing altogether (in a version controlled
codebase, the commits are a much better source of date of publication
than a hardcoded copyright statement).

We also now list Godot Engine contributors first as we're collectively
the current maintainers of the project, and we clarify that the
"exclusive" copyright of the co-founders covers the timespan before
opensourcing (their further contributions are included as part of Godot
Engine contributors).

Also fixed "cf." Frenchism - it's meant as "refer to / see".
2023-01-05 13:25:55 +01:00
Adam Scott 4a16b8630e Fix `ResourceLoader::thread_load_tasks` crash 2022-12-07 16:59:10 -05:00
Rémi Verschelde 5947f22be9
Merge pull request #67578 from KoBeWi/GEDITOR
Unify usage of GLOBAL/EDITOR_GET
2022-10-31 13:15:58 +01:00
kobewi d06a8320e5 Simplify GDVIRTUAL_CALL calls 2022-10-19 00:05:48 +02:00
kobewi e48c5daddf Unify usage of GLOBAL/EDITOR_GET 2022-10-18 19:01:48 +02:00
Gilles Roudière 8aa904f56c Exposes ResourceFormatLoader.recognize_path to scripting 2022-10-11 12:35:27 +02:00
bruvzg 0103af1ddd
Fix MSVC warnings, rename shadowed variables, fix uninitialized values, change warnings=all to use /W4. 2022-10-07 11:32:33 +03:00
Rémi Verschelde 9b19a02f31
Merge pull request #63632 from dsnopek/fix-locale-remap-with-binary-resources-4.x
[4.x] Fix locale resource remapping with binary conversion on export
2022-08-08 20:03:23 +02:00
David Snopek 6bbc126c34 Fix locale resource remapping with binary conversion on export 2022-08-08 09:31:35 -05:00
PucklaMotzer09 b32b570d7a Show dependency warning when removing remaps and fallback if translation
remap does not exist
2022-08-03 12:31:29 +02:00
reduz 6236a688b7 Implement Feature Build Profiles
This PR is a continuation of #50381 (which was implemented exactly a year ago!)

* Add a visual interface to select which classes should not be built into Godot (well, they are built if something else uses them, but if not used the optimizer will remove them out).
* Add a detection system to scan the project and figure out the actual classes used.
* Added the ability for SCons to load build profiles.

Obligatory Screen:

A simple test with a couple of nodes in the scene resulted in a 25% reduction for the final binary size

TODO:

* Script languages need to implement used class detection (left for another PR).
* Options to disable servers or server functionalities (like 2D or 3D physics, navigation, etc). Are missing, that should also greatly aid in reducing binary size.
* Options to disable some modules would be desired.
* More options to disable drivers (OpenGL, Vulkan, etc) would be desired.

In general this PR is a starting point for more contributors to improve and enhance this functionality.
2022-07-22 10:53:23 +02:00
reduz e772b65d92 Remake resource thread safety and API
* Ensures thread safety when resources are destroyed.
* Simplified API by always forcing `ResourceCache::get_ref`, which needs less hacks and is fully thread safe.
* Removed RWLock for resources because its not possible to use for the new logic. Should not be a problem.

Supersedes #57533
2022-06-22 13:46:46 +02:00
Aaron Record 900c676b02 Use range iterators for RBSet in most cases 2022-05-19 12:09:16 +02:00
reduz 746dddc067 Replace most uses of Map by HashMap
* Map is unnecessary and inefficient in almost every case.
* Replaced by the new HashMap.
* Renamed Map to RBMap and Set to RBSet for cases that still make sense
  (order matters) but use is discouraged.

There were very few cases where replacing by HashMap was undesired because
keeping the key order was intended.
I tried to keep those (as RBMap) as much as possible, but might have missed
some. Review appreciated!
2022-05-16 10:37:48 +02:00
reduz 0a57f964a3 Implement missing Node & Resource placeholders
Implemented by request of @neikeq to advance in the GDExtension version of Mono.

* If a Resource type is missing upon load, it will be remembered together with its data (Unless manually overriden).
* If a Node type is missing upon load, it will be also be remembered together with its data (unless deleted).

This feature makes working with GDExtension much easier, as it ensures that missing types no longer cause data loss.
2022-05-03 17:08:09 +02:00
Hugo Locurcio 180e5d3028
Remove `RES` and `REF` typedefs in favor of spelled out `Ref<>`
These typedefs don't save much typing compared to the full `Ref<Resource>`
and `Ref<RefCounted>`, yet they sometimes introduce confusion among
new contributors.
2022-05-03 01:43:50 +02:00
bruvzg 9381acb6a4
Make FileAccess and DirAccess classes reference counted. 2022-04-11 13:28:51 +03:00
Rémi Verschelde b8b4580448
Style: Cleanup single-line blocks, semicolons, dead code
Remove currently unused implementation of TextureBasisU, could be re-added
later on if needed and ported.
2022-02-16 14:06:29 +01:00
bruvzg 9456454109
Fix translation with multiple sources for the same language.
Remove unnecessary locale length checks.
Add "C" -> "en" locale remap.
2022-01-24 18:58:16 +02:00
bruvzg de48d5101b
Fix locale always selecting translation instead of "en", when no match found. 2022-01-19 16:46:48 +02:00
bruvzg 40c56ed410
Improve locale detection.
Use separate language, script and country lists.
Add locale selection dialog and property hint.
2022-01-18 14:30:00 +02:00
Rémi Verschelde fe52458154
Update copyright statements to 2022
Happy new year to the wonderful Godot community!
2022-01-03 21:27:34 +01:00
Nathan Franke 49403cbfa0
Replace String comparisons with "", String() to is_empty()
Also:
- Adds two stress tests to test_string.h
- Changes to .empty() on std::strings
2021-12-09 04:48:38 -06:00
Lightning_A c63b18507d Use range iterators for `Map` 2021-09-30 15:09:12 -06:00
Wilson E. Alvarez d11c1afc04
Rename String::is_rel_path to String::is_relative_path 2021-08-29 20:41:29 -04:00
reduz 3682978aee Replace BIND_VMETHOD by new GDVIRTUAL syntax
* New syntax is type safe.
* New syntax allows for type safe virtuals in native extensions.
* New syntax permits extremely fast calling.

Note: Everything was replaced where possible except for `_gui_input` `_input` and `_unhandled_input`.
These will require API rework on a separate PR as they work different than the rest of the functions.

Added a new method flag METHOD_FLAG_OBJECT_CORE, used internally. Allows to not dump the core virtuals like `_notification` to the json API, since each language will implement those as it is best fits.
2021-08-22 08:23:58 -03:00
Rémi Verschelde ac3322b0af
Use const references where possible for List range iterators 2021-07-25 12:22:25 +02:00
Rémi Verschelde 2b1e6e303e
Merge pull request #50786 from reduz/implement-resource-uids
Implement Resource UIDs
2021-07-24 17:18:12 +02:00
reduz 32b43cfeb3 Implement Resource UIDs
* Most resource types now have unique identifiers.
* Applies to text, binary and imported resources.
* File formats reference both by text and UID (when available). UID always has priority.
* Resource UIDs are 64 bits for better compatibility with the engine.
* Can be represented and used textually, example `uuid://dapwmgsmnl28u`.
* A special binary cache file is used and exported, containing the mappings.

Example of how it looks:

```GDScript
[gd_scene load_steps=2 format=3 uid="uid://dw86wq31afig2"]

[ext_resource type="PackedScene" uid="uid://bt36ojelx8q6c" path="res://subscene.scn" id="1_t56hs"]
```

GDScript, shaders and other special resource files can't currently provide UIDs, but this should be doable with special keywords on the files.
This will be reserved for future PRs.
2021-07-24 09:16:52 -03:00
Aaron Franke 4e6efd1b07
Use C++ iterators for Lists in many situations 2021-07-23 17:38:28 -04:00
Lightning_A e28fd07b2b Rename `instance()`->`instantiate()` when it's a verb 2021-06-19 20:49:18 -06:00
kobewi 7ff135b015 Consistently prefix bound virtual methods with _ 2021-06-12 00:55:52 +02:00
Rémi Verschelde 9e328bb5b7
Core: Move DirAccess and FileAccess to `core/io`
File handling APIs are typically considered part of I/O, and we did have most
`FileAccess` implementations in `core/io` already.
2021-06-11 14:52:39 +02:00
Rémi Verschelde c1c76850cb
Style: Cleanup uses of double spaces between words
Or after punctuation. Tried to leave third-party stuff alone, unless it has
been heavily modified for Godot.
2021-06-07 11:03:08 +02:00
Tomasz Chabora b1859510ab Change behavior of String.right 2021-05-20 23:07:57 +02:00
Rémi Verschelde 02e96d9748
Fix error handling for user-defined `ResourceFormatLoader.load`
See #48463.
2021-05-05 15:43:49 +02:00
Anshul7sp1 91181c2086 Fixes small typos and grammar correction 2021-03-12 19:05:16 +05:30
Mateo Kuruk Miccino 472673b38d ResourceLoader: Fix inverse 'if' on the cache mode 2021-02-27 21:53:27 -03:00
Rémi Verschelde 0d1d719178 doc: Sync classref with current source
And fix various bogus bindings following previous PRs.
2021-02-19 14:39:14 +01:00
reduz 64140eaf42 Reorganize Project Settings
-Advanced Settings toggle also hides advanced properties when disabled
-Simplified Advanced Bar (errors were just plain redundant)
-Reorganized rendering quality settings.
-Reorganized miscelaneous settings for clean up.
2021-02-18 11:23:34 -03:00
reduz f8d03b98e7 Improve resource load cache
-Added a new method in Resource: reset_state , used for reloading the same resource from disk
-Added a new cache mode "replace" in ResourceLoader, which reuses existing loaded sub-resources but resets their data from disk (or replaces them if they chaged type)
-Because the correct sub-resource paths are always loaded now, this fixes bugs with subresource folding or subresource ordering when saving.
2021-02-11 15:44:28 -03:00
Pedro J. Estébanez 99fe462452 Modernize Thread
- Based on C++11's `thread` and `thread_local`
- No more need to allocate-deallocate or check for null
- No pointer anymore, just a member variable
- Platform-specific implementations no longer needed (except for the few cases of non-portable functions)
- Simpler for `NO_THREADS`
- Thread ids are now the same across platforms (main is 1; others follow)
2021-01-29 12:02:13 +01:00
Pedro J. Estébanez 8ed259b792 Modernize RWLock
- Based on C++14's `shared_time_mutex`
- No more need to allocate-deallocate or check for null
- No pointer anymore, just a member variable
- Platform-specific implementations no longer needed
- Simpler for `NO_THREADS`
2021-01-19 11:53:10 +01:00
Rémi Verschelde b5334d14f7
Update copyright statements to 2021
Happy new year to the wonderful Godot community!

2020 has been a tough year for most of us personally, but a good year for
Godot development nonetheless with a huge amount of work done towards Godot
4.0 and great improvements backported to the long-lived 3.2 branch.

We've had close to 400 contributors to engine code this year, authoring near
7,000 commit! (And that's only for the `master` branch and for the engine code,
there's a lot more when counting docs, demos and other first-party repos.)

Here's to a great year 2021 for all Godot users 🎆
2021-01-01 20:19:21 +01:00
Zae fa3e0bdd73 fix custom loader/saver broken 2020-11-15 22:14:06 +08:00
reduz 127458ed17 Reorganized core/ directory, it was too fatty already
-Removed FuncRef, since Callable makes it obsolete
-Removed int_types.h as its obsolete in c++11+
-Changed color names code
2020-11-07 20:17:12 -03:00
Rémi Verschelde f57ade8f80
Merge pull request #41460 from Calinou/improve-resource-load-fail-message
Improve the resource loading error message to mention the need to import
2020-09-03 11:31:27 +02:00
Yakov Borevich 604bd75438 [funexpected] clear missed remaps on deinitialization, fixes godotengine/godot#34221 2020-08-31 16:04:46 +03:00
Hugo Locurcio 58ed3037c0
Improve the resource loading error message to mention the need to import
This is a common pitfall when setting up projects in a headless
environment.
2020-08-22 22:19:08 +02:00
Stijn Hinlopen 929b98d24b Remove String::find_last (same as rfind) 2020-07-03 15:26:22 +02:00
Rémi Verschelde 0ee0fa42e6 Style: Enforce braces around if blocks and loops
Using clang-tidy's `readability-braces-around-statements`.
https://clang.llvm.org/extra/clang-tidy/checks/readability-braces-around-statements.html
2020-05-14 21:57:34 +02:00
Rémi Verschelde 07bc4e2f96 Style: Enforce separation line between function definitions
I couldn't find a tool that enforces it, so I went the manual route:
```
find -name "thirdparty" -prune \
  -o -name "*.cpp" -o -name "*.h" -o -name "*.m" -o -name "*.mm" \
  -o -name "*.glsl" > files
perl -0777 -pi -e 's/\n}\n([^#])/\n}\n\n\1/g' $(cat files)
misc/scripts/fix_style.sh -c
```

This adds a newline after all `}` on the first column, unless they
are followed by `#` (typically `#endif`). This leads to having lots
of places with two lines between function/class definitions, but
clang-format then fixes it as we enforce max one line of separation.

This doesn't fix potential occurrences of function definitions which
are indented (e.g. for a helper class defined in a .cpp), but it's
better than nothing. Also can't be made to run easily on CI/hooks so
we'll have to be careful with new code.

Part of #33027.
2020-05-14 16:54:55 +02:00
Rémi Verschelde 0be6d925dc Style: clang-format: Disable KeepEmptyLinesAtTheStartOfBlocks
Which means that reduz' beloved style which we all became used to
will now be changed automatically to remove the first empty line.

This makes us lean closer to 1TBS (the one true brace style) instead
of hybridating it with some Allman-inspired spacing.

There's still the case of braces around single-statement blocks that
needs to be addressed (but clang-format can't help with that, but
clang-tidy may if we agree about it).

Part of #33027.
2020-05-14 16:54:55 +02:00
Rémi Verschelde 1f6f364a56 Port member initialization from constructor to declaration (C++11)
Using `clang-tidy`'s `modernize-use-default-member-init` check and
manual review of the changes, and some extra manual changes that
`clang-tidy` failed to do.

Also went manually through all of `core` to find occurrences that
`clang-tidy` couldn't handle, especially all initializations done
in a constructor without using initializer lists.
2020-05-14 10:01:56 +02:00
Juan Linietsky ae09b55a19 Exposed RenderingDevice to script API
Also added an easier way to load native GLSL shaders.

Extras:

Had to fix no-cache for subresources in resource loader, it was not properly working, making shaders not properly reload.

Note:

The precommit hooks are broken because they don't seem to support enums from one class being used in another.
Feel free to fix this after merging this PR.
2020-04-20 21:21:58 -03:00
lupoDharkael 95a1400a2a Replace NULL with nullptr 2020-04-02 13:38:00 +02:00
Rémi Verschelde cb282c6ef0 Style: Set clang-format Standard to Cpp11
For us, it practically only changes the fact that `A<A<int>>` is now
used instead of the C++03 compatible `A<A<int> >`.

Note: clang-format 10+ changed the `Standard` arguments to fully
specified `c++11`, `c++14`, etc. versions, but we can't use `c++17`
now if we want to preserve compatibility with clang-format 8 and 9.
`Cpp11` is still supported as deprecated alias for `Latest`.
2020-03-17 07:36:24 +01:00