Commit Graph

367 Commits

Author SHA1 Message Date
Pedro J. Estébanez 28d0d56a69 Fix zero-sized WorkerThreadPool not processing group tasks 2023-06-29 19:44:42 +02:00
Ryan Roden-Corrent 731a13a1ba
Fix race in ScriptServer.
Fixes #76581.

TSAN flagged this issue on starting the editor:

1. main calls register_core_types, which calls IP::create(), which calls Thread::start on the resolver thread
2. Thread::callback calls ScriptServer::thread_enter(), as "Scripts may need to attach a stack."
3. ScriptServer::thread_enter() accesses ScriptServer::_languages, which is still being initialized on the main thread by initialize_gdscript_module

This fixes the issue by skipping thread enter/exit notifications if languages have not finished initializing yet.
I'm assuming that notifying un-initialized languages of thread starts/stops would have been pointless anyways. If we need to somehow notify languages of threads before the languages initialize, we'll need a different solution.

```
Godot Engine v4.0.2.stable.custom_build.7a0977ce2 - https://godotengine.org
==================
WARNING: ThreadSanitizer: data race (pid=9426)
  Write of size 4 at 0x55615b187cd0 by main thread:
    #0 ScriptServer::register_language(ScriptLanguage*) /home/rcorre/src/godot/godot/core/object/script_language.cpp:177:28 (godot.linuxbsd.editor.x86_64.llvm.san+0x9e52ab9) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #1 initialize_gdscript_module(ModuleInitializationLevel) /home/rcorre/src/godot/godot/modules/gdscript/register_types.cpp:118:3 (godot.linuxbsd.editor.x86_64.llvm.san+0x36f9c6f) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #2 initialize_modules(ModuleInitializationLevel) /home/rcorre/src/godot/godot/modules/register_module_types.gen.cpp:93:2 (godot.linuxbsd.editor.x86_64.llvm.san+0x2f50499) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #3 Main::setup2(unsigned long) /home/rcorre/src/godot/godot/main/main.cpp:1961:2 (godot.linuxbsd.editor.x86_64.llvm.san+0x2f1d40d) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #4 Main::setup(char const*, int, char**, bool) /home/rcorre/src/godot/godot/main/main.cpp:1879:10 (godot.linuxbsd.editor.x86_64.llvm.san+0x2f16370) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #5 main /home/rcorre/src/godot/godot/platform/linuxbsd/godot_linuxbsd.cpp:61:14 (godot.linuxbsd.editor.x86_64.llvm.san+0x2e67e1f) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)

  Previous read of size 4 at 0x55615b187cd0 by thread T1:
    #0 ScriptServer::thread_enter() /home/rcorre/src/godot/godot/core/object/script_language.cpp:244:22 (godot.linuxbsd.editor.x86_64.llvm.san+0x9e54aed) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #1 Thread::callback(unsigned long, Thread::Settings const&, void (*)(void*), void*) /home/rcorre/src/godot/godot/core/os/thread.cpp:61:2 (godot.linuxbsd.editor.x86_64.llvm.san+0x9464ab0) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #2 void std::__invoke_impl<void, void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>(std::__invoke_other, void (*&&)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long&&, Thread::Settings&&, void (*&&)(void*), void*&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/invoke.h:61:14 (godot.linuxbsd.editor.x86_64.llvm.san+0x9465283) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #3 std::__invoke_result<void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>::type std::__invoke<void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>(void (*&&)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long&&, Thread::Settings&&, void (*&&)(void*), void*&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/invoke.h:96:14 (godot.linuxbsd.editor.x86_64.llvm.san+0x9465283)
    #4 void std:🧵:_Invoker<std::tuple<void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>>::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/std_thread.h:258:13 (godot.linuxbsd.editor.x86_64.llvm.san+0x9465283)
    #5 std:🧵:_Invoker<std::tuple<void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>>::operator()() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/std_thread.h:265:11 (godot.linuxbsd.editor.x86_64.llvm.san+0x9465283)
    #6 std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<void (*)(unsigned long, Thread::Settings const&, void (*)(void*), void*), unsigned long, Thread::Settings, void (*)(void*), void*>>>::_M_run() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/12.2.1/../../../../include/c++/12.2.1/bits/std_thread.h:210:13 (godot.linuxbsd.editor.x86_64.llvm.san+0x9465283)
    #7 execute_native_thread_routine /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:82:18 (libstdc++.so.6+0xd72c2) (BuildId: 6fe66a2d539a78c993bd2d377e00fad389220963)

  Location is global 'ScriptServer::_language_count' of size 4 at 0x55615b187cd0 (godot.linuxbsd.editor.x86_64.llvm.san+0xbf46cd0)

  Thread T1 (tid=9431, running) created by main thread at:
    #0 pthread_create <null> (godot.linuxbsd.editor.x86_64.llvm.san+0x2de5776) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #1 __gthread_create /usr/src/debug/gcc/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:663:35 (libstdc++.so.6+0xd73a9) (BuildId: 6fe66a2d539a78c993bd2d377e00fad389220963)
    #2 std:🧵:_M_start_thread(std::unique_ptr<std:🧵:_State, std::default_delete<std:🧵:_State>>, void (*)()) /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:147:37 (libstdc++.so.6+0xd73a9)
    #3 IP::IP() /home/rcorre/src/godot/godot/core/io/ip.cpp:347:19 (godot.linuxbsd.editor.x86_64.llvm.san+0x962cbcd) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #4 IPUnix::IPUnix() /home/rcorre/src/godot/godot/drivers/unix/ip_unix.cpp:261:9 (godot.linuxbsd.editor.x86_64.llvm.san+0x4aee599) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #5 IPUnix::_create_unix() /home/rcorre/src/godot/godot/drivers/unix/ip_unix.cpp:258:9 (godot.linuxbsd.editor.x86_64.llvm.san+0x4aee599)
    #6 IP::create() /home/rcorre/src/godot/godot/core/io/ip.cpp:339:9 (godot.linuxbsd.editor.x86_64.llvm.san+0x962ca5e) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #7 register_core_types() /home/rcorre/src/godot/godot/core/register_core_types.cpp:279:7 (godot.linuxbsd.editor.x86_64.llvm.san+0x93e2333) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #8 Main::setup(char const*, int, char**, bool) /home/rcorre/src/godot/godot/main/main.cpp:690:2 (godot.linuxbsd.editor.x86_64.llvm.san+0x2f08a49) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)
    #9 main /home/rcorre/src/godot/godot/platform/linuxbsd/godot_linuxbsd.cpp:61:14 (godot.linuxbsd.editor.x86_64.llvm.san+0x2e67e1f) (BuildId: 780a9db7c37d88e78d5ee659c4fa1cd378abd048)

SUMMARY: ThreadSanitizer: data race /home/rcorre/src/godot/godot/core/object/script_language.cpp:177:28 in ScriptServer::register_language(ScriptLanguage*)
```

Co-authored-by: Pedro J. Estébanez <RandomShaper@users.noreply.github.com>
2023-06-21 09:52:10 -04:00
Yuri Sizov 329652b6fe
Merge pull request #78392 from Gallilus/master
Fix `Ref<>.is_valid()` for ScriptInstanceExtension
2023-06-20 21:19:20 +02:00
David Cambré ceed30c442 Fix Ref<>.is_valid() for ScriptInstanceExtension
Only check the reference not the value.
2023-06-20 19:45:47 +02:00
Rémi Verschelde c9c941e339
CI: Update static checks to black 23.3.0
And apply it to the codebase, removing empty lines at the start of blocks.
2023-06-19 23:33:02 +02:00
fuzzy@home 09fa22002a fixes #76473, where filling a large amount of tiles all at once crashes
the engine due to segmentation fault.
2023-06-18 22:32:23 +09:00
ajreckof e9a2060b27 Fix wrong alphabetical order in autocompletion sorting.
Co-Authored-By: A Thousand Ships <96648715+AThousandShips@users.noreply.github.com>
2023-06-16 14:04:07 +02:00
Rémi Verschelde 8c5c29f2a3
Merge pull request #78061 from dsnopek/gdextension-class-not-exposed
Fix wrapping Object's in GDExtension that aren't exposed
2023-06-15 10:50:27 +02:00
ajreckof 81aa5ad999 add backward_undo_ops as property for action 2023-06-13 15:32:05 +02:00
Rémi Verschelde 37d1dfef9d
Merge pull request #78000 from RandomShaper/let_users_mess_up
Let user scripts disable thread safety checks
2023-06-10 12:12:13 +02:00
David Snopek 36f4b99638 Fix wrapping Object's in GDExtension that aren't exposed 2023-06-09 13:41:57 -05:00
Pedro J. Estébanez 2b001db76e Let user scripts disable thread safety checks 2023-06-09 18:24:01 +02:00
Rémi Verschelde 577ab3c565
Merge pull request #75746 from ajreckof/order_autocomplete
Sort code autocompletion with rules
2023-06-08 18:14:31 +02:00
Rémi Verschelde aa5dfff3f0
Merge pull request #77410 from dsnopek/object-pointer-pointer-encoding
Standardize Object ptrcall encoding on `Object **`
2023-06-07 13:31:49 +02:00
Yuri Sizov 97a6b40e4e
Merge pull request #70386 from sfrembling/70141
Add Human Readable Error Message for Object::set_meta
2023-05-29 17:24:25 +02:00
David Snopek 77733faede Attempt to standardize Object ptrcall encoding on `Object **` 2023-05-25 21:46:22 -05:00
ajreckof 006e899bb3 sort code completions with rules
Fixups

Add levenshtein distance for comparisons, remove kind sort order, try to improve as many different use cases as possible

Trying again to improve code completion

Sort code autocompletion options by similarity based on input

To make it really brief, uses a combination `String.similiary`, the category system introduced in a previous PR, and some filtering to yield more predictable results, instead of scattering every completion option at seemingly random.

It also gives much higher priority to strings that contain the base in full, closer to the beginning or are perfect matches.

Also moves CodeCompletionOptionCompare to code_edit.cpp

Co-Authored-By: Micky <66727710+Mickeon@users.noreply.github.com>
Co-Authored-By: Eric M <41730826+EricEzaM@users.noreply.github.com>
2023-05-23 05:12:34 +02:00
Rémi Verschelde 80bf8fd186
Merge pull request #73511 from dsnopek/gdextension-object-name
Add GDExtension function to get Object class name
2023-05-22 14:02:59 +02:00
David Snopek c6b0d4aae3 Add GDExtension function to get Object class name 2023-05-22 06:58:59 -05:00
Rémi Verschelde 9ad9820f5d
Merge pull request #77229 from RandomShaper/fix_mt_issues
Fix message queue issues
2023-05-19 14:30:10 +02:00
Pedro J. Estébanez c450260e3e Fix message queue issues
- Missing flush in resource loading.
- Wrong checks about message queue instance.
2023-05-19 13:11:34 +02:00
RedworkDE 9c9fd9ffbf Improve error message when disconnecting callable from signal it was not connected to 2023-05-18 00:46:27 +02:00
Rémi Verschelde 26f96aec9d
Merge pull request #77143 from RandomShaper/fix_wtp_deadlocks
Avoid multiple possibilites of deadlock in resource loading
2023-05-17 15:59:18 +02:00
Rémi Verschelde 56fc6316b7
Merge pull request #64789 from derammo/derammo_const_ref
Change Ref<T> to allow non const access to ptr
2023-05-17 11:25:13 +02:00
Pedro J. Estébanez a6e43f71b6 Avoid multiple possibilites of deadlock in resource loading 2023-05-17 01:52:39 +02:00
Rémi Verschelde 8cfa19a078
Merge pull request #77000 from reduz/make-more-base-nodes-thread-safe
Make more base nodes thread safe
2023-05-16 00:31:56 +02:00
Shea Frembling 0d6487073f update error from Object::set_meta to be more human readable
update error from Object::set_meta to be more human readable

Add metadata to error message

Add metadata to identifier
2023-05-15 15:14:45 -05:00
Juan Linietsky 0a9f72d5a8 Make more base nodes thread safe
Ongoing work to make more of the base nodes thread safe.
2023-05-15 16:54:10 +02:00
Rémi Verschelde 5c653c27cd
Merge pull request #77093 from RandomShaper/fix_mq_copy
Fix transfer of messages between queues
2023-05-15 13:47:18 +02:00
Rémi Verschelde 70dcfdab1f
Merge pull request #76446 from reduz/add-gdextension-api-compatibility
Add a backwards-compatibility system for GDExtension
2023-05-15 13:43:46 +02:00
Juan Linietsky d8078d3f4c Add a backwards-compatibility system for GDExtension method
This adds a way to ensure that methods that were modified in the Godot API will continue working in older builds of GDExtension even if the new signature is different.

```C++
// New version (changed)
ClassDB::bind_method(D_METHOD("add_sphere","radius","position"),&MyShapes::add_sphere);
// Compatibility version (still available to extensions).
ClassDB::bind_compatibility_method(D_METHOD("add_sphere","radius"),&MyShapes::_compat_add_sphere);
```

**Q**: If I add an extra argument and provide a default value (hence can still be called the same), do I still have to provide the compatibility version?
**A**: Yes, you must still provide a compatibility method. Most language bindings use the raw method pointer to do the call and process the default parameters in the binding language, hence if the actual method signature changes it will no longer work.

**Q**: If I removed a method, can I still bind a compatibility version even though the main method no longer exists?
**A**: Yes, for methods that were removed or renamed, compatibility versions can still be provided.

**Q**: Would it be possible to automate checking that methods were removed by mistake?
**A**: Yes, as part of a future PR, the idea is to add a a command line option to Godot that can be run like : `$ godot --test-api-compatibility older_api_dump.json`, which will also be integrated to the CI runs.
2023-05-15 12:05:40 +02:00
Pedro J. Estébanez 93de52942a Fix transfer of messages between queues 2023-05-15 11:51:58 +02:00
Rémi Verschelde 20ed51a912
Merge pull request #72704 from myaaaaaaaaa/single-main-thread
Partial fixes for `threading/worker_pool/max_threads` = 0
2023-05-12 15:09:50 +02:00
Pedro J. Estébanez 123ba9d464 WorkerThreadPool: Handle exit signal in the tentative scheduling done during waits 2023-05-12 13:31:04 +02:00
myaaaaaaaaa fcd8ce52e9 Partial fixes for threading/worker_pool/max_threads = 0 2023-05-12 07:27:35 -04:00
Rémi Verschelde 8f17f26233
Merge pull request #76978 from raulsntos/property-hint-compat
Reorder `PropertyHint` enum to avoid breaking compat
2023-05-12 10:06:32 +02:00
Raul Santos a34207b812
Reorder PropertyHint enum to avoid breaking compat
Also syncs the Core enum with the C# enum for the source generators.
2023-05-12 01:48:07 +02:00
Pedro J. Estébanez 9077bb9232 Fix multiple issues in WorkerThreadPool
- Fix project settings being ignored.
- Made usages of `native_thread_allocator` thread-safe.
- Remove redundant thread-safety from `low_priority_threads_used`, `exit_threads`.
- Fix deadlock due to unintended extra lock of `task_mutex`.
2023-05-11 16:10:09 +02:00
Pedro J. Estébanez 5a4613f551 Avoid sync issues in resources with deferred updates 2023-05-10 18:53:41 +02:00
Rémi Verschelde 74e5ad5c60
Merge pull request #76165 from and-rad/safe-credentials
Store sensitive export options in dedicated credentials file
2023-05-10 12:47:07 +02:00
Rémi Verschelde 5271186f2f
Merge pull request #75901 from reduz/refactor-node-processing
Refactor Node Processing to allow Scene Multithreading
2023-05-10 12:46:44 +02:00
Andreas Raddau fab160ce70 Store sensitive export options in dedicated credentials file 2023-05-10 11:40:17 +02:00
smix8 a6ac305f96 Rework Navigation Avoidance
Rework Navigation Avoidance.
2023-05-10 05:01:58 +02:00
Juan Linietsky 98c655ec8d Refactor Node Processing
* Node processing works on the concept of process groups.
* A node group can be inherited, run on main thread, or a sub-thread.
* Groups can be ordered.
* Process priority is now present for physics.

This is the first steps towards implementing https://github.com/godotengine/godot-proposals/issues/6424.
No threading or thread guards exist yet in most of the scene code other than Node. That will have to be added later.
2023-05-09 19:17:51 +02:00
Rémi Verschelde ee931e2be5
Merge pull request #76814 from KoBeWi/underdata
Don't refresh inspector when changing internal meta
2023-05-09 17:44:49 +02:00
Rémi Verschelde 3e6a731904
Merge pull request #76418 from reduz/method-bind-validated-call
Add ValidatedCall to MethodBind
2023-05-08 13:52:42 +02:00
Rémi Verschelde 353efd774f
Merge pull request #76754 from RandomShaper/mq_less_lock
Avoid redundant locking in CallQueue
2023-05-08 12:20:53 +02:00
Rémi Verschelde 0daa634f24
Merge pull request #76747 from RandomShaper/fix_mq_redundant
Mend some checks in CallQueue
2023-05-08 12:20:45 +02:00
kobewi 5e3a985c91 Don't refresh inspector when changing internal meta 2023-05-07 22:12:22 +02:00
Pedro J. Estébanez 5a0c186bc0 Avoid redundant locking in CallQueue 2023-05-05 19:39:11 +02:00