godotengine
/
godot
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
godot/servers/rendering/renderer_rd/shader_rd.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

710 lines
23 KiB
C++
Raw Normal View History

A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
/*************************************************************************/
/* shader_rd.cpp */
/*************************************************************************/
/* This file is part of: */
/* GODOT ENGINE */
/* https://godotengine.org */
/*************************************************************************/
Update copyright statements to 2022 Happy new year to the wonderful Godot community!
2022-01-03 20:27:34 +00:00
/* Copyright (c) 2007-2022 Juan Linietsky, Ariel Manzur. */
/* Copyright (c) 2014-2022 Godot Engine contributors (cf. AUTHORS.md). */
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
/* */
/* Permission is hereby granted, free of charge, to any person obtaining */
/* a copy of this software and associated documentation files (the */
/* "Software"), to deal in the Software without restriction, including */
/* without limitation the rights to use, copy, modify, merge, publish, */
/* distribute, sublicense, and/or sell copies of the Software, and to */
/* permit persons to whom the Software is furnished to do so, subject to */
/* the following conditions: */
/* */
/* The above copyright notice and this permission notice shall be */
/* included in all copies or substantial portions of the Software. */
/* */
/* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, */
/* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF */
/* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.*/
/* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY */
/* CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, */
/* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE */
/* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */
/*************************************************************************/
#include "shader_rd.h"
Remove unused classes and stray headers Found by reviewing headers with 1 or less matching includes: ``` find -name thirdparty -prune -o -name "*.h" -exec basename {} \; | sort -u > headers for header in $(cat headers); do echo "$header: "; rg -l "#include \"(.*/)?$header\"" | wc -l; done > list-includes ```
2020-03-24 08:50:51 +00:00
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
#include "core/io/compression.h"
Core: Move DirAccess and FileAccess to `core/io` File handling APIs are typically considered part of I/O, and we did have most `FileAccess` implementations in `core/io` already.
2021-06-11 12:51:48 +00:00
#include "core/io/dir_access.h"
#include "core/io/file_access.h"
RenderingServer reorganization
2020-12-04 18:26:24 +00:00
#include "renderer_compositor_rd.h"
Renaming of servers for coherency. VisualServer -> RenderingServer PhysicsServer -> PhysicsServer3D Physics2DServer -> PhysicsServer2D NavigationServer -> NavigationServer3D Navigation2DServer -> NavigationServer2D Also renamed corresponding files.
2020-03-27 18:21:27 +00:00
#include "servers/rendering/rendering_device.h"
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
#include "thirdparty/misc/smolv.h"
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
void ShaderRD::_add_stage(const char *p_code, StageType p_stage_type) {
Vector<String> lines = String(p_code).split("\n");
String text;
for (int i = 0; i < lines.size(); i++) {
String l = lines[i];
bool push_chunk = false;
StageTemplate::Chunk chunk;
if (l.begins_with("#VERSION_DEFINES")) {
chunk.type = StageTemplate::Chunk::TYPE_VERSION_DEFINES;
push_chunk = true;
} else if (l.begins_with("#GLOBALS")) {
switch (p_stage_type) {
case STAGE_TYPE_VERTEX:
chunk.type = StageTemplate::Chunk::TYPE_VERTEX_GLOBALS;
break;
case STAGE_TYPE_FRAGMENT:
chunk.type = StageTemplate::Chunk::TYPE_FRAGMENT_GLOBALS;
break;
case STAGE_TYPE_COMPUTE:
chunk.type = StageTemplate::Chunk::TYPE_COMPUTE_GLOBALS;
break;
default: {
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
}
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
push_chunk = true;
} else if (l.begins_with("#MATERIAL_UNIFORMS")) {
chunk.type = StageTemplate::Chunk::TYPE_MATERIAL_UNIFORMS;
push_chunk = true;
} else if (l.begins_with("#CODE")) {
chunk.type = StageTemplate::Chunk::TYPE_CODE;
push_chunk = true;
chunk.code = l.replace_first("#CODE", String()).replace(":", "").strip_edges().to_upper();
} else {
text += l + "\n";
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
if (push_chunk) {
Replace String comparisons with "", String() to is_empty() Also: - Adds two stress tests to test_string.h - Changes to .empty() on std::strings
2021-12-09 09:42:46 +00:00
if (!text.is_empty()) {
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
StageTemplate::Chunk text_chunk;
text_chunk.type = StageTemplate::Chunk::TYPE_TEXT;
text_chunk.text = text.utf8();
stage_templates[p_stage_type].chunks.push_back(text_chunk);
text = String();
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
stage_templates[p_stage_type].chunks.push_back(chunk);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
}
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
Replace String comparisons with "", String() to is_empty() Also: - Adds two stress tests to test_string.h - Changes to .empty() on std::strings
2021-12-09 09:42:46 +00:00
if (!text.is_empty()) {
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
StageTemplate::Chunk text_chunk;
text_chunk.type = StageTemplate::Chunk::TYPE_TEXT;
text_chunk.text = text.utf8();
stage_templates[p_stage_type].chunks.push_back(text_chunk);
text = String();
}
}
void ShaderRD::setup(const char *p_vertex_code, const char *p_fragment_code, const char *p_compute_code, const char *p_name) {
name = p_name;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
if (p_compute_code) {
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_add_stage(p_compute_code, STAGE_TYPE_COMPUTE);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
is_compute = true;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
} else {
is_compute = false;
if (p_vertex_code) {
_add_stage(p_vertex_code, STAGE_TYPE_VERTEX);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
}
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
if (p_fragment_code) {
_add_stage(p_fragment_code, STAGE_TYPE_FRAGMENT);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
}
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
StringBuilder tohash;
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
tohash.append("[SpirvCacheKey]");
tohash.append(RenderingDevice::get_singleton()->shader_get_spirv_cache_key());
tohash.append("[BinaryCacheKey]");
tohash.append(RenderingDevice::get_singleton()->shader_get_binary_cache_key());
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
tohash.append("[Vertex]");
tohash.append(p_vertex_code ? p_vertex_code : "");
tohash.append("[Fragment]");
tohash.append(p_fragment_code ? p_fragment_code : "");
tohash.append("[Compute]");
tohash.append(p_compute_code ? p_compute_code : "");
base_sha256 = tohash.as_string().sha256_text();
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
RID ShaderRD::version_create() {
//initialize() was never called
ERR_FAIL_COND_V(variant_defines.size() == 0, RID());
Version version;
version.dirty = true;
version.valid = false;
version.initialize_needed = true;
Replace NULL with nullptr
2020-04-01 23:20:12 +00:00
version.variants = nullptr;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
return version_owner.make_rid(version);
}
void ShaderRD::_clear_version(Version *p_version) {
//clear versions if they exist
if (p_version->variants) {
for (int i = 0; i < variant_defines.size(); i++) {
Fix 'Attempted to remove invalid ID' errors
2021-07-31 06:52:50 +00:00
if (variants_enabled[i]) {
RD::get_singleton()->free(p_version->variants[i]);
}
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
memdelete_arr(p_version->variants);
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
if (p_version->variant_data) {
memdelete_arr(p_version->variant_data);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
}
Replace NULL with nullptr
2020-04-01 23:20:12 +00:00
p_version->variants = nullptr;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
}
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
void ShaderRD::_build_variant_code(StringBuilder &builder, uint32_t p_variant, const Version *p_version, const StageTemplate &p_template) {
for (uint32_t i = 0; i < p_template.chunks.size(); i++) {
const StageTemplate::Chunk &chunk = p_template.chunks[i];
switch (chunk.type) {
case StageTemplate::Chunk::TYPE_VERSION_DEFINES: {
builder.append("\n"); //make sure defines begin at newline
builder.append(general_defines.get_data());
builder.append(variant_defines[p_variant].get_data());
for (int j = 0; j < p_version->custom_defines.size(); j++) {
builder.append(p_version->custom_defines[j].get_data());
}
builder.append("\n"); //make sure defines begin at newline
if (p_version->uniforms.size()) {
builder.append("#define MATERIAL_UNIFORMS_USED\n");
}
Use range iterators for `Map`
2021-08-09 20:13:42 +00:00
for (const KeyValue<StringName, CharString> &E : p_version->code_sections) {
builder.append(String("#define ") + String(E.key) + "_CODE_USED\n");
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
}
Rename OSX to macOS and iPhoneOS to iOS.
2022-07-20 06:28:22 +00:00
#if defined(MACOS_ENABLED) || defined(IOS_ENABLED)
[macOS / iOS] Use storage buffers instead of unsupported images for the volumetric fog on MoltenVK.
2021-11-24 07:14:19 +00:00
builder.append("#define MOLTENVK_USED\n");
#endif
Polish rendering driver refactor further Mainly: - Make `max_descriptors_per_pool` project setting Vulkan-specific. - Use a common, render driver agnostic magic FourCC for shader binary data. - Downgrade spirv_reflect to Vulkan-only dependency. - Add a `RENDER_DRIVER_*` macro to GLSL shader code for per-driver customizations.
2022-10-06 18:45:56 +00:00
builder.append(String("#define RENDER_DRIVER_") + OS::get_singleton()->get_current_rendering_driver_name().to_upper() + "\n");
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
} break;
case StageTemplate::Chunk::TYPE_MATERIAL_UNIFORMS: {
builder.append(p_version->uniforms.get_data()); //uniforms (same for vertex and fragment)
} break;
case StageTemplate::Chunk::TYPE_VERTEX_GLOBALS: {
builder.append(p_version->vertex_globals.get_data()); // vertex globals
} break;
case StageTemplate::Chunk::TYPE_FRAGMENT_GLOBALS: {
builder.append(p_version->fragment_globals.get_data()); // fragment globals
} break;
case StageTemplate::Chunk::TYPE_COMPUTE_GLOBALS: {
builder.append(p_version->compute_globals.get_data()); // compute globals
} break;
case StageTemplate::Chunk::TYPE_CODE: {
if (p_version->code_sections.has(chunk.code)) {
builder.append(p_version->code_sections[chunk.code].get_data());
}
} break;
case StageTemplate::Chunk::TYPE_TEXT: {
builder.append(chunk.text.get_data());
} break;
}
}
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
void ShaderRD::_compile_variant(uint32_t p_variant, Version *p_version) {
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
if (!variants_enabled[p_variant]) {
return; //variant is disabled, return
}
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
Vector<RD::ShaderStageSPIRVData> stages;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
String error;
String current_source;
RD::ShaderStage current_stage = RD::SHADER_STAGE_VERTEX;
bool build_ok = true;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
if (!is_compute) {
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
//vertex stage
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, p_variant, p_version, stage_templates[STAGE_TYPE_VERTEX]);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
current_source = builder.as_string();
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
RD::ShaderStageSPIRVData stage;
stage.spir_v = RD::get_singleton()->shader_compile_spirv_from_source(RD::SHADER_STAGE_VERTEX, current_source, RD::SHADER_LANGUAGE_GLSL, &error);
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (stage.spir_v.size() == 0) {
build_ok = false;
} else {
stage.shader_stage = RD::SHADER_STAGE_VERTEX;
stages.push_back(stage);
}
}
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
if (!is_compute && build_ok) {
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
//fragment stage
current_stage = RD::SHADER_STAGE_FRAGMENT;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, p_variant, p_version, stage_templates[STAGE_TYPE_FRAGMENT]);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
current_source = builder.as_string();
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
RD::ShaderStageSPIRVData stage;
stage.spir_v = RD::get_singleton()->shader_compile_spirv_from_source(RD::SHADER_STAGE_FRAGMENT, current_source, RD::SHADER_LANGUAGE_GLSL, &error);
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (stage.spir_v.size() == 0) {
build_ok = false;
} else {
stage.shader_stage = RD::SHADER_STAGE_FRAGMENT;
stages.push_back(stage);
}
}
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
if (is_compute) {
//compute stage
current_stage = RD::SHADER_STAGE_COMPUTE;
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, p_variant, p_version, stage_templates[STAGE_TYPE_COMPUTE]);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
current_source = builder.as_string();
Split particle shader entry points * Particle shaders now have start() and process() * Particle collision happens between them. * The RESTART property is kept, so porting an old shader is still possible. This fixes the problem of particle collisions not functioning on the first particle frame.
2021-04-17 15:21:03 +00:00
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
RD::ShaderStageSPIRVData stage;
stage.spir_v = RD::get_singleton()->shader_compile_spirv_from_source(RD::SHADER_STAGE_COMPUTE, current_source, RD::SHADER_LANGUAGE_GLSL, &error);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
if (stage.spir_v.size() == 0) {
build_ok = false;
} else {
stage.shader_stage = RD::SHADER_STAGE_COMPUTE;
stages.push_back(stage);
}
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (!build_ok) {
Reimplement Mutex with C++'s <mutex> Main: - It's now implemented thanks to `<mutex>`. No more platform-specific implementations. - `BinaryMutex` (non-recursive) is added, as an alternative for special cases. - Doesn't need allocation/deallocation anymore. It can live in the stack and be part of other classes. - Because of that, it's methods are now `const` and the inner mutex is `mutable` so it can be easily used in `const` contexts. - A no-op implementation is provided if `NO_THREADS` is defined. No more need to add `#ifdef NO_THREADS` just for this. - `MutexLock` now takes a reference. At this point the cases of null `Mutex`es are rare. If you ever need that, just don't use `MutexLock`. - Thread-safe utilities are therefore simpler now. Misc.: - `ScopedMutexLock` is dropped and replaced by `MutexLock`, because they were pretty much the same. - Every case of lock, do-something, unlock is replaced by `MutexLock` (complex cases where it's not straightfoward are kept as as explicit lock and unlock). - `ShaderRD` contained an `std::mutex`, which has been replaced by `Mutex`.
2020-02-26 10:28:13 +00:00
MutexLock lock(variant_set_mutex); //properly print the errors
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
ERR_PRINT("Error compiling " + String(current_stage == RD::SHADER_STAGE_COMPUTE ? "Compute " : (current_stage == RD::SHADER_STAGE_VERTEX ? "Vertex" : "Fragment")) + " shader, variant #" + itos(p_variant) + " (" + variant_defines[p_variant].get_data() + ").");
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
ERR_PRINT(error);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
#ifdef DEBUG_ENABLED
ERR_PRINT("code:\n" + current_source.get_with_code_lines());
#endif
return;
}
Moved the shader source compilation code outside RenderingDevice and Vulkan
2019-07-28 22:58:32 +00:00
Fixes to mobile renderer * Make sure shaders are named, to aid in debug in case of failure * SceneRenderRD was being wrongly initialized (virtual functions being called when derivative class not initialized). * Fixed some bugs resulting on the above being corrected.
2021-08-16 17:51:29 +00:00
Vector<uint8_t> shader_data = RD::get_singleton()->shader_compile_binary_from_spirv(stages, name + ":" + itos(p_variant));
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
ERR_FAIL_COND(shader_data.size() == 0);
RID shader = RD::get_singleton()->shader_create_from_bytecode(shader_data);
Reimplement Mutex with C++'s <mutex> Main: - It's now implemented thanks to `<mutex>`. No more platform-specific implementations. - `BinaryMutex` (non-recursive) is added, as an alternative for special cases. - Doesn't need allocation/deallocation anymore. It can live in the stack and be part of other classes. - Because of that, it's methods are now `const` and the inner mutex is `mutable` so it can be easily used in `const` contexts. - A no-op implementation is provided if `NO_THREADS` is defined. No more need to add `#ifdef NO_THREADS` just for this. - `MutexLock` now takes a reference. At this point the cases of null `Mutex`es are rare. If you ever need that, just don't use `MutexLock`. - Thread-safe utilities are therefore simpler now. Misc.: - `ScopedMutexLock` is dropped and replaced by `MutexLock`, because they were pretty much the same. - Every case of lock, do-something, unlock is replaced by `MutexLock` (complex cases where it's not straightfoward are kept as as explicit lock and unlock). - `ShaderRD` contained an `std::mutex`, which has been replaced by `Mutex`.
2020-02-26 10:28:13 +00:00
{
MutexLock lock(variant_set_mutex);
p_version->variants[p_variant] = shader;
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
p_version->variant_data[p_variant] = shader_data;
Reimplement Mutex with C++'s <mutex> Main: - It's now implemented thanks to `<mutex>`. No more platform-specific implementations. - `BinaryMutex` (non-recursive) is added, as an alternative for special cases. - Doesn't need allocation/deallocation anymore. It can live in the stack and be part of other classes. - Because of that, it's methods are now `const` and the inner mutex is `mutable` so it can be easily used in `const` contexts. - A no-op implementation is provided if `NO_THREADS` is defined. No more need to add `#ifdef NO_THREADS` just for this. - `MutexLock` now takes a reference. At this point the cases of null `Mutex`es are rare. If you ever need that, just don't use `MutexLock`. - Thread-safe utilities are therefore simpler now. Misc.: - `ScopedMutexLock` is dropped and replaced by `MutexLock`, because they were pretty much the same. - Every case of lock, do-something, unlock is replaced by `MutexLock` (complex cases where it's not straightfoward are kept as as explicit lock and unlock). - `ShaderRD` contained an `std::mutex`, which has been replaced by `Mutex`.
2020-02-26 10:28:13 +00:00
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
}
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added ability to visualize native shaders
2021-01-05 23:01:50 +00:00
RS::ShaderNativeSourceCode ShaderRD::version_get_native_source_code(RID p_version) {
Rename RID's `getornull()` to `get_or_null()`
2021-09-29 17:08:41 +00:00
Version *version = version_owner.get_or_null(p_version);
Added ability to visualize native shaders
2021-01-05 23:01:50 +00:00
RS::ShaderNativeSourceCode source_code;
ERR_FAIL_COND_V(!version, source_code);
source_code.versions.resize(variant_defines.size());
for (int i = 0; i < source_code.versions.size(); i++) {
if (!is_compute) {
//vertex stage
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, i, version, stage_templates[STAGE_TYPE_VERTEX]);
Added ability to visualize native shaders
2021-01-05 23:01:50 +00:00
RS::ShaderNativeSourceCode::Version::Stage stage;
stage.name = "vertex";
stage.code = builder.as_string();
source_code.versions.write[i].stages.push_back(stage);
}
if (!is_compute) {
//fragment stage
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, i, version, stage_templates[STAGE_TYPE_FRAGMENT]);
Added ability to visualize native shaders
2021-01-05 23:01:50 +00:00
RS::ShaderNativeSourceCode::Version::Stage stage;
stage.name = "fragment";
stage.code = builder.as_string();
source_code.versions.write[i].stages.push_back(stage);
}
if (is_compute) {
//compute stage
StringBuilder builder;
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
_build_variant_code(builder, i, version, stage_templates[STAGE_TYPE_COMPUTE]);
Added ability to visualize native shaders
2021-01-05 23:01:50 +00:00
RS::ShaderNativeSourceCode::Version::Stage stage;
stage.name = "compute";
stage.code = builder.as_string();
source_code.versions.write[i].stages.push_back(stage);
}
}
return source_code;
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
String ShaderRD::_version_get_sha1(Version *p_version) const {
StringBuilder hash_build;
hash_build.append("[uniforms]");
hash_build.append(p_version->uniforms.get_data());
hash_build.append("[vertex_globals]");
hash_build.append(p_version->vertex_globals.get_data());
hash_build.append("[fragment_globals]");
hash_build.append(p_version->fragment_globals.get_data());
hash_build.append("[compute_globals]");
hash_build.append(p_version->compute_globals.get_data());
Vector<StringName> code_sections;
Use range iterators for `Map`
2021-08-09 20:13:42 +00:00
for (const KeyValue<StringName, CharString> &E : p_version->code_sections) {
code_sections.push_back(E.key);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
}
code_sections.sort_custom<StringName::AlphCompare>();
for (int i = 0; i < code_sections.size(); i++) {
hash_build.append(String("[code:") + String(code_sections[i]) + "]");
hash_build.append(p_version->code_sections[code_sections[i]].get_data());
}
for (int i = 0; i < p_version->custom_defines.size(); i++) {
hash_build.append("[custom_defines:" + itos(i) + "]");
hash_build.append(p_version->custom_defines[i].get_data());
}
return hash_build.as_string().sha1_text();
}
static const char *shader_file_header = "GDSC";
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
static const uint32_t cache_file_version = 2;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
bool ShaderRD::_load_from_cache(Version *p_version) {
String sha1 = _version_get_sha1(p_version);
Rename String `plus_file` to `path_join`
2022-08-30 00:34:01 +00:00
String path = shader_cache_dir.path_join(name).path_join(base_sha256).path_join(sha1) + ".cache";
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Make FileAccess and DirAccess classes reference counted.
2022-03-23 09:08:58 +00:00
Ref<FileAccess> f = FileAccess::open(path, FileAccess::READ);
if (f.is_null()) {
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
return false;
}
char header[5] = { 0, 0, 0, 0, 0 };
f->get_buffer((uint8_t *)header, 4);
ERR_FAIL_COND_V(header != String(shader_file_header), false);
uint32_t file_version = f->get_32();
if (file_version != cache_file_version) {
return false; // wrong version
}
uint32_t variant_count = f->get_32();
ERR_FAIL_COND_V(variant_count != (uint32_t)variant_defines.size(), false); //should not happen but check
for (uint32_t i = 0; i < variant_count; i++) {
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
uint32_t variant_size = f->get_32();
ERR_FAIL_COND_V(variant_size == 0 && variants_enabled[i], false);
if (!variants_enabled[i]) {
continue;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
}
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
Vector<uint8_t> variant_bytes;
variant_bytes.resize(variant_size);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
uint32_t br = f->get_buffer(variant_bytes.ptrw(), variant_size);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
ERR_FAIL_COND_V(br != variant_size, false);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
p_version->variant_data[i] = variant_bytes;
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
for (uint32_t i = 0; i < variant_count; i++) {
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
if (!variants_enabled[i]) {
MutexLock lock(variant_set_mutex);
p_version->variants[i] = RID();
continue;
}
RID shader = RD::get_singleton()->shader_create_from_bytecode(p_version->variant_data[i]);
if (shader.is_null()) {
for (uint32_t j = 0; j < i; j++) {
RD::get_singleton()->free(p_version->variants[i]);
}
ERR_FAIL_COND_V(shader.is_null(), false);
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
{
MutexLock lock(variant_set_mutex);
p_version->variants[i] = shader;
}
}
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
memdelete_arr(p_version->variant_data); //clear stages
p_version->variant_data = nullptr;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
p_version->valid = true;
return true;
}
void ShaderRD::_save_to_cache(Version *p_version) {
String sha1 = _version_get_sha1(p_version);
Rename String `plus_file` to `path_join`
2022-08-30 00:34:01 +00:00
String path = shader_cache_dir.path_join(name).path_join(base_sha256).path_join(sha1) + ".cache";
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Make FileAccess and DirAccess classes reference counted.
2022-03-23 09:08:58 +00:00
Ref<FileAccess> f = FileAccess::open(path, FileAccess::WRITE);
ERR_FAIL_COND(f.is_null());
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
f->store_buffer((const uint8_t *)shader_file_header, 4);
f->store_32(cache_file_version); //file version
uint32_t variant_count = variant_defines.size();
f->store_32(variant_count); //variant count
for (uint32_t i = 0; i < variant_count; i++) {
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
f->store_32(p_version->variant_data[i].size()); //stage count
f->store_buffer(p_version->variant_data[i].ptr(), p_version->variant_data[i].size());
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
}
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
void ShaderRD::_compile_version(Version *p_version) {
_clear_version(p_version);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
p_version->valid = false;
p_version->dirty = false;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
p_version->variants = memnew_arr(RID, variant_defines.size());
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
typedef Vector<uint8_t> ShaderStageData;
p_version->variant_data = memnew_arr(ShaderStageData, variant_defines.size());
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
if (shader_cache_dir_valid) {
if (_load_from_cache(p_version)) {
return;
}
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
#if 1
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
Remove ThreadWorkPool, replace by WorkerThreadPool The former needs to be allocated once per usage. The later is shared for all threads, which is more efficient. It can also be better debugged.
2022-07-23 17:12:41 +00:00
WorkerThreadPool::GroupID group_task = WorkerThreadPool::get_singleton()->add_template_group_task(this, &ShaderRD::_compile_variant, p_version, variant_defines.size(), -1, true, SNAME("ShaderCompilation"));
WorkerThreadPool::get_singleton()->wait_for_group_task_completion(group_task);
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
#else
for (int i = 0; i < variant_defines.size(); i++) {
_compile_variant(i, p_version);
}
#endif
Moved the shader source compilation code outside RenderingDevice and Vulkan
2019-07-28 22:58:32 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
bool all_valid = true;
for (int i = 0; i < variant_defines.size(); i++) {
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
if (!variants_enabled[i]) {
continue; //disabled
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (p_version->variants[i].is_null()) {
all_valid = false;
break;
Moved the shader source compilation code outside RenderingDevice and Vulkan
2019-07-28 22:58:32 +00:00
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
}
Moved the shader source compilation code outside RenderingDevice and Vulkan
2019-07-28 22:58:32 +00:00
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (!all_valid) {
//clear versions if they exist
for (int i = 0; i < variant_defines.size(); i++) {
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
if (!variants_enabled[i]) {
continue; //disabled
}
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
if (!p_version->variants[i].is_null()) {
RD::get_singleton()->free(p_version->variants[i]);
}
}
memdelete_arr(p_version->variants);
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
if (p_version->variant_data) {
memdelete_arr(p_version->variant_data);
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
}
Replace NULL with nullptr
2020-04-01 23:20:12 +00:00
p_version->variants = nullptr;
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
p_version->variant_data = nullptr;
Added a spinlock template as well as a thread work pool class. Also, optimized shader compilation to happen on threads.
2019-07-29 15:59:18 +00:00
return;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
} else if (shader_cache_dir_valid) {
//save shader cache
_save_to_cache(p_version);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
Implement Binary Shader Compilation * Added an extra stage before compiling shader, which is generating a binary blob. * On Vulkan, this allows caching the SPIRV reflection information, which is expensive to parse. * On other (future) RenderingDevices, it allows caching converted binary data, such as DXIL or MSL. This PR makes the shader cache include the reflection information, hence editor startup times are significantly improved. I tested this well and it appears to work, and I added a lot of consistency checks, but because it includes writing and reading binary information, rare bugs may pop up, so be aware. There was not much of a choice for storing the reflection information, given shaders can be a lot, take a lot of space and take time to parse.
2021-07-25 14:22:55 +00:00
memdelete_arr(p_version->variant_data); //clear stages
p_version->variant_data = nullptr;
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
p_version->valid = true;
}
Replace most uses of Map by HashMap * Map is unnecessary and inefficient in almost every case. * Replaced by the new HashMap. * Renamed Map to RBMap and Set to RBSet for cases that still make sense (order matters) but use is discouraged. There were very few cases where replacing by HashMap was undesired because keeping the key order was intended. I tried to keep those (as RBMap) as much as possible, but might have missed some. Review appreciated!
2022-05-13 13:04:37 +00:00
void ShaderRD::version_set_code(RID p_version, const HashMap<String, String> &p_code, const String &p_uniforms, const String &p_vertex_globals, const String &p_fragment_globals, const Vector<String> &p_custom_defines) {
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
ERR_FAIL_COND(is_compute);
Rename RID's `getornull()` to `get_or_null()`
2021-09-29 17:08:41 +00:00
Version *version = version_owner.get_or_null(p_version);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
ERR_FAIL_COND(!version);
version->vertex_globals = p_vertex_globals.utf8();
version->fragment_globals = p_fragment_globals.utf8();
Completed material/2D shader support (missing SCREEN_TEXTURE)
2019-07-21 14:31:30 +00:00
version->uniforms = p_uniforms.utf8();
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
version->code_sections.clear();
Use range iterators for `Map`
2021-08-09 20:13:42 +00:00
for (const KeyValue<String, String> &E : p_code) {
version->code_sections[StringName(E.key.to_upper())] = E.value.utf8();
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
}
Completed material/2D shader support (missing SCREEN_TEXTURE)
2019-07-21 14:31:30 +00:00
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
version->custom_defines.clear();
for (int i = 0; i < p_custom_defines.size(); i++) {
version->custom_defines.push_back(p_custom_defines[i].utf8());
}
version->dirty = true;
if (version->initialize_needed) {
_compile_version(version);
version->initialize_needed = false;
}
}
Replace most uses of Map by HashMap * Map is unnecessary and inefficient in almost every case. * Replaced by the new HashMap. * Renamed Map to RBMap and Set to RBSet for cases that still make sense (order matters) but use is discouraged. There were very few cases where replacing by HashMap was undesired because keeping the key order was intended. I tried to keep those (as RBMap) as much as possible, but might have missed some. Review appreciated!
2022-05-13 13:04:37 +00:00
void ShaderRD::version_set_compute_code(RID p_version, const HashMap<String, String> &p_code, const String &p_uniforms, const String &p_compute_globals, const Vector<String> &p_custom_defines) {
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
ERR_FAIL_COND(!is_compute);
Rename RID's `getornull()` to `get_or_null()`
2021-09-29 17:08:41 +00:00
Version *version = version_owner.get_or_null(p_version);
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
ERR_FAIL_COND(!version);
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
version->compute_globals = p_compute_globals.utf8();
version->uniforms = p_uniforms.utf8();
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
version->code_sections.clear();
Use range iterators for `Map`
2021-08-09 20:13:42 +00:00
for (const KeyValue<String, String> &E : p_code) {
version->code_sections[StringName(E.key.to_upper())] = E.value.utf8();
Refactor GLSL shader compilation -Used a more consistent set of keywords for the shader -Remove all harcoded entry points -Re-wrote the GLSL shader parser, new system is more flexible. Allows any entry point organization. -Entry point for sky shaders is now sky(). -Entry point for particle shaders is now process().
2021-04-13 20:01:43 +00:00
}
Untested support for compute shaders
2019-09-25 19:44:44 +00:00
version->custom_defines.clear();
for (int i = 0; i < p_custom_defines.size(); i++) {
version->custom_defines.push_back(p_custom_defines[i].utf8());
}
version->dirty = true;
if (version->initialize_needed) {
_compile_version(version);
version->initialize_needed = false;
}
}
Custom material support seems complete.
2019-07-27 13:23:24 +00:00
bool ShaderRD::version_is_valid(RID p_version) {
Rename RID's `getornull()` to `get_or_null()`
2021-09-29 17:08:41 +00:00
Version *version = version_owner.get_or_null(p_version);
Custom material support seems complete.
2019-07-27 13:23:24 +00:00
ERR_FAIL_COND_V(!version, false);
if (version->dirty) {
_compile_version(version);
}
return version->valid;
}
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
bool ShaderRD::version_free(RID p_version) {
if (version_owner.owns(p_version)) {
Rename RID's `getornull()` to `get_or_null()`
2021-09-29 17:08:41 +00:00
Version *version = version_owner.get_or_null(p_version);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
_clear_version(version);
version_owner.free(p_version);
} else {
return false;
}
return true;
}
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
void ShaderRD::set_variant_enabled(int p_variant, bool p_enabled) {
ERR_FAIL_COND(version_owner.get_rid_count() > 0); //versions exist
ERR_FAIL_INDEX(p_variant, variants_enabled.size());
variants_enabled.write[p_variant] = p_enabled;
}
bool ShaderRD::is_variant_enabled(int p_variant) const {
ERR_FAIL_INDEX_V(p_variant, variants_enabled.size(), false);
return variants_enabled[p_variant];
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
bool ShaderRD::shader_cache_cleanup_on_start = false;
Rewrote how barriers work for faster rendering -Added more finegrained control in RenderingDevice API -Optimized barriers (use less ones for thee same) -General optimizations -Shadows render all together unbarriered -GI can render together with shadows. -SDFGI can render together with depth-preoass. -General fixes -Added GPU detection
2021-02-02 19:51:36 +00:00
ShaderRD::ShaderRD() {
// Do not feel forced to use this, in most cases it makes little to no difference.
bool use_32_threads = false;
if (RD::get_singleton()->get_device_vendor_name() == "NVIDIA") {
use_32_threads = true;
}
String base_compute_define_text;
if (use_32_threads) {
base_compute_define_text = "\n#define NATIVE_LOCAL_GROUP_SIZE 32\n#define NATIVE_LOCAL_SIZE_2D_X 8\n#define NATIVE_LOCAL_SIZE_2D_Y 4\n";
} else {
base_compute_define_text = "\n#define NATIVE_LOCAL_GROUP_SIZE 64\n#define NATIVE_LOCAL_SIZE_2D_X 8\n#define NATIVE_LOCAL_SIZE_2D_Y 8\n";
}
base_compute_defines = base_compute_define_text.ascii();
}
Modified light rendering to make it more compatible. Modified polygon management to make it more compatible with MoltenVK
2019-07-10 20:44:55 +00:00
void ShaderRD::initialize(const Vector<String> &p_variant_defines, const String &p_general_defines) {
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
ERR_FAIL_COND(variant_defines.size());
ERR_FAIL_COND(p_variant_defines.size() == 0);
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
Modified light rendering to make it more compatible. Modified polygon management to make it more compatible with MoltenVK
2019-07-10 20:44:55 +00:00
general_defines = p_general_defines.utf8();
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
for (int i = 0; i < p_variant_defines.size(); i++) {
variant_defines.push_back(p_variant_defines[i].utf8());
Add support for low-end 3D rendering. -Reduce number of uniform sets from 6 to 4. -Remove features in low end mode, in order to reduce the number of texture units fit to 16.
2020-12-07 21:27:38 +00:00
variants_enabled.push_back(true);
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
Replace String comparisons with "", String() to is_empty() Also: - Adds two stress tests to test_string.h - Changes to .empty() on std::strings
2021-12-09 09:42:46 +00:00
if (!shader_cache_dir.is_empty()) {
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
StringBuilder hash_build;
hash_build.append("[base_hash]");
hash_build.append(base_sha256);
hash_build.append("[general_defines]");
hash_build.append(general_defines.get_data());
for (int i = 0; i < variant_defines.size(); i++) {
hash_build.append("[variant_defines:" + itos(i) + "]");
hash_build.append(variant_defines[i].get_data());
}
base_sha256 = hash_build.as_string().sha256_text();
Make FileAccess and DirAccess classes reference counted.
2022-03-23 09:08:58 +00:00
Ref<DirAccess> d = DirAccess::open(shader_cache_dir);
ERR_FAIL_COND(d.is_null());
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
if (d->change_dir(name) != OK) {
Error err = d->make_dir(name);
ERR_FAIL_COND(err != OK);
d->change_dir(name);
}
//erase other versions?
if (shader_cache_cleanup_on_start) {
}
//
if (d->change_dir(base_sha256) != OK) {
Error err = d->make_dir(base_sha256);
ERR_FAIL_COND(err != OK);
}
shader_cache_dir_valid = true;
print_verbose("Shader '" + name + "' SHA256: " + base_sha256);
}
}
void ShaderRD::set_shader_cache_dir(const String &p_dir) {
shader_cache_dir = p_dir;
}
void ShaderRD::set_shader_cache_save_compressed(bool p_enable) {
shader_cache_save_compressed = p_enable;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
}
Implement shader caching * Shader compilation is now cached. Subsequent loads take less than a millisecond. * Improved game, editor and project manager startup time. * Editor uses .godot/shader_cache to store shaders. * Game uses user://shader_cache * Project manager uses $config_dir/shader_cache * Options to tweak shader caching in project settings. * Editor path configuration moved from EditorSettings to new class, EditorPaths, so it can be available early on (before shaders are compiled). * Reworked ShaderCompilerRD to ensure deterministic shader code creation (else shader may change and cache will be invalidated). * Added shader compression with SMOLV: https://github.com/aras-p/smol-v
2021-05-25 00:25:11 +00:00
void ShaderRD::set_shader_cache_save_compressed_zstd(bool p_enable) {
shader_cache_save_compressed_zstd = p_enable;
}
void ShaderRD::set_shader_cache_save_debug(bool p_enable) {
shader_cache_save_debug = p_enable;
}
String ShaderRD::shader_cache_dir;
bool ShaderRD::shader_cache_save_compressed = true;
bool ShaderRD::shader_cache_save_compressed_zstd = true;
bool ShaderRD::shader_cache_save_debug = true;
A lot of progress with canvas rendering, still far from working.
2019-06-16 02:45:24 +00:00
ShaderRD::~ShaderRD() {
List<RID> remaining;
version_owner.get_owned_list(&remaining);
if (remaining.size()) {
ERR_PRINT(itos(remaining.size()) + " shaders of type " + name + " were never freed");
while (remaining.size()) {
version_free(remaining.front()->get());
remaining.pop_front();
}
}
}