Variable Inspector

Variables visible in the inspector

Problem: Having to change the code every time you want to test something.

Solution: developed a way to change variables in the editor. You can mark a variable with [editable] and it would show up in the inspector.

Variable Serialization

Variables serialized to a file

Problem: Changing variables in the editor is nice, but if you can't save them, it's useless.

Solution: I added a way to serialize the properties of a script. You just mark it with [serialize]. This would save the variables to a file and load them.

Structure

Structure of when scripts are called

Demos

All demos shown are made with the scripting language.

Below is a highly-detailed explanation of how the Prefab system works.

Prefab Explained

With many more gameplay programmers now on the team, they need a way to work independently on their own features. Prefabs provide exactly that. A programmer creates a prefab, designers drop it into scenes and tweak its parameters, and then whenever the programmer updates the prefab, those changes automatically propagate to every scene that uses it.

This is a huge time save when we have multiple scenes, with multiple of the same prefab "entities". You only have to change the prefab once, and it updates everywhere.

How it works

We have 2 files.

• Scene
  • This file contains all the entities in the scene with their components and properties.
• Prefab
  • Is the same as a scene file. Entities, components, properties.

A scene might look something like this (example):

- Entity "Player"
    - Transform
        - Position
            - 0,0,0
        - Rotation
            - 0,0,0,0
        - Scale
            - 1,1,1
    - Script
        - Path
            - "Assets/Scripts/PlayerScript.as"
        - Properties
            - Health
                - 100
            - Speed
                - 10
- Entity "John Doe"
...

It contains all the entities with their components and properties.

Now let's say we have a prefab called ``"Ball.prefab"``. This might look something like this (same format as scene):

- Entity "Ball"
    - Transform
        - Position
            - 0,0,0
        - Rotation
            - 0,0,0,0
        - Scale
            - 1,1,1
    - Mesh
        - Path
            - "Assets/Models/Ball.obj"
        - Color
            - 1,0,0

It is the same as a scene file.

Now we want to use this prefab in our scene. The easiest thing to do would just be to put the prefab in the scene file. Like this:

- Entity "Player"
... (same as before, not important)
- Entity "John Doe"
...
- Entity "Ball"
    - Transform
        - Position
            - 0,0,0
        - Rotation
            - 0,0,0,0
        - Scale
            - 1,1,1
    - Mesh
        - Path
            - "Assets/Models/Ball.obj"
        - Color
            - 1,0,0

Now this would work... once. You have an exact copy of the prefab in your scene. But if you change the prefab, it won't change in the scene. This is not what we want. I like to call this a Blueprint (not to be confused with Unreal Engine Blueprints). This is a single time copy.

So what we want is a reference to the prefab. This is what we call a Prefab Instance. This is a reference to the prefab. So if you change the prefab, it changes in the scene. The scene file would look like this:

- Entity "Player"
... (same as before, not important)
- Entity "John Doe"
...
- Entity "Ball"
    - Prefab
        - Path
            - "Assets/Prefabs/Ball.prefab"

Notice how it doesn't have the actual components. It just has a reference to the prefab. So when loading the scene, we load the prefab and add it to the scene. This is a Prefab Instance. No matter how the prefab file looks, it will always be the same in the scene

This is where most people stop. While it's nice, it's still not a real prefab in my opinion.

What I wanted is to be able to change the prefab in the scene. So if you have a prefab instance in the scene, and being able to change properties of the prefab instance.

This way can customize the prefab specifically for a specific scene. For example, we want to change the scale of the prefab in the scene. The scene file would look something like this:

- Entity 1
...
- Entity 2
...
- Entity 3
    - Prefab
        - Path
            - "Assets/Prefabs/Ball.prefab"
        - Overrides
            - Transform
                - Scale
                    - 2,2,2

This way it still uses all the data from the prefab, but it overrides the scale. This way you can have a prefab in multiple scenes, but each scene can have its own customizability.

This is what I call real prefabs.

The Code

While the scene example I showed look simple, “Just get the overrides”. In reality, it’s quite complex. It requires to loop through every entity, every component, and every property of said component recursively. This is not something you want to do by hand. So I needed a way to do this automatically. Here comes my library Reflection Based Prefab system into play.

For every C++ object/component I needed these functions:

• Serialization
• Deserialization
• Compare
• Override

I’ll go through each of these functions and explain what they do.

Serialization / Deserialization

These speak for themselves. Being able to serialize a C++ object to a file and the other way around. For this I used:

• <nlohmann/json>, for json serialization.
• <cbeck88/vis_struct>, for reflection.

See svh/serializer.hpp.

It uses the visit_struct library to loop through the struct. Additionally, I added the ability to implement custom serialize functions.

For example, this class:

struct MyStruct {
	int a = 0;
	float b = 0.0f;
	std::string c = "default";
};

I could either do this:

VISITABLE_STRUCT(MyStruct, a, b, c);

Or implement a custom serialize function to allow for more control over the serialization:

static inline svh::json SerializeImpl(const MyStruct& s) {
	svh::json result = svh::json::object();
	result["a"] = svh::Serializer::Serialize(s.a); // int
	result["b"] = svh::Serializer::Serialize(s.b); // float
	result["c"] = svh::Serializer::Serialize(s.c); // string
	return result;
}

static inline void DeserializeImpl(const svh::json& j, MyStruct& s) {
	if (j.is_object()) {
		svh::Deserializer::FromJson(j["a"], s.a); // int
		svh::Deserializer::FromJson(j["b"], s.b); // float
		svh::Deserializer::FromJson(j["c"], s.c); // string
	}
} 

This way the Serializer class will find these functions at compile time and use that to serialize the struct.

This required me to use/learn template metaprogramming to "direct" the compiler to the right function. For example to detect if a class has a custom serialize function, I used the following code:

/* Serializer function detection */
template <typename T>
using serialize_fn = decltype(SerializeImpl(std::declval<const T&>()));

template <typename T>
constexpr bool has_serialize_v = is_detected<serialize_fn, T>::value;

template <typename T, typename R>
using enable_if_has_serialize = std::enable_if_t<has_serialize_v<T>, R>;

Additionally, I wanted to provide serialize/deserialize functions for all the STL types. This was quite a lot of work and I had to use a lot of template metaprogramming to get it to work. For example, to detect if a type is a vector, I used the following code:

/* Has std::begin/std::end */
template <class, class = void>
struct has_begin_end : std::false_type {};

template <class T>
struct has_begin_end<T, std::void_t<decltype(std::begin(std::declval<&>())), decltype(std::end(std::declval<&>()))>> : std::true_type {};

template <class T>
constexpr bool has_begin_end_v = has_begin_end<>::value;

template <class T, class R = void>
using enable_if_has_begin_end = std::enable_if_t<has_begin_end_v<>, R>;

/* Generic container for vector, array, set, unordered_set, multiset, unordered_multiset, deque, list, foward_list, initializer_lists */
template<class T>
static inline auto SerializeImpl(const T& c) -> svh::enable_if_has_begin_end<, svh::json> {
    svh::json result = svh::json::array();
    for (auto const& item : c) {
        result.push_back(svh::Serializer::ToJson(item));
    }
    return result;
}

See svh/std_types.hpp for the full list of STL types I implemented.

This was already a lot of work, but in hindsight, serialization/deserialization was the easy part...

Compare

In the scene examples I showed we have a list of overrides. This way each scene can have its own custom properties for the prefab.

To allow for this, we need a way to see what the difference is between the prefab and the prefab instance. This is where the compare function comes in.

For example, a transform component:

Transform component compare example

It checks all the properties, and returns a json object with the changes that happened to the right object.

Code-wise this is implemented the same way as serialization, but instead of serializing the object, it compares the properties and returns a json object with the changes:

Again, this uses reflection and template metaprogramming to detect if an object has any changes.

So for example, a transform component:

Transform A;
A.position = { 1.0f, 2.0f, 3.0f };
A.rotation = { 1.0f, 0.0f, 0.0f, 0.0f };
A.scale = { 1.0f, 1.0f, 1.0f };

Transform B;
B.position = { 4.0f, 5.0f, 6.0f };
B.rotation = { 1.0f, 0.0f, 0.0f, 0.0f }; // rotation unchanged
B.scale = { 11.0f, 12.0f, 13.0f };

svh::json changes = svh::Compare::GetChanges(A, B); 

Would return:

{"position":[4.0,5.0,6.0],"scale":[11.0,12.0,13.0]}

This way you can see what changed in the prefab instance. This is used to override the properties in the scene.

The hardest part was containers. If you have something like a vector, there are 3 modification you can do. Add, remove, and change. It would have to account for all of these. Some examples from my unit tests:

No changes:

std::vector<int> left{ 1,2,3 };
std::vector<int> right{ 1,2,3 };

{}

Add:

std::vector<int> left{ 1,2,3 };
std::vector<int> right{ 1,2,3,4 };

{"added":[{"index":[3],"value":4}]}

Remove:

std::vector<int> left{ 1,2,3,4 };
std::vector<int> right{ 1,2,3 };

{"removed":[[3]]}

Change:

std::vector<int> left{ 1,2,3 };
std::vector<int> right{ 4,5,6 };

{"changed":[{"index":[0],"value":4},{"index":[1],"value":5},{"index":[2],"value":6}]}

Now this looks all quite do-able. But what if you have a vector of vectors? Or a vector of structs with pairs that contain maps? You see where I'm going with this?

This can go out of hand really quickly. Luckily I found a library that does sequence comparisons for you: cubicdaiya/dtl.

You would provide 2 vectors, and a way of comparing the elements. This would then return the indices that got removed, added, and changed. For example:

template
struct CustomCompare {
    bool impl(const T& a, const T& b) const {
        auto changes = svh::Compare::GetChanges(a, b);
        return changes.empty();
    }
};

template
static inline svh::json CompareImpl(const std::vector& left, const std::vector& right) {
    // Use the custom comparator to compare elements
    dtl::Diff, CustomCompare> d(left, right, false, CustomCompare{});
    d.compose();
    auto ses = d.getSes();

    // ses now contains the changes like added, removed, and changed
    // ...
}

Override

The last function is the override function. It's basically the opposite of the compare function. It takes the changes from the compare function and applies them to the prefab instance.

Conclusion

The prefab system was a complex system that required a lot of work. It learned me a lot about template metaprogramming, reflection, and serialization.

This system is heavily used in the engine and is a crucial part of the workflow.