Plugin system

[vyPal]

Time spent on this PR:

Description

A plugin system using dynamic library loading. Library metadata is stored directly in the library file (.so, .dylib, or .dll) so all the server admin has to do is put the os-specific file into the plugin directory. Plugin metadata is pulled from the Cargo.toml file at compile time.

This system is still in an early stage of development, I need to implement a lot of things and split up the code to make it easier to work with in the future, as well as add tests, and some other things.

TODO List

• Dynamic library loading
• On load/unload
• Event priorities
• Async support (if anyone could help with this I would appreciate it)
• Custom commands
• Command to manage plugins
• List plugins in query
• Custom events
• Player events
• World events
• Extism (WASM) support - might not implement
• Lua support - will be a separate pr

Usage

A separate crate named pumpkin-api-macros defines macros to simplify plugin development:

#[plugin_method]
fn on_load(&mut self, server: &dyn PluginContext) -> Result<(), String> {
    server.get_logger().info("Plugin loaded!");
    Ok(())
}

#[plugin_event(blocking = true, priority = Highest)]
async fn on_player_join(
    &mut self,
    server: &dyn PluginContext,
    player: &PlayerEvent,
) -> Result<bool, String> {
    server.get_logger().info(
        format!(
            "Player {} joined the game. Config is {:#?}",
            player.gameprofile.name, self.config
        )
        .as_str(),

    let _ = player
        .send_message(
            TextComponent::text_string(format!(
                "Hello {}, welocme to the server",
                player.gameprofile.name
            ))
            .color_named(NamedColor::Green),
        )
        .await;
    Ok(true)
}

#[plugin_impl]
pub struct MyPlugin {}

There is an example plugin available at plugins/hello-plugin-source/

Testing

Works on my machine ™️
Tested on:

• Arch linux
• Windows 11

Checklist

Things need to be done before this Pull Request can be merged.

• Code is well-formatted and adheres to project style guidelines: cargo fmt
• Code does not produce any clippy warnings: cargo clippy
• All unit tests pass: cargo test
• I added new unit tests, so other people don't accidentally break my code by changing other parts of the codebase. How?

[Commandcracker]

A method for generating plugin IDs is needed. I would like to have something decentralized. Using uuid4 could be a good option ?

To explain the importance of plugin IDs: plugins need a reliable way to declare dependencies on one another, and using names alone is not a robust long-term solution. Additionally, the plugin manager must be able to handle dependencies efficiently. It would also be worth investigating whether we can support scenarios where two plugins have circular dependencies, requiring each other.

We need a way to detect if a plugin is outdated. I propose that plugin_metadata should include the current plugin API version being used by the plugin. Then, the plugin loader can simply check whether the plugin's API version is compatible with the supported API versions (using semantic versioning).

I want to implement side-loading for plugins, specifically one for WASM and another for Lua. For this to work, we would need to ensure that the plugin stack can handle side-loaded plugins in a way that allows a normally loaded plugin to require a side-loaded one. This might involve allowing a plugin to load another "plugin". I don't know how we can make this work, but it would be really nice.

[Commandcracker]

Btw: The system you made looks really good!

Forgot to say this: We also need to implement something like events or hooks.

[vyPal]

plugins need a reliable way to declare dependencies on one another, and using names alone is not a robust long-term solution.

One of my main targets is to make the plugins similair to spigot plugins, they use names as the plugin identifiers and for dependencies, but having some unique id system would probably be better.

It would also be worth investigating whether we can support scenarios where two plugins have circular dependencies, requiring each other.

Depends on what functionality they would require from each other. Another good question is how exactly would the plugins interface with each other. I'd probably recommend a system where plugins can register certain methods or structs with the plugin manager, so that other plugins can access them.

We need a way to detect if a plugin is outdated. I propose that plugin_metadata should include the current plugin API version being used by the plugin. Then, the plugin loader can simply check whether the plugin's API version is compatible with the supported API versions (using semantic versioning).

This is definetly a good idea. It would also be nice if the plugin manager could either periodically or on startup check if any plugins have newer versions available, but this would require implementing a plugin marketplace. The plugin marketplace could also help with providing unique IDs to plugins, but this would no longer be a decentralized system. Maybe if IDs were prefixed with the provisioner of the ID (like 'market:abcd1234' or something similair)?

I want to implement side-loading for plugins, specifically one for WASM and another for Lua. For this to work, we would need to ensure that the plugin stack can handle side-loaded plugins in a way that allows a normally loaded plugin to require a side-loaded one.

If I were to implement the functionality where plugins can register their methods on the manager and expose them to other plugins, this would allow us to use WASM and Lua as full plugins that could interface with other plugins as well

Forgot to say this: We also need to implement something like events or hooks.

I am working on those now, I am primarily thinking up some efficient system for distributing these events between plugins, and a system for plugins to register events they will be listening for, so that all events dont need to be sent to all plugins

[Snowiiii]

First of all, Good Job so far this PR looks pretty promising and you put a good amount of effort into it already. For now one thing i would change is, to remove the plugin_metadata and require it in the plugins Cargo.toml.

[DataM0del]

@vyPal If you want to add support for Lua (I would prefer Luau because it's basically Lua 5.1 but typings, compound assignments, and some other features, while still preserving support for Lua 5.1), you should consider using WebIDL so that people can generate bindings for other languages.

[Commandcracker]

@vyPal If you want to add support for Lua (I would prefer Luau because it's basically Lua 5.1 but typings, compound assignments, and some other features, while still preserving support for Lua 5.1), you should consider using WebIDL so that people can generate bindings for other languages.

Don't know about Web IDL, something like wasmer or extism looks better. But ill do Lua support with mlua it will be a separate plugin that allows sideloading Lua.

[EricApostal]

@vyPal If you want to add support for Lua (I would prefer Luau because it's basically Lua 5.1 but typings, compound assignments, and some other features, while still preserving support for Lua 5.1), you should consider using WebIDL so that people can generate bindings for other languages.

It should be noted that Luau is not backwards compatible with Lua 5.1. There are some small technical differences at runtime (naturally), but it isn't compatible with some Lua 5.1 features by default. Roblox built Luau to be backwards compatible only with the parts of 5.1 they used, not the entire stack.

That being said, Luau should definitely be a consideration, especially over Lua. Furthermore, Luau is much easier to sandbox, which is quite nice for securely running mods. I have used mlua and it is quite nice.

[Commandcracker]

I think we can support all Lua versions with mlua. Your plugin just needs to define what it wants to use.

[lokka30]

I personally prefer sticking with the legacy plugin names rather than using any other identifier such as UUID. I don't think it solves anything and adds extra complexity. However, a way of handling circular dependencies is certainly worth looking into.

[suprohub]

I want it

[DataM0del]

I want it

then make it!

[samolego]

If I add my 2 cents, coming from fabric modding world; I'd vote for simple name ids, perhaps author:plugin_name, as suggested already. User will get confused if their server didn't start because of missing 637rhud7282j.

[lokka30]

If I add my 2 cents, coming from fabric modding world; I'd vote for simple name ids, perhaps author:plugin_name, as suggested already. User will get confused if their server didn't start because of missing 637rhud7282j.

Big +1 for a namespaced key type solution if we have to do anything beyond a simple plugin name.

[0xnim]

I'm not very involved in pumkin, but i'd say there should be a standard way to package them for distribution, possibly a archive/zip file with each platform in there. so/dylib/dll. sounds simple enough

[vyPal]

I'm not very involved in pumkin, but i'd say there should be a standard way to package them for distribution, possibly a archive/zip file with each platform in there. so/dylib/dll. sounds simple enough

It would be useful to have something like that, but packing all three binaries in an archive seems like a waste of space. There are plans for an official pluign website (something like modrinth), which could be integrated directly into the server, so maybe there would be the possibility of just specifying a list of plugins in a configuration file somewhere, and the server would handle downloading and updating the version of the plugin specific to the OS that the server is running on (so you'd specify that you want to use plugin a and the server will recognize that it is on windows, so it will download a.dll from the plugin website and place it in the plugins folder

[kralverde]

My +1 is that you should never ever ever ever ever ever ever ever just plug and play a dll. Have instructions for users to compile their own plugins and have the source code avaliable

[OfficialKris]

Maybe the plugins folder could just contain metadata info about each plugin/settings through a config file. Then when pumpkin runs, it could download and install the requisite binaries in the system with manual input confirmation and hashing validation. That way, the binaries could be secured under administrator privages and be protected against erroneous/malicious writes (which I assume is the security issue here). Otherwise, Lua plugins could be used and be installed alongside in the plugins folder.