Kong / Kong/deck

Feature Request: Support External Lua Files for custom_plugins Definitions

Open
#2,213 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
503
Forks
137
Avg merge
3d 5h
Merged PRs (30d)
20

Description

## Problem Statement

Currently, decK supports custom plugin deployment through the custom_plugins section in the declarative configuration file.

### Example:

```yaml
_format_version: "3.0"
_transform: true

custom_plugins:
- name: my-example-plugin
schema: |
return {
name = "my-example-plugin",
fields = {
{
config = {
type = "record",
fields = {
{ example_field = { type = "string", required = true } },
{ another_example_field = { type = "string", required = true } },
},
},
},
},
}
handler: |
local MyPluginHandler = {
PRIORITY = 1000,
VERSION = "0.0.1",
}

function MyPluginHandler:access(config)
-- plugin logic here
end

return MyPluginHandler
```
Deployment:
```shell
deck gateway apply --include-plugin-definitions kong.yaml
```
or
```
deck gateway sync --include-plugin-definitions kong.yaml
```

While this works well for distributing plugin definitions, it creates challenges during plugin development, testing, and debugging.

## Current Challenges
The plugin source code is typically developed and maintained as standalone Lua files:
```
plugins/
├── my-example-plugin/
│ ├── handler.lua
│ └── schema.lua
```

A common workflow is:

1. Develop and test handler.lua and schema.lua.
2. Copy the contents into the decK YAML file.
3. Run deck gateway apply.
4. Make changes during debugging.
5. Copy the updated code back into the YAML file.
6. Re-apply with decK.

This results in:

- Duplication of source code.
- Additional manual steps during development.
- Risk of YAML and Lua files becoming out of sync.
- Reduced IDE support when the source of truth becomes embedded YAML content.
- Larger and harder-to-maintain declarative configuration files.
## Proposed Enhancement

Allow custom_plugins to reference external files for the schema and handler definitions.

Example:

```yaml
custom_plugins:
- name: my-example-plugin
schema_file: ./plugins/my-example-plugin/schema.lua
handler_file: ./plugins/my-example-plugin/handler.lua
```

During deck gateway apply --include-plugin-definitions, decK would:

1. Read the referenced files.
2. Load their contents internally.
3. Upload the plugin definitions exactly as it does today.

The resulting payload sent to Kong would remain unchanged.

Benefits

- Improved developer experience.
- Faster plugin development and debugging iterations.
- Eliminates duplication between Lua source files and YAML definitions.
- Better integration with IDE features such as syntax highlighting, formatting, linting, and testing.
- Easier version control and code reviews.
- Keeps declarative configuration files concise and readable.
- Aligns with common Infrastructure-as-Code practices where large artifacts are stored in dedicated files and referenced from configuration.

## Use Case

Teams developing custom Kong plugins often maintain plugin source code in a repository and use decK to publish plugin definitions to Konnect or Kong Gateway.

Supporting external file references would enable a workflow where the Lua files remain the single source of truth, allowing developers to:
```shell
# Edit Lua files
vim handler.lua

# Apply changes directly
deck gateway apply --include-plugin-definitions kong.yaml
```

without requiring a manual copy-and-paste step into the YAML file.

## Expected Outcome

decK should support reading plugin schema and handler code from external Lua files and automatically embedding their contents during the apply/sync operation, while maintaining full backward compatibility with the current inline YAML approach.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how `deck gateway apply` and `sync` load `custom_plugins` when `--include-plugin-definitions` is set. Use the existing inline `schema` and `handler` handling as the reference, then verify that `schema_file` and `handler_file` can load `schema.lua` and `handler.lua` while preserving the current inline behavior and resulting Kong payload.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, lua, yaml
Domain
cli, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.