ash-project / ash-project/spark
Is type: :quoted intended to apply to entity keyword options?
- Dominant language
- Elixir
- Stars
- 204
- Forks
- 50
- Avg merge
- 14h 52m
- Merged PRs (30d)
- 1
Description
## Question
Is `type: :quoted` intended to preserve AST when an entity field is supplied through the entity keyword option list?
Jido Flow V3 is currently in beta. We are considering support for both a short keyword form and a block form. We are flexible about that public API, so I want to understand the intended Spark behavior before we settle the Flow syntax or keep a custom wrapper.
## Versions
- Spark 2.7.2
- Elixir 1.20.3
- Erlang/OTP 29
## Behavior I observed
A field with `type: :quoted` has different behavior based on how it is supplied:
- Spark preserves the AST when the field is an entity argument.
- Spark preserves the AST when the field is set with its field macro inside the entity block.
- Spark does not preserve the AST when the field is in the entity keyword option list. The expression is inserted as code into the generated entity setup call and is compiled in the DSL module.
## Minimal reproduction
```elixir
defmodule QuotedOptionRepro.Item do
defstruct [:name, :code, __spark_metadata__: nil]
end
defmodule QuotedOptionRepro.Extension do
@item %Spark.Dsl.Entity{
name: :item,
target: QuotedOptionRepro.Item,
args: [:name],
schema: [
name: [type: :atom, required: true],
code: [type: :quoted, required: true]
]
}
@items %Spark.Dsl.Section{
name: :items,
entities: [@item]
}
use Spark.Dsl.Extension, sections: [@items]
end
defmodule QuotedOptionRepro.Dsl do
use Spark.Dsl,
default_extensions: [
extensions: [QuotedOptionRepro.Extension]
]
end
```
The block field form preserves the AST:
```elixir
defmodule QuotedOptionRepro.BlockForm do
use QuotedOptionRepro.Dsl
items do
item :block do
code unknown_call(:value)
end
end
end
```
The stored `code` field is:
```elixir
{:unknown_call, [...], [:value]}
```
The keyword option form tries to compile the expression:
```elixir
defmodule QuotedOptionRepro.KeywordForm do
use QuotedOptionRepro.Dsl
items do
item :keyword, code: unknown_call(:value)
end
end
```
Compilation stops before the entity is stored:
```text
** (CompileError) undefined function unknown_call/1
```
## Exact code path in Spark 2.7.2
`Spark.Dsl.Extension.build_entity/8` creates the entity macro and handles the three forms through separate paths.
### Entity argument path
The generated macro zips only the names returned by `Spark.Dsl.Entity.arg_names/1` with their argument values. It then calls `escape_quoted/3` on that list before `shuffle_opts_to_end/5`:
```elixir
{args_without_opts, opts} =
entity_args
|> Enum.zip([unquote_splicing(arg_vars)])
|> Spark.Dsl.Extension.escape_quoted(entity_schema, __CALLER__)
|> Spark.Dsl.Extension.shuffle_opts_to_end(
entity.args,
entity_schema,
entity.entities,
opts
)
```
`escape_quoted/3` calls `Macro.escape/1` when the schema type for that argument is `:quoted`. This is the path that preserves quoted entity arguments.
Source: [`Spark.Dsl.Extension`](https://github.com/ash-project/spark/blob/v2.7.2/lib/spark/dsl/extension.ex#L1442-L1451)
### Block field path
For schema fields that are not entity arguments, `build_entity_options/3` generates a field macro. That macro calls `Spark.Dsl.Extension.EntityOption.value_and_function/6`.
`value_and_function/6` calls `Macro.escape/1` when the supplied type is `:quoted`. The escaped value is then sent to `set_entity_option/4`. This is the path that preserves a field set inside the entity block.
Source: [`EntityOption.value_and_function/6`](https://github.com/ash-project/spark/blob/v2.7.2/lib/spark/dsl/extension/entity_option.ex#L8-L31)
### Entity keyword option path
For this call:
```elixir
item :keyword, code: unknown_call(:value)
```
`:code` is not in `entity.args`. It is present only in the separate `opts` argument received by the generated entity macro.
Because that `opts` argument is not `nil`, `shuffle_opts_to_end/5` returns it unchanged through this clause:
```elixir
def shuffle_opts_to_end(keyword, _entity_args, _, _, opts) do
{replace_not_specified(keyword), opts}
end
```
Source: [`shuffle_opts_to_end/5`](https://github.com/ash-project/spark/blob/v2.7.2/lib/spark/dsl/extension.ex#L1968-L1970)
The generated macro reduces the returned keyword options through `Spark.CodeHelpers.lift_functions/3`. This path does not inspect `entity_schema[key][:type]`. It does not call `escape_quoted/3` or `EntityOption.value_and_function/6`.
Source: [keyword option reduction](https://github.com/ash-project/spark/blob/v2.7.2/lib/spark/dsl/extension.ex#L1458-L1468)
The resulting keyword list is then unquoted into the call to `Spark.Dsl.Extension.Entity.setup/6`:
```elixir
Spark.Dsl.Extension.Entity.setup(
__MODULE__,
unquote(entity.recursive_as),
unquote(nested_key),
unquote(Keyword.delete(opts, :do)),
unquote(arg_values),
unquote(Macro.escape(anno))
)
```
Source: [generated `Entity.setup/6` call](https://github.com/ash-project/spark/blob/v2.7.2/lib/spark/dsl/extension.ex#L1572-L1586)
Since the `:code` value was not escaped, this `unquote` inserts `unknown_call(:value)` as executable code. The compiler tries to resolve the function before `Entity.setup/6` can run.
## History I checked
The `:quoted` type was added in [`37cd97f`](https://github.com/ash-project/spark/commit/37cd97f0923904a4aa07cad57e7189e8d1221de0) on January 13, 2023.
That commit added:
- `escape_quoted/2` for entity arguments.
- `Macro.escape/1` for generated section and entity field macros.
- A test for a quoted positional entity argument.
The test entity also defined a quoted `default_message` field that was not an entity argument. The test did not supply that field through the entity keyword option list.
Commit [`fbdd633`](https://github.com/ash-project/spark/commit/fbdd633be3f93ef595a16f47494932d1a314cede) later moved block-field handling into `Spark.Dsl.Extension.EntityOption.value_and_function/6`. This kept the `Macro.escape/1` behavior for block fields. It did not change the entity keyword option path.
I did not find a current test that supplies a `type: :quoted` field through the entity keyword option list.
## Jido Flow context
Jido Flow is considering support for both forms while the V3 API is in beta:
```elixir
step "example", params: %{value: result("previous")}
```
```elixir
step "example" do
params %{value: result("previous")}
end
```
Jido currently makes the short form work by keeping a list of quoted fields in its wrapper macros and calling `Macro.escape/1` before it calls the generated Spark entity macro.
That is a workable option for us during beta. It does duplicate the quoted-field information from the Spark entity schema, so I want to confirm whether that is the intended integration method before we make it part of the stable Flow DSL.
## Questions
- Is `type: :quoted` intended to preserve AST when the field is supplied through the entity keyword option list?
- Or is `type: :quoted` intentionally limited to entity arguments and field macros inside a block?
- If this difference is intentional, is calling `Macro.escape/1` before the generated entity macro the supported approach for wrapper DSLs?
- Is there another supported Spark API for a short-form entity constructor with quoted keyword fields?
Contributor guide
Research direction
Start with lib/spark/dsl/extension.ex, especially build_entity/8, escape_quoted/3, shuffle_opts_to_end/5, and the generated Entity.setup/6 call; compare that path with lib/spark/dsl/extension/entity_option.ex. Run the minimal QuotedOptionRepro keyword and block forms, then determine from maintainer guidance or a regression test whether quoted keyword fields should preserve their AST and what supported wrapper behavior is expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100