cockroachdb / cockroachdb/datadriven

support multi-command statements

Open
#46 2 comments 0 reactions 1 assignee Claimed by @pav-kv View on GitHub
enhancement good first issue
Dominant language
Go
Stars
64
Forks
20
PR merge metrics
No merged PRs in 30d

Description

The amount of boilerplate to run a sequence of commands in a `datadriven` test is excessive. For example, [here](https://github.com/etcd-io/raft/blob/026484c68b7bc59cd208e216d35e1ff747d24d9c/testdata/replicate_pause.txt#L10-L55) is a typical script. This is visually unsatisfying (the amount of signal per screen is low, and one has to scroll to read it), and slow to type.

It would be good to be able to batch multiple commands in a single statement, i.e. instead of:

```
command1
----
output1

command2
----
output2

command3
----
output3
```

have something like:

```
command1
command2
command3
----
output1
output2
output3
```

Variations are possible. For example, multiple `ok` outputs could be squashed into a single `ok`:

```
command1
command2
command3
----
ok
```

This format is in conflict with the [current syntax](https://github.com/cockroachdb/datadriven/blob/e384cf455877a2a2d83fd835940d4061a8d203b4/datadriven.go#L72-L75) which treats a multi-line statement as a single command with additional input.

We may need some way to mark the multi-command statements. For example:

```
+ command1 # assumming there are no existing tests with "+" command
+ command2 # we could restrict command names to be "identifiers" starting with a letter
+ command3
----
output1
output2
output3
```

or:

```
---- # or other special preamble
command1
command2
command3
----
output1
output2
output3
```

The first option works better with multiple multi-line commands:

```
+ command1
# must not have a line starting with "+"
+ command2

+ command3

----
output1
output2
output3
```

Although the given syntax could be restricted to single-line commands, for simplicity. This would give most of the value anyway.

For extra niceness, it would be good to support inline comments next to the commands. With all the modifications, the scripts can look more satisfactory:

Details

```
# Start with 3 nodes, with a limited in-flight capacity.
add-nodes 3 voters=(1,2,3) index=10 inflight=3
----
ok

+ campaign 1 # elect node 1 to be the leader
+ stabilize
----
ok

+ propose 1 prop_1_12 # propose a few entries
+ propose 1 prop_1_13
+ propose 1 prop_1_14
+ process-ready 1 # store entries and send proposals to followers
----
ok

# Expect that in-flight tracking to nodes 2 and 3 is saturated.
+ log-level debug
+ status 1
+ log-level none
----
1: StateReplicate match=14 next=15
2: StateReplicate match=11 next=15 paused inflight=3[full]
3: StateReplicate match=11 next=15 paused inflight=3[full]

# etc
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.