ClickHouse / ClickHouse/clickhouse-go
Allow use test utils outside of this repository
- Dominant language
- Go
- Stars
- 3.3k
- Forks
- 684
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 14
Description
## Observed
I'm trying to set up an in-memory or localhost ClickHouse instance to autostart directly from Golang test code, so as not to require manual installation and running dependencies. As far as I understand [from the examples in the repository](https://github.com/ClickHouse/clickhouse-go/blob/28fd6a4954a5dbf09b7dcc0fcce597beb2dd0b58/examples/std/connect.go), it's enough to create a test environment and connect to it just as if it were running in production.
But in the process I discovered some unpleasant features: dependencies in the form of docker and broken operation in the presence of vendor. The first I'm still able to ignore for the time being. But I can't give up on the latter because of the privacy of the repository and some of its dependencies, which are easier to vend together with the code than to try to set up special individual accesses in environments I don't control. Also, some dependencies can be moved or removed without my knowledge, which I have unfortunately already encountered and avoided by vendoring.
## Expected behaviour
I expect the SDK to have tools to create and configure short-lived ClickHouse instances locally or directly in memory right from go-tests, without having to go online for dependencies. For example, by vendoring the SDK in a project.
## Code example
```go
package main_test
import (
"database/sql"
"fmt"
"log"
"os"
"testing"
"github.com/ClickHouse/clickhouse-go/v2"
clickhousetest "github.com/ClickHouse/clickhouse-go/v2/tests"
)
var testDB *sql.DB
func TestMain(m *testing.M) {
env, err := clickhousetest.CreateClickHouseTestEnvironment("sample.test")
if err != nil {
log.Fatalln("cannot create environment for testing clickhouse server:", err)
}
testDB = clickhouse.OpenDB(&clickhouse.Options{
Addr: []string{fmt.Sprintf("%s:%d", env.Host, env.Port)},
Auth: clickhouse.Auth{
Database: env.Database,
Username: env.Username,
Password: env.Password,
},
})
if err = testDB.Ping(); err != nil {
log.Fatalln("failed ping of testing clickhouse database:", err)
}
os.Exit(m.Run())
}
// Use testDB in other tests...
```
## Error log
```log
2024/05/24 01:08:04 github.com/testcontainers/testcontainers-go - Connected to docker:
Server Version: 26.1.3
API Version: 1.45
Operating System: Arch Linux
Total Memory: 15844 MB
Resolved Docker Host: unix:///var/run/docker.sock
Resolved Docker Socket Path: /var/run/docker.sock
Test SessionID: 43088ad406089a9a6b8c842f193ac703c8489b4bbe18d000dee8176e5b327762
Test ProcessID: 148a6eca-5316-4ad6-b2b7-9eae95102d2e
Using Docker for IT tests
2024/05/24 01:08:04 🐳 Creating container for image testcontainers/ryuk:0.6.0
2024/05/24 01:08:04 ✅ Container created: 8ee4b517b100
2024/05/24 01:08:04 🐳 Starting container: 8ee4b517b100
2024/05/24 01:08:04 ✅ Container started: 8ee4b517b100
2024/05/24 01:08:04 🚧 Waiting for container id 8ee4b517b100 image: testcontainers/ryuk:0.6.0. Waiting for: &{Port:8080/tcp timeout: PollInterval:100ms}
2024/05/24 01:08:04 🔔 Container is ready: 8ee4b517b100
2024/05/24 01:08:04 🐳 Creating container for image clickhouse/clickhouse-server:latest
2024/05/24 01:08:04 cannot create environment for testing clickhouse server: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /home/toby3d/go/src/source.toby3d.me/toby3d/playground/vendor/github.com/ClickHouse/clickhouse-go/v2/tests/resources/custom.xml: failed to create container
exit status 1
FAIL source.toby3d.me/toby3d/playground 0.431s
```
## Details
### Environment
* [x] `clickhouse-go` version: `v2.24.0`
* [x] Interface: `database/sql` compatible driver
* [x] Go version: `1.22.3`
* [x] Operating system: Arch linux/amd64
* [x] ClickHouse version: `local version 24.4.1.2088 (official build)`
* [x] Is it a ClickHouse Cloud? `No`
* [ ] ClickHouse Server non-default settings, if any: `none`
* [ ] `CREATE TABLE` statements for tables involved: `none`
* [ ] Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
Contributor guide
Research direction
Start by tracing CreateClickHouseTestEnvironment from the tests package and inspect tests/resources/custom.xml, then compare its assumptions with the external Go example in examples/std/connect.go. Reproduce the failure with the shown vendored project and verify that test utilities can be consumed outside this repository without the resource path breaking or requiring unavailable dependency access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- databases, testing, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100