ClickHouse / ClickHouse/ClickHouse

Clickhouse get lost when multiple CH instances are running on the same host

Open
#2,434 1 comment 0 reactions 0 assignees View on GitHub
comp-distributed enhancement help wanted minor
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

On a machine with ip aliases, it is fairly easy to setup multiple clickhouse instances that will be binded to a specified ip.

with

```c++
inline bool isLocal(const Cluster::Address & address, const Poco::Net::SocketAddress & resolved_address, UInt16 clickhouse_port)
{
/// If there is replica, for which:
/// - its port is the same that the server is listening;
/// - its host is resolved to set of addresses, one of which is the same as one of addresses of network interfaces of the server machine*;
/// then we must go to this shard without any inter-process communication.
///
/// * - this criteria is somewhat approximate.
///
/// Also, replica is considered non-local, if it has default database set
/// (only reason is to avoid query rewrite).

return address.default_database.empty() && isLocalAddress(resolved_address, clickhouse_port);
}
```

Clickhouse will treat the two instances as in the same process, which is false, inducing errors in sharding.

I've seen the parameter `treat_local_as_remote` for the Cluster Class but that is not updateable and default to `false`

It would be nice to have a settable `treat_local_as_remote` like setting to enforce tcp instead of local process.

system.clusters table with 2 shards per host

```sql
SELECT *
FROM system.clusters

┌─cluster─────────────────────┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬──port─┬─is_local─┬─user────┬─default_database─┐
│ clickhouse2 │ 1 │ 1 │ 1 │ 10.0.0.1 │ 10.0.0.1 │ 9000 │ 1 │ default │ │
│ clickhouse2 │ 2 │ 1 │ 1 │ 10.0.0.2 │ 10.0.0.2 │ 9000 │ 1 │ default │ │
│ clickhouse2 │ 3 │ 1 │ 1 │ 10.0.0.3 │ 10.0.0.3 │ 9000 │ 0 │ default │ │
│ clickhouse2 │ 4 │ 1 │ 1 │ 10.0.0.4 │ 10.0.0.4 │ 9000 │ 0 │ default │ │
│ test_shard_localhost │ 1 │ 1 │ 1 │ localhost │ 127.0.0.1 │ 9000 │ 0 │ default │ │
│ test_shard_localhost_secure │ 1 │ 1 │ 1 │ localhost │ 127.0.0.1 │ 9440 │ 0 │ default │ │
└─────────────────────────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴───────┴──────────┴─────────┴──────────────────┘
```

Changing the instance port is a working turn around which I'd like to avoid.

system.clusters table with 2 shards per host with custom tcp ports
```xml

9001

```

```sql
SELECT *
FROM system.clusters

┌─cluster─────────────────────┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬──port─┬─is_local─┬─user────┬─default_database─┐
│ clickhouse2 │ 1 │ 1 │ 1 │ 10.0.0.1 │ 10.0.0.1 │ 9001 │ 1 │ default │ │
│ clickhouse2 │ 2 │ 1 │ 1 │ 10.0.0.2 │ 10.0.0.2 │ 9002 │ 0 │ default │ │
│ clickhouse2 │ 3 │ 1 │ 1 │ 10.0.0.3 │ 10.0.0.3 │ 9001 │ 0 │ default │ │
│ clickhouse2 │ 4 │ 1 │ 1 │ 10.0.0.4 │ 10.0.0.4 │ 9002 │ 0 │ default │ │
│ test_shard_localhost │ 1 │ 1 │ 1 │ localhost │ 127.0.0.1 │ 9000 │ 0 │ default │ │
│ test_shard_localhost_secure │ 1 │ 1 │ 1 │ localhost │ 127.0.0.1 │ 9440 │ 0 │ default │ │
└─────────────────────────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴───────┴──────────┴─────────┴──────────────────┘
````
ch: 1.1.54378
Enterprise linux

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.