DNS rules are not in effect during rule-set initialization, and the ECH config fetch can deadlock startup permanently

Operating system

macOS

System version

Darwin 25.5.0, arm64 (both bugs are platform independent)

Installation type

Original sing-box Command Line

Version

built from source at testing @ 1f886d7c

sing-box version unknown

Environment: go1.26.0 darwin/arm64
CGO: enabled

Description

Two distinct bugs. They are independent, but they compound into the "stuck at startup" symptom described in #4489, so I am reporting them together — happy to split them into separate issues if preferred.

---

Bug 1 — dns.rules are not in effect while remote rule-sets are initialized

box.go#L563 starts the route router before the DNS router:

err = adapter.Start(s.ctx, s.logger, adapter.StartStateStart, s.router, s.dnsRouter)

• route.Router.Start(StartStateStart) is what downloads remote rule-sets — route/router.go#L136-L157
• dns.Router.Start(StartStateStart) is what assigns r.rules = newRules — dns/router.go#L150

So for the whole duration of the initial rule-set fetch, dns.Router.rules is still nil. walkDNSRules matches nothing and every DNS query issued on that path silently falls through to dns.final. In the debug log this is visible as a dns: lookup domain ... / dns: exchange ... IN HTTPS line with no dns: match[...] line following it.

This is not ECH specific — it also applies to the rule-set's own domain lookup, and it is why a dns.rules entry "only works after the first successful start": on later starts the rule-set is restored from cache.db, nothing is downloaded during Start(), and the query then happens at runtime when the rules are live.

Bug 2 — ECHClientConfig.access is held across the ECH DNS query, which deadlocks permanently on re-entry

common/tls/ech.go#L130-L150:

func (s *ECHClientConfig) fetchAndHandshake(ctx context.Context, conn net.Conn) (aTLS.Conn, error) {
	s.access.Lock()
	defer s.access.Unlock()          // held for the entire DNS query below
	if len(s.ECHConfigList()) == 0 || s.lastTTL == 0 || time.Since(s.lastUpdate) > s.lastTTL {
		…
		response, err := s.dnsRouter.Exchange(ctx, message, adapter.DNSQueryOptions{})

One ECHClientConfig instance is shared by every dial of that outbound. If the ECH query's own resolution path dials the same outbound again — e.g. the DNS server it lands on has detour pointing at that outbound — the nested dial reaches fetchAndHandshake again and blocks on the same mutex, while the holder is waiting for that very query. sync.Mutex is not context aware, so no timeout can break it, and nothing on the rule-set start path has a deadline either (ruleSetStartGroup.Run(r.ctx) uses the root context, and RemoteRuleSet.fetch sets none). The process never finishes starting.

Bug 1 makes bug 2 much easier to hit: during startup a dns.rules entry cannot be used to steer the ECH query away from a proxied dns.final, because the rules are not loaded yet. That combination is, I believe, the mechanism behind "会卡在启动状态" in #4489.

Reproduction

Shared setup — a local file server standing in for the rule-set host, and something that accepts TCP so dials reach the TLS stage:

mkdir -p /tmp/rs && echo '{"version":3,"rules":[]}' > /tmp/rs/test.json
cd /tmp/rs && python3 -m http.server 18080 --bind 127.0.0.1

---

Bug 1 — dns.rules skipped during rule-set init

bug1.json. The rule sends rules.example to a hosts server that can answer it; final is a dead resolver. If the rule applied, startup would succeed.

{
  "log": { "level": "debug" },
  "dns": {
    "servers": [
      { "tag": "hosts", "type": "hosts", "predefined": { "rules.example": ["127.0.0.1"] } },
      { "tag": "dead", "type": "udp", "server": "127.0.0.1", "server_port": 5353 }
    ],
    "rules": [{ "domain": "rules.example", "server": "hosts" }],
    "final": "dead"
  },
  "inbounds": [{ "type": "mixed", "tag": "in", "listen": "127.0.0.1", "listen_port": 12080 }],
  "outbounds": [{ "type": "direct", "tag": "direct" }],
  "route": {
    "rule_set": [
      { "type": "remote", "tag": "test", "format": "source", "url": "http://rules.example:18080/test.json" }
    ],
    "final": "direct"
  }
}

sing-box run -c bug1.json — expected: rule matches, hosts answers, startup succeeds. Actual: the rule is skipped, the query goes to dead, startup fails after the DNS timeout.

Control, same rule, same config, only adding "initial_path": "/tmp/rs/test.json" to the rule-set so that the fetch happens as a background update after Start() instead of during it — the rule then matches (log below).

Bug 2 — permanent startup deadlock

bug2.json. Same shared setup. proxied is a DoT server whose detour is the ECH outbound; the ECH query lands on it because of bug 1.

{
  "log": { "level": "debug" },
  "dns": {
    "servers": [
      { "tag": "hosts", "type": "hosts",
        "predefined": { "rules.example": ["127.0.0.1"], "proxy.example": ["127.0.0.1"] } },
      { "tag": "proxied", "type": "tls", "server": "127.0.0.1", "server_port": 18080, "detour": "proxy",
        "tls": { "enabled": true, "server_name": "dns.example", "insecure": true } }
    ],
    "rules": [
      { "domain": "cloudflare-ech.example", "query_type": ["HTTPS"], "server": "hosts" }
    ],
    "final": "proxied"
  },
  "http_clients": [
    { "tag": "http2", "version": 2, "detour": "proxy", "domain_resolver": "hosts" }
  ],
  "inbounds": [{ "type": "mixed", "tag": "in", "listen": "127.0.0.1", "listen_port": 12080 }],
  "outbounds": [
    { "type": "direct", "tag": "direct" },
    { "type": "trojan", "tag": "proxy", "server": "proxy.example", "server_port": 18080, "password": "pw",
      "tls": { "enabled": true, "server_name": "proxy.example", "insecure": true,
               "ech": { "enabled": true, "query_server_name": "cloudflare-ech.example" } } }
  ],
  "route": {
    "rule_set": [
      { "type": "remote", "tag": "test", "format": "source", "url": "http://rules.example:18080/test.json" }
    ],
    "final": "direct",
    "default_domain_resolver": { "server": "hosts" }
  }
}

sing-box run -c bug2.json — never finishes starting; still blocked after 45s, and in a Go harness still blocked after 120s.

Control: change only dns.final from proxied to hosts — startup now fails immediately and cleanly instead of hanging:

DEBUG[0000] dns: exchange cloudflare-ech.example. IN HTTPS
FATAL[0000] start service: initialize rule-set[0]: initial rule-set: test: Get "http://rules.example:18080/test.json": fetch ECH config list: NXDOMAIN

Note the dns: match[...] line is missing in the control too — final is the only lever that works during startup, which is bug 1.

Logs

### Bug 1, sing-box run -c bug1.json — rule skipped, no `dns: match` line

DEBUG[0000] router: updating rule-set test from URL: http://rules.example:18080/test.json
INFO[0000] outbound/direct[direct]: outbound connection to rules.example:18080
DEBUG[0000] dns: lookup domain rules.example
DEBUG[0010] dns: lookup failed for rules.example: (exchange6: context deadline exceeded | exchange4: context deadline exceeded)
WARN[0010] router: initialize rule-set take too much time to finish!
FATAL[0010] start service: initialize rule-set[0]: initial rule-set: test: Get "http://rules.example:18080/test.json": lookup rules.example: (exchange6: context deadline exceeded | exchange4: context deadline exceeded)

### Same rule, same binary, fetch happening after Start() (initial_path added) — rule matches

INFO[0000] inbound/mixed[in]: tcp server started at 127.0.0.1:12080
DEBUG[0000] router: updating rule-set test from URL: http://rules.example:18080/test.json
DEBUG[0000] dns: lookup domain rules.example
DEBUG[0000] dns: match[0] domain=rules.example => route(hosts)
DEBUG[0000] dns: lookup succeed for rules.example: 127.0.0.1
INFO[0000] router: updated rule-set test

### Bug 2, sing-box run -c bug2.json — ECH query skips the rule, lands on the proxied final, never returns

DEBUG[0000] router: updating rule-set test from URL: http://rules.example:18080/test.json
DEBUG[0000] dns: lookup succeed for rules.example: 127.0.0.1
DEBUG[0000] dns: lookup succeed for proxy.example: 127.0.0.1
DEBUG[0000] dns: exchange cloudflare-ech.example. IN HTTPS
DEBUG[0000] dns: lookup domain proxy.example
DEBUG[0000] dns: lookup succeed for proxy.example: 127.0.0.1
(no further output; process never reaches "sing-box started")

Goroutine dump of the deadlock, all four on the same ECHClientConfig instance 0x54cf5d6c4e0 (trimmed):

goroutine 10 [chan receive]:                       # waiting for the rule-set fetch
	route/rule/rule_set_remote.go:248   (*RemoteRuleSet).fetch
	route/rule/rule_set_remote.go:133   (*RemoteRuleSet).StartContext
	route/router.go:144                 (*Router).Start.func1

goroutine 11 [chan receive]:                       # HOLDS ECHClientConfig.access
	common/tls/ech.go:150               (*ECHClientConfig).fetchAndHandshake -> dnsRouter.Exchange
	dns/router.go:1209                  (*Router).Exchange
	dns/router.go:777                   (*Router).exchangeWithRules
	dns/router.go:920                   (*Router).finishPendingExchange
	dns/client.go:387                   (*Client).Exchange
	dns/transport/tls.go:112            (*TLSTransport).Exchange
	dns/transport/multiplexer.go:144    (*queryMultiplexer).Exchange

goroutine 50 [sync.Mutex.Lock]:                    # BLOCKED on the same mutex
	common/tls/ech.go:131               (*ECHClientConfig).fetchAndHandshake
	common/tls/ech.go:119               (*ECHClientConfig).ClientHandshake
	common/tls/client.go:159            (*defaultDialer).dialContext
	protocol/trojan/outbound.go:164     (*trojanDialer).DialContext
	common/dialer/detour.go:85          (*DetourDialer).DialContext
	dns/transport/tls.go:70             NewTLSRaw.func1        # the DoT server dialing its detour

goroutine 51 [sync.Mutex.Lock]:                    # same cycle, IPv6 branch
	common/tls/ech.go:131               (*ECHClientConfig).fetchAndHandshake
	...

For bug 1 I also ran an A/B inside a single instance, with two mock UDP resolvers and one rule {"domain": "rules.example", "server": "a"}, "final": "b":

when the lookup for rules.example happens	resolver that received it
rule-set download inside Start()	b — dns.final, rule skipped
ordinary connection after Start() returned	a — rule matched

Support us

• I have sponsored

Integrity requirements

• I have read the documentation and understand the meaning of all the configuration items I have written, rather than pasting a bunch of seemingly useful options or default values.
• I provide the server and client configuration files and process that can reproduce the problem locally, rather than a desensitized complex client configuration file.
• I provide the minimal configuration that can be used to reproduce the error I reported, rather than relying on remote servers, TUN, graphical client programs or other closed source software.
• I provide complete configuration files and logs, rather than only providing the parts that I think are useful out of confidence in my own intelligence.

Read box.go, route/router.go, dns/router.go, and common/tls/ech.go to trace startup ordering and ECH handshakes. Run the provided bug1.json and bug2.json reproductions first, then use their logs and goroutine dump as the baseline. Done means startup DNS rules apply during rule-set initialization and the ECH configuration path no longer remains permanently blocked.