Server shutdown hangs after client streaming method early response

### Problem description

According to [gRPC docs](https://grpc.io/docs/what-is-grpc/core-concepts/#client-streaming-rpc), a server may respond to a client streaming method before reading all of client messages. However, if the server only reads one message and then invokes a response callback, the `server.tryShutdown` callback never gets called, even after all connections are closed.

### Reproduction steps

```proto
syntax = "proto3";

package test;

service Test {
rpc TestClientStream(stream TestRequest) returns(TestResponse){};
}

message TestRequest {}
message TestResponse {}
```

```js
const {Server, ServerCredentials, ChannelCredentials} = require('@grpc/grpc-js');
const {TestClient, TestService} = require('./test_grpc_pb');
const {TestRequest, TestResponse} = require('./test_pb');

setInterval(() => {}, 10_000);

const server = new Server();

server.addService(TestService, {
testClientStream(call, callback) {
call.read();
callback(null, new TestResponse());
},
});

server.bindAsync('localhost:8080', ServerCredentials.createInsecure(), () => {
server.start();

const client = new TestClient(
'localhost:8080',
ChannelCredentials.createInsecure(),
);

const stream = client.testClientStream((err, res) => {
console.log('response', err, res?.toObject());

client.close();

server.tryShutdown(err => {
console.log('shutdown', err);
});
});

stream.write(new TestRequest());
stream.write(new TestRequest());
});
```

### Environment
- OS name, version and architecture: macOS 11.1
- Node version: 14.3.0
- Node installation method: homebrew
- Package name and version: `@grpc/grpc-js@1.2.1`

### Additional context

The logs generated by above code:

```
response null { }
```

If I remove `setInterval`, the NodeJS process would exit after response, but there's still no `shutdown` log.

I tried to add `call.destroy()` in the server method after reading a message, but no luck.