netty / netty/netty

Sent Messages order is lost if using the IO Thread (NIOEventLoop)

Open
#7,783 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
35.1k
Forks
16.3k
Avg merge
1d 5h
Merged PRs (30d)
143

Description

Hi,
Got an ordering problem when using the IOThread directly rather than going to the Thread pool. The reason we use the IOThread is that we have so little to do before forwarding to an ActorSystem that we spare ourselves an extra context switch for every price that enters in the system (that's a lot). But from time to time, we send a reply while being in this IOThread and it conflicts with other Theads.

In the class "AbstractChannelHandlerContext", method "write(msg, flush, promise)", the code behaves differently whether it is in the EventLoop or in an other Thread. For intermediary stages of the pipeline I understand that, but for the first step (TailContext) it should enqueue to preserve ordering.

Expected behavior

I expect that when sending messages, they are sent in the order I am calling the writeAndFlush method.

Actual behavior

When using the IOThread (NIOEventLoop), sending a message will be executed directly. For other Threads, messages are put in a queue to be executed by the IOThread. Therefore, even though the other Threads have called writeAndFlush before, the messages get sent after because the IOThread is busy sending its own message.

Steps to reproduce
  1. Listen to arriving messages
  2. Handle some with a separate Thread, and others directly with the current IOThread.
  3. Put a lock around the code that calls "writeAndFlush" (to make sure in which order the call is made)
  4. Put a "Thread.sleep(100L)" inside so that processing the message takes a long time and enables the arrival of an other message.
  5. You should see that the message processed by the IOThread called "writeAnFlush" after, but got sent before.
Netty version

4.1.22.Final

JVM version (e.g. java -version)

1.8.0_152

OS version (e.g. uname -a)

Linux pc-518 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in AbstractChannelHandlerContext.write(msg, flush, promise), comparing the EventLoop and other-thread paths described in the issue. Reproduce the ordering problem with the listed lock and delay steps on Netty 4.1.22.Final, then verify that writeAndFlush calls are sent in invocation order when IOThread and other threads are involved.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.