alibaba / alibaba/DataX

你们的datax是否也有这个问题 【HDFS报错】 DataXceiver error processing WRITE_BLOCK operation

Open
#1,448 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

2022-07-16 17:13:43,182 ERROR [DataXceiver for client DFSClient_attempt_1657804987000_15524_m_000040_0_-1450386456_1 at /172.16.9.143:36326 [Receiving block BP-12047675-172.16.9.141-1608793845868:blk_1152735511_79014410]] org.apache.hadoop.hdfs.server.datanode.DataNode: hadoop-dn004:50010:DataXceiver error processing WRITE_BLOCK operation src: /172.16.9.143:36326 dst: /172.16.9.142:50010
java.io.IOException: Premature EOF from inputStream
at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:211)
at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211)
at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134)
at org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109)
at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528)
at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290)
at java.lang.Thread.run(Thread.java:748)
2022-07-16 17:14:14,950 INFO [DataXceiver for client DFSClient_attempt

Contributor guide

No contributing guide indexed for this repository

Research direction

No DataX file, test, entry point, reproduction steps, or expected behavior are provided. Begin by tracing the HDFS write path from the reported WRITE_BLOCK failure; completion would require a reproducible cause, a targeted fix, and a regression test or documented diagnosis.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.