happyfish100 / happyfish100/fastdfs-client-java

fastdfs集群分片上传问题

Open
#87 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
1.6k
Forks
891
PR merge metrics
No merged PRs in 30d

Description

环境使用docker部署fastdfs集群,2个tracker,6个storage(分为3组,每组2个),前后端分离,使用fastdfs-client-java 1.29-SNAPSHOT版本的客户端(自己通过源码编译打成jar包使用),上传300多M的文件,每次上传10M,第一次调用StorageClient类的upload_appender_file(byte[] file_buff, String file_ext_name,NameValuePair[] meta_list),后面都是调用的StorageClient类的append_file(String group_name, String appender_filename, byte[] file_buff)方法进行续传,每次续传前端会传参已经上传的filesize,后端在续传前会调用get_file_info(String group_name, String remote_filename)方法获取在fastdfs中的filesize,多次续传后get_file_info返回的filesize会越来越比前端传的已上传的filesize小,上传300多M的文件实际上在fastdfs中只有180M(甚至有时候第一个分片上传成功后 get_file_info返回的是null),程序没有报任何异常,请问是否是storage同步的问题(续传的storage不是上一次上传的storage,当前storage还未从上次上传的storage同步过来全部数据)应如何实现在集群环境下的分片上传?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported calls to StorageClient.upload_appender_file, append_file, and get_file_info, then trace how they behave across the Docker-deployed tracker and storage groups. Reproduce the multi-part upload and compare the reported uploaded size with the size returned by FastDFS; done means the cause of the inconsistent size or null result is identified and documented with a reliable cluster upload approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.