hpc / hpc/mpifileutils

[Request] dcp/dsync: add support to remote src&dest

Open
#303 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

user-request
Dominant language
C
Stars
200
Forks
85
Avg merge
3d 21h
Merged PRs (30d)
2

Description

dcp/dysnc only works when clients mount both filesystem (src/dst), if people want dcp/dsync over the network, but no mount points. So we might need to have something like "dcp -e ssh user@ip:/src /dest" that rsync is doing, means we need add support to ssh stream into libmfu (MPI aware ssh I/O?).
With a pre-investigation, seems it's not easy to do so. And @adammoody @dinatale2 , can you give some advices here? thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing dcp/dsync and the libmfu interfaces involved in filesystem access, then investigate how an SSH stream could support remote sources and destinations without mounted filesystems. The issue's target is an rsync-like command such as dcp -e ssh user@ip:/src /dest, with MPI-aware SSH I/O, but it notes that this needs pre-investigation.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
cli, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.