futureverse / futureverse/future

Using functions with different ports in future/furrrr

Open
#431 0 comments 0 reactions 0 assignees View on GitHub
feature/resources
Dominant language
R
Stars
1k
Forks
92
PR merge metrics
No merged PRs in 30d

Description

I'm trying to rewrite some code from the snow package so that I can use map functions from furrr. In my code I open a remote connection with different ports. I tried creating similar functions using future_map but did not succeed to create anything for every cluster. Is there something similar to clusterEvalQ in future?

My snow code:

library(snow)

clus <- makeCluster(2)
pids <- unlist(clusterEvalQ(clus, Sys.getpid()))

remote_creator <- function(){


remote_function <<- function(){print(port)}

}

port_printer <- function(x){

remote_function()

}

clusterExport(clus, c('pids', 'remote_creator'))
clusterEvalQ(clus, {port <- (4567L:4568L)[Sys.getpid() == pids] })
clusterEvalQ(clus, remote_creator())
parLapply(clus, 1:2, port_printer)
stopCluster(clus)

Contributor guide

Open the contributing guide

Research direction

Start with the snow example, especially clusterEvalQ, clusterExport, and future_map. Determine whether future provides an equivalent per-worker evaluation mechanism for assigning ports and defining remote_function; done should be a documented, reproducible way to run port_printer across workers, or a clearly scoped feature request if it does not.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.