77 questions
0 votes
0 answers
58 views
R doParallel: how to suppress the "loaded package and set parent environment" messages?
I'm calling a foreach loop, with doParallel backend, from within a script. For each core I receive the message 'package ... loaded and set parent environment', which fills the output console. I see ...
1 vote
2 answers
273 views
How to display console progress bar output in shiny app?
I am currently making a Shiny app which computes a correlation matrix via the function denoted f()in the example below from an input dataset. Because it takes quite some time, I would like to display ...
0 votes
1 answer
361 views
R fixest: Main variables of interest are fixed effects in a huge data set
I am interested in estimating a Poisson fixed effects model with: where is the "age" of the observation. I am interested in the coefficients, not the other fixed effects. My first ...
0 votes
1 answer
64 views
mcmapply stalls with Rcpp/Armadillo function using openMP
I have issues with mcmapply that only occures if an Rcpp-function, testfun(), is first called outside mcmapply and then called through mcmapply. The issue only exists if: testfun() uses openMP ...
4 votes
0 answers
99 views
R parallel: Stopping a cluster you don't have the object for?
I am working with a function that starts with parallel::makeCluster() and ends with parallel::stopCluster(). Sometimes it crashes in the middle and does not stop the cluster. Demonstration code: ...
0 votes
1 answer
107 views
Error in parallel calculation on R function
I am mastering parallel programming in R simple examples tried to implement the function given in the code, it loads the processor, but the results of calculations are not. Could you please tell me ...
0 votes
0 answers
122 views
How to run R commands in parallel with controlled memory usage?
I have a large object (~50GB) that I wish to break into roughly 10 non-disjoint subsets to perform analysis on concurrently. The issue is that I have had numerous bad experiences with parallelism in R ...
0 votes
1 answer
151 views
How to use imported names from box modules inside parallel code?
Here is a minimal example showing the issue: mod.r: #' @export run_sqrt <- function (x) { sqrt(x) } mwe.r box::use( ./mod[...], parallel, dp = doParallel, foreach[foreach, `%dopar%`], ) ...
1 vote
1 answer
788 views
Error in checkForRemoteErrors(val) : 7 nodes produced errors; first error: could not find function "fread"
All of the code included in this question is from the script called "LASSO code (Version for Antony)" in my GitHub Repo for this project. And you can run it on the file folder called "...
1 vote
1 answer
487 views
Passing objects via clusterExport or as function arguments
Is it more efficient to pass objects to parallel::parLapply and parallel::parLapplyLB as function arguments or to export them with parallel::clusterExport? I.e. parallel::parLapply(cl, 1:1000, ...
1 vote
1 answer
95 views
improve readability of output list of foreach loop
I have a simple example of foreach nested loop below. How can I improve the readability of the result res? Below is another example of nested for loop with more readable results, each element of list ...
1 vote
1 answer
455 views
Getting an error in using foreach in R - no applicable method for 'filter' applied to an object of class "list"
I am getting in using foreach. Can someone please explain the reason. Error in UseMethod("filter") : no applicable method for 'filter' applied to an object of class "list" ...
0 votes
1 answer
114 views
R: Compare each element with all the other elements below in a list using foreach, parallel and doParallel
Aim: I'm trying to compare each element in a list with all the other elements below it using Levenshtein distance from this package stringsim to find text that is similar. Obstacle: The problem is ...
3 votes
1 answer
655 views
Debug R parallel mclapply
It is difficult to debug the error code of mclapply because all values of a job are affected. I prepared a simple example. library(parallel) library(dplyr) data(iris) ## Parallel Version parFun <-...
0 votes
1 answer
462 views
R optimParallel uses huge amounts of RAM
On my (large) server (Windows with 255GB RAM) my optimparallel skript is running out of memory and then crashes with Error in serialize(data, node$con) : error writing to connection. While I would ...