Store output of images processed with magick in parallel

Question

I have hundreds of images that I need to reformat using the magick package. I've written a function that uses parLapply from the parallel package to modify the images and store them in a list. However, when I try to view any of the images in the resulting list, I get: Error: Image pointer is dead. You cannot save or cache image objects between R sessions. When I use the same function with a regular lapply statement, the function works as intended, so I know it has something to do with my attempt to process in parallel.

Here is a small reprex. Thanks in advance for any help.

# Load images. img_url <- c("https://images.freeimages.com/images/large-previews/ce3/puppies-1-1308839.jpg", "https://images.freeimages.com/images/large-previews/006/young-dachshund-1362378.jpg") img_fun <- function (img) { # Set up parallel environment. require(parallel) cl <- makeCluster(2) clusterEvalQ(cl, { library(tidyverse) library(magick) }) # Process images. parLapply(cl, img, function (i) { image_read(i) %>% image_rotate(90) }) }

F. Privé · Accepted Answer · 2020-08-07 14:21:49Z

1

{magick} uses external pointers that can't be serialized, which means they can't be sent across R processes.

A solution would be to store them to disk, return nothing, and then read all the results afterwards.

answered Aug 7, 2020 at 14:21

F. Privé

11.8k3 gold badges31 silver badges81 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

nad7wf Over a year ago

Thanks for the suggestion, I'll give it a try. In the future, this would be better suited as a comment rather than an answer, as there's no working solution provided.

nad7wf Over a year ago

Also, it seems like the I/O time of this solution would eat into the speed gain achieved by parallelizing the function.

Collectives™ on Stack Overflow

Store output of images processed with magick in parallel

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related