Timeline for How can I reduce RAM usage when using ParallelMap with large data?
Current License: CC BY-SA 4.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 5 at 14:42 | comment | added | internet | @azerbajdzan I guess you're right. The output is quite large, and it probably causes the RAM to overflow. | |
| Nov 5 at 10:34 | comment | added | azerbajdzan | If the output of your function is similarly large as an input than splitting the data will not improve things. So maybe saving output to hard disk for each part and joining them afterwards may help. | |
| Nov 5 at 7:33 | history | edited | internet | CC BY-SA 4.0 | added 14 characters in body |
| Nov 5 at 7:27 | history | asked | internet | CC BY-SA 4.0 |