java 8 splitting bytes into chunks

Question

I have to create a method where I need to chunk files into multiple bytes.

examples byte[] into List<byte[]> , lets say each of it is size 1 MB (sizeMB=1 * 1024 * 1024)

so 5.2 MB file should be list of five 1MB and one 2KB. [2kb, 1MB, 1MB, 1MB, 1MB,1MB].

byte[] mainFile=getFIle(); List<bute[]> listofSplitBytes=getFileChunks(mainFile); public void list<bute[]> getFileChunks(byte[] mainFile) { int sizeMB = 1 * 1024 * 1024; // Split the files logic }

I am trying to avoid adding, if then else ,to handle . I am trying to find if there is a cleaner way to do it, like using streams or something like that?

this post may help you : stackoverflow.com/questions/39399398/… — JHDev
– JHDev, Commented Mar 13, 2021 at 13:11
For large files, this will wreak havoc on your program’s memory usage. Imagine reading a 500 MB file and having two copies of it in RAM. A better approach is returning chunks one at a time as the file is read. If that is not an option, put the main byte array in a ByteBuffer, and return slices of that ByteBuffer, so you aren’t duplicating the bytes. — VGR
– VGR, Commented Mar 13, 2021 at 16:00
Streams tag because, I wanted to look for some sort of functional programming solution. — athenatechie
– athenatechie, Commented Mar 14, 2021 at 17:41
I’m not exactly sure you what you mean by avoiding if/then/else. You pretty much have to have a condition statement since you are truncating the first chunk rather than the last one. You can use roundabout forms like ?…: or Math.max/min, but one way or another, a conditional construct is going to be necessary. — VGR
– VGR, Commented Mar 14, 2021 at 19:12

ETO · Accepted Answer · 2021-03-15 13:07:33Z

Try this:

public List<byte[]> getFileChunks(byte[] mainFile) { int sizeMB = 1 * 1024 * 1024; List<byte[]> chunks = new ArrayList<>(); for (int i = 0; i < mainFile.length; ) { byte[] chunk = new byte[Math.min(sizeMB, mainFile.length - i)]; for (int j = 0; j < chunk.length; j++, i++) { chunk[j] = mainFile[i]; } chunks.add(chunk); } return chunks; }

or if you want a functional solution, then try this:

public List<byte[]> getFileChunks(byte[] mainFile) { final int sizeMB = 1 * 1024 * 1024; return IntStream.iterate(0, i -> i + sizeMB) .limit((mainFile.length + sizeMB - 1) / sizeMB) .mapToObj(i -> Arrays.copyOfRange(mainFile, i, Math.min(i + sizeMB, mainFile.length))) .collect(Collectors.toList()); }

Collectives™ on Stack Overflow

java 8 splitting bytes into chunks

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related