Consider the following code:
tab = RandomReal[{0, 1}, {3*10^7, 2}]; lenblock = 10^3; len = Length[tab]; tabfinal = Partition[tab, lenblock]; // AbsoluteTiming Here, I prepare a table with the number of rows len equal to lenblock times some integer number, and then use partitioning to split it into sub-tables with lenblock. It works fast, for 0.06 s.
Now, consider another situation:
tab = RandomReal[{0, 1}, {3*10^7 + 103, 2}]; lenblock = 10^3; len = Length[tab]; lenlastblock = Length[tab] - lenblock*IntegerPart[len/lenblock]; Here, len $\neq$ lenblock times some integer. Then, I want to partition it either into n1 blocks with length lenblock and the remaining one with length lenlastblock given by len-n1*lenblock, or into n1 blocks with varying number of elements (say, the first m blocks have lenblock+1 elements, and the remaining ones have lenblock elements).
The fastest way I arrive at this is
tabfinal = Join[Partition[Drop[tab, -lenlastblock], lenblock], {Drop[tab, (len - lenlastblock)]}]; // AbsoluteTiming Note Drop in Partition[Drop[tab, -lenlastblock], lenblock] - without it, the code would be even slower. Alternatives, like TakeList and Partition[tab, UpTo[lenblock]], are comparable in timing or even (Partition) much slower. This is why I prefer to use Take and Join in combination with Partition.
The bottleneck of this code is joining - it slows down the computation by a factor of ~4-8.
Is there any way to avoid this expensive operation and keep the speed?
Partition[tab, UpTo[lenblock]]$\endgroup$