4
$\begingroup$

I have the following two lists (each containing over 500,000 elements). Here is a sample:

lis1 = { {1.86582, 1.70162, 1.25256}, {1.82707, 1.29901, 1.10659}, {1.76547, 1.21544, 1.09433}, {1.18306, 1.28322, 1.75524}, {1.12555, 1.98011, 1.53359}, {1.10584, 1.12299, 1.88411}, {1.83799, 1.5275, 1.76179}, {1.42352, 1.45163, 1.45318}, {1.63669, 1.78145, 1.60307}, {1.61749, 1.44287, 1.57405} };

and 

lis2 = {0.826095, 0.73286, 0.918137, 0.937434, 0.506525, 0.562795, 0.664915, 0.789321, 0.6559, 0.398447}

They both have equal Lengths. I want to combine both Lists to obtain a new list that looks like this

{ {1.86582, 1.70162, 1.25256, 0.826095}, {1.82707, 1.29901, 1.10659, 0.73286}, {1.76547, 1.21544, 1.09433, 0.918137}, {1.18306, 1.28322, 1.75524, 0.937434}, {1.12555, 1.98011, 1.53359, 0.506525}, {1.10584,1.12299, 1.88411, 0.562795}, {1.83799, 1.5275, 1.76179, 0.664915}, {1.42352, 1.45163, 1.45318, 0.789321}, {1.63669, 1.78145,1.60307, 0.6559}, {1.61749, 1.44287, 1.57405, 0.398447} }

That is, each element of the combined list contains the corresponding elements of lis1 and lis2, with the first three elements being from lis1 and the fourth element from lis2. Here is what I did to combine them in this fashion:

Transpose[{lis1, lis2}] //. {{a_, b_, c_}, d_} :> {a, b, c, d}

Is there a faster way to achieve this as my lists are huge ?

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ Join[lis1, List /@ lis2, 2] $\endgroup$ Commented Mar 5, 2013 at 19:58
  • $\begingroup$ Flatten /@ Thread[{lis1, lis2}] ... some benchmarking might be in order... $\endgroup$ Commented Mar 5, 2013 at 20:03
  • $\begingroup$ This answer can help you... $\endgroup$ Commented Mar 5, 2013 at 20:10
  • $\begingroup$ Just a quick note, as you are discussing structure, readability is very important, so I re-organized your lists to emphasize their structure and make it more easily apparent what you are trying to achieve. $\endgroup$ Commented Mar 5, 2013 at 20:19
  • 2
    $\begingroup$ @Rojo Nice one, very fast. The bottleneck in your code is in List /@ .... This version will be considerably faster still: Join[lis1, Transpose[{lis2}], 2]. $\endgroup$ Commented Mar 5, 2013 at 20:41

1 Answer 1

Reset to default
4
$\begingroup$

There is also Transpose : 

Transpose @ Append[ Transpose[lis1], lis2]

Transpose is a really fast approach, and you need to append only one list lis2. So this probably can be faster than Rojo's approach and :

Transpose @ Append[ Transpose[lis1], lis2] == Join[lis1, List /@ lis2, 2] 
 True

Edit

Leonid Shifrin provided in the comments another neat approach :

Join[ large1, Transpose[{large2}], 2]

which appears to be even faster on one-core machines and sometimes on multi-core ones. Following his idea we can use this testing function :

test := ( large1 = RandomReal[{0, 2.}, {10^6, 3}]; large2 = RandomReal[{0, 2.}, 10^6]; { Do[ Join[ large1, List /@ large2, 2], {100}]// AbsoluteTiming //First, Do[ Transpose @ Append[ Transpose[large1], large2], {100} ] // AbsoluteTiming // First, Do[ Join[ large1, Transpose[{large2}], 2], {100}]// AbsoluteTiming// First})

Here are some benchmarks :

0n Win 7 64 bit AMD Phenom II x6 2.8 GHz, V9_0 :

 test 
{10.413086, 8.477539, 5.245117}

while RunnyKine on his machine :

Win 8 64bit Intel Core i7-2600 3.4GHz 16GB RAM, V9.0, got

test 
{6.006038, 1.794012, 1.825212}

Mr.Wizard on v7.0, Win7-64, i5-2500K 8GB RAM:

test 
{6.2400087, 2.5000035, 2.2700032}

on one-core old Pentium(R) 3.0 GHz 1GB RAM, V9.0

 test 
{16.812500, 14.328125, 9.203125}

All the above benchmarks seem to suggest that Transpose remarkably accelerates on multi-core machines and then Transpose @ Append[ Transpose[lis1], lis2] might be faster than Leonid's solution.

Improve this answer
$\endgroup$
13
  • $\begingroup$ This is fast. +1. It's even better as the List size grows. $\endgroup$ Commented Mar 5, 2013 at 20:26
  • $\begingroup$ Good one, +1. See my comment on @Rojo's code, where I suggest an improvement, which makes it about twice faster than yours. $\endgroup$ Commented Mar 5, 2013 at 20:43
  • $\begingroup$ @LeonidShifrin, actually as the List gets bigger, his is actually faster (2x) at least on my computer. $\endgroup$ Commented Mar 5, 2013 at 20:47
  • $\begingroup$ @RunnyKine Did you see my comment? I suggested an improvement on Rojo's code, and it is that improved version that, at least on my machine, is about 4 times faster than the original one, and twice faster than Artes's version, for large lists. I tested for the length 10^5 and 10^6. Heuristically, it seems that the reason my version is twice faster is that it has only one Transpose operation. $\endgroup$ Commented Mar 5, 2013 at 20:53
  • 1
    $\begingroup$ @LeonidShifrin, thanks for adding the benchmarks, we were editing at the same time. On my computer the corresponding times are: {6.006038, 1.794012, 1.825212}. $\endgroup$ Commented Mar 5, 2013 at 22:01

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.