0
$\begingroup$

Does Mathematica have a function similar to Table[...] but with an extra argument to specify a condition, such that terms are added to the generated list only if that condition is True?

Currently I apply Select after generating a Table, but that consumes too much memory because it generates a large list, most terms in which I do not need.

If instead I use Do[...] and check the condition for every term separately using If before adding it to the the list with AppendTo, the performance drops considerably, and the resulting code is longer than it needs to be.

I am surprized there is no version of the Table function that would take an extra condition, or maybe it is just called differently?

Apparently, the third option proposed by azerbajdzan is not always faster. I have now compared the following three codes, and the one with ## &[] actually wins!

Option 1:

CountNonCoprimes10[i_] := ParallelTable[ If[! CoprimeQ @@ #, #, ## &[]] &[{FromDigits[#], FromDigits[# /. {0 -> 1, 1 -> 0}]} &[ IntegerDigits[n, 2]]], {n, 2^(i - 1), 2^i - 1}] // Length; MaxMemoryUsed[CountNonCoprimes10[20]] // Timing

Output: {5.1875, 17993160}

Option 2:

CountNonCoprimes10[i_] := ParallelTable[ If[! CoprimeQ @@ #, #, Nothing] &[{FromDigits[#], FromDigits[# /. {0 -> 1, 1 -> 0}]} &[ IntegerDigits[n, 2]]], {n, 2^(i - 1), 2^i - 1}] // Length; MaxMemoryUsed[CountNonCoprimes10[20]] // Timing

Output: {5.84375, 17993288}

Option 3:

CountNonCoprimes10[i_] := Last@Last@ Reap@Do[If[! CoprimeQ @@ #, Sow@#] &[{FromDigits[#], FromDigits[# /. {0 -> 1, 1 -> 0}]} &[ IntegerDigits[n, 2]]], {n, 2^(i - 1), 2^i - 1}] // Length; MaxMemoryUsed[CountNonCoprimes10[20]] // Timing

Output: {6.79688, 34961728}

So now the third option clearly loses the competition, but also here I am using the nonparallelized version of Do. Is there a way to make Sow work inside ParallelDo?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ See this answer: stackoverflow.com/a/8877880/4954079 $\endgroup$ Commented Nov 28, 2024 at 10:39
  • $\begingroup$ Excellent! That version with Table[If[i < 3, i, ## &[]], {i, 5}] is exactly what I need. $\endgroup$ Commented Nov 28, 2024 at 10:44
  • 3
    $\begingroup$ @Vosoni Table[If[i < 3, i, Nothing], {i, 5}] $\endgroup$ Commented Nov 28, 2024 at 11:10
  • $\begingroup$ Oh, thanks. I didn't even know that Nothing does the same thing. Never too late to learn. $\endgroup$ Commented Nov 28, 2024 at 13:29
  • $\begingroup$ Is Nothing removed from the list after the whole list is already created, or it is not added there at all? In other words, does a list with many Nothing's take more memory? I am now testing the version with ## &[] inside ParallelTable, and that's extremely quick. $\endgroup$ Commented Nov 28, 2024 at 13:41

1 Answer 1

Reset to default
3
$\begingroup$

These three codes produce the same output {1, 2}.

Table[If[i < 3, i, Nothing], {i, 1000000}] == Table[If[i < 3, i, ## &[]], {i, 1000000}] == Last@Last@Reap@Do[If[i < 3, Sow@i], {i, 1000000}]

But if we compare the memory and time used the third code beats the first two codes in both criteria.

MaxMemoryUsed[Table[If[i < 3, i, Nothing], {i, 1000000}]] // Timing MaxMemoryUsed[Table[If[i < 3, i, ## &[]], {i, 1000000}]] // Timing MaxMemoryUsed[ Last@Last@Reap@Do[If[i < 3, Sow@i], {i, 1000000}]] // Timing 
{0.78125, 16389120} {1.01563, 48039408} {0.5625, 10888}
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks! Do you know if Sow would work correctly also inside ParallelDo in the last example? $\endgroup$ Commented Nov 28, 2024 at 17:25
  • $\begingroup$ Please see my edit of the question with 3 examples. $\endgroup$ Commented Nov 28, 2024 at 17:41
  • $\begingroup$ In the examples I provided, ParallelTable outperforms Do both in time and memory. Also ##&[] seems to be a bit faster than Nothing. Sow is incompatible with ParallelDo, I have read. $\endgroup$ Commented Nov 28, 2024 at 17:45
  • $\begingroup$ @Vosoni What did you expect when you compare job done on one processor versus on several processors? Of course it is faster on several processors. Use what is preferable for you. Sow will not work well with parallelization and probably would be slower. $\endgroup$ Commented Nov 28, 2024 at 18:24
  • $\begingroup$ The speed is clear, but why ParallelTable uses less memory than Table? I suppose that as soon as the structure no longer fits in memory, the Sow version may still outperform the parallelized version of Table. Will have to check. $\endgroup$ Commented Nov 28, 2024 at 19:57

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.