Sporadic bugs in multidimensional SparseArray objects

Bug introduced in 9.0 and fixed in 10.0

Update

This SparseArray[] bug is (apparently) fixed on version 10.0

$System//Print; $Version//Print; Mac OS X x86 (64-bit) 10.0 for Mac OS X x86 (64-bit) (June 29, 2014) 

Thanks, Wolfram!

Resolution

• WRI affirms that SparseArray[__] is broken (in 9.0 and 9.01)
• WRI's suggested workaround is SparseArray//Normal

My experience with this particular SparseArray bug has been that reliable coding requires that the Normal workaround be applied to every instance of SparseArray, because the failures of SparseArray are sporadic, occult, and associated to unknown preconditions.

Ouch!

My thanks are extended to all who commented.

Wolfram Technical Support email: [email protected] Re: [CASE:646986] Sporadic bugs (?) in multidimensional `SparseArray` objects Hi John, Thank you for getting back to us. I understand that for now your temporary workaround of `aTest = aTest // Normal` is holding the fort. Our developers are working on this issue and we anticipate that the resolution will appear in the future release of Mathematica. Sincerely, <name redacted> Wolfram Technology Group Wolfram Research, Inc. 

What appear to be sporadic bugs (?) in multidimensional SparseArray objects (?) are plaguing our code development efforts.

$Version//Print; 9.0 for Mac OS X x86 (64-bit) (January 24, 2013) 

The following code elicits wrong answers and/or kernel crashes:

Do[ (* aTest is a SparseArray[...] that crashes the kernel and/or yields incorrect results unless wrapped in Normal[...] *) aTest = RandomReal[NormalDistribution[0,1], {2,2,2,2}] + I*RandomReal[NormalDistribution[0,1], {2,2,2,2}]// SparseArray[ { {i_,j_,k_,1 ,i_} :> #[[k,1,i,j]]*#[[k,2,1,j]], {i_,j_,k_,2 ,j_} :> #[[k,1,i,j]]*#[[k,2,2,i]], {i_,j_,k_,n_,j_} :> #[[k,1,i,j]]/;Equal[n-2,i] }, {2,2,2,4,2}, Complex[0.0,0.0] ]&//Flatten[#,1]&// Transpose[#,{4,1,2,3}]&// Flatten[#,2]&// Transpose; (* enable the following line to "fix" aTest *) (* aTest = aTest//Normal; *) (* the following all should be 0.0 (to machine precision) *) (aTest//Normal)-aTest// Flatten//Abs//Max//Print[" (aTest//Normal)-aTest =?= 0 = ",#]&; id = SparseArray[{i_,i_}\[Rule]1,{16,16},0]; (aTest//Normal).id-aTest.id// Flatten//Abs//Max//Print["(aTest//Normal).id-aTest.id =?= 0 =",#]&; ,{10}]; 

Results (when the kernel doesn't crash immediately)

 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =7.43925 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =3.14861 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =1.27373 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =1.8666 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =2.03269 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =3.26388 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =1.30013 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =3.14138 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =0.953447 (aTest//Normal)-aTest =?= 0 = 0. (aTest//Normal).id-aTest.id =?= 0 =0.841252 

Remarks Variants of the above code crash/fail for Complex-valued and Integer-valued SparseArrays; apparently the multidimensionality is the essential trait.

Help, advice, and workarounds all are appreciated!