Define the function twice, once without the memoization, and once with the memoization and your condition.

Consider this simple example:

f[{}] := {} f[a_List] := Rest@a f[a_List /; Length@a <= 3] := f[a] = Rest@a 

If we run

FixedPointList[f, Range[5]] (* {{1, 2, 3, 4, 5}, {2, 3, 4, 5}, {3, 4, 5}, {4, 5}, {5}, {}, {}} *) 

then

Notice that it has saved only those lists of length less than 4.

Define the function twice, once without the memoization, and once with the memoization and your condition.

Consider this simple example:

Clear@f f[{}] = {}; f[a_List] := Rest@a f[a_List /; Length@a <= 3] := f[a] = Rest@a 

If we run

FixedPointList[f, Range[5]] (* {{1, 2, 3, 4, 5}, {2, 3, 4, 5}, {3, 4, 5}, {4, 5}, {5}, {}, {}} *) 

then

Notice that it has saved only those lists of length less than 4.

Alternatively, if it is annoying to define the function twice, do this instead. It works the same.

Clear@f f[{}] = {}; f[a_List] := If[Length@a > 3, #, f[a] = #]&@Rest@a