programming - ++ is dangerous for C programmers

I noticed this fact, that may be misleading for programmers used to C language.

In Mathematica, if you have a function f[] and an array v, and you write

v[[ f[] ]]++

the function f is called twice. Probably experts knows this very well, but I used MMA for years ignoring this. Normally this behaviour is harmless, but this should be taken into account if f is costly, has side-effects or can return different values based on the same input.

Indeed, I realized this property of ++ because of this:

t={0,0,0}; r[]:=RandomInteger[{1,3}]; Do[t[[r[]]]++,{10000}]; Print[t," ",Total[t]]

where I increment a random entry of t 10000 times, but at the end the sum of entries of t is not 10000.

This is insidious for C programmers, because in C if you write a similar code v[f()]++, the function f() is called just once.

I would like to ask if this semantic of ++ is somehow "forced" by the overall structure of Mathematica language or if they could have implemented it differently.

Keywords: Increment Preincrement HoldAll Hold Evaluate twice

Answer

I believe Increment (more accurately PreIncrement as george2079 noted) is essentially this:

SetAttributes[inc, HoldFirst]

inc[a_] := a = (a + 1)

This exhibits the same behavior, e.g.

f[] := (Print[#]; #) &@RandomInteger[{1, 3}]

v = {0, 0, 0};
inc @ v[[f[]]];

Here f[] is evaluated twice because parameter a is used twice within Set. On the right hand side it draws the actual element to which to add one, and on the left hand side it is evaluated to get a valid index for assignment. (HoldFirst prevents the expression from being evaluated before it is substituted into the definition; a requirement for such an assignment to work correctly.)

This does follow from Mathematica design as a natural way to implement something like this, and evaluation follows established, if at times confusing, rules. This becomes apparent if one tries to avoid this problem. How exactly does one prevent double evaluation in a generic way? belisarius recommended in a comment memorizing f but that is a specific solution, not a broad one.

For the specific case of a Part one could add a rule:

inc[a_[[p__]]] := With[{eval = p}, a[[eval]] = (a[[eval]] + 1)]

Test:

v = {0, 0, 0};

inc @ v[[f[]]];
v

3

{0, 0, 1}

However this introduces a special case, and rather than clarifying behavior it may serve to confuse instead. After observing the above behavior one might expect this also to evaluate f[] only once, but it does not:

w[_] = 0;

inc @ w[f[]];

2

3

So we may end up chasing our tails trying to pin down special cases, rather than accepting the simple rule inc[a_] := a = (a + 1) and the evaluation it implies.

Other operators affected

This same mechanism affects not only Increment, Decrement, and their Pre- forms, but all special assignment operators:

v[[ f[] ]] += 2;

1

2

v[[ f[] ]] -= 2;

3

2

v[[ f[] ]] /= 2;

1

3

v[[ f[] ]] *= 2;

2

2

In the case of AppendTo and PrependTo this can lead to seemingly errant behavior as noted in Problem with function inside brackets. Bug?:

v = {{1}, {2}, {3}};

AppendTo[v[[ f[] ]], 4];

v

{{1}, {3, 4}, {3}}

PrependTo[v[[ f[] ]], 5];

v

{{1}, {5, 1}, {3}}

Rethinking my assertions

This answer has proven unexpectedly popular, and in such cases I try to "channel" Leonid and take it to the next level in an effort to deserve the attention.

While I believe what I wrote above holds if we are using the standard evaluation elements to emulate this functionality it is also true that there is nonstandard evaluation in various parts of the familiar language, one of these cases being Set itself. Consider that the statement v[[ f[] ]] = 1 manages to correctly change a part of the vector assigned to v, rather than generating an error as would happen if the entire LHS were fully evaluated, and yet it does evaluate f[] rather than attempting to assign something to part "f[]" verbatim. This partial LHS evaluation has been discussed before:

It occurs to me that Increment and PreIncrement could potentially use a similar special evaluation in an attempt to prevent exactly the kind of double evaluation under discussion. Unfortunately I cannot think of any simple way to emulate this special LHS evaluation in order to implement this myself. I shall continue to think on the problem, but perhaps WReach or Leonid will have an implementation to offer in the mean time.

Blog

Search This Blog