assignment - Why can AppendTo modify a referenced list in-place but Part cannot?

Part, AppendTo, PrependTo, AddTo, etc. allow in-place modification of a list, but only Part requires that the list be referenced through a simple symbol, e.g. the following all does what you'ld expect:

foo = {1, 2, 3, 4};
AppendTo[foo, 5]

{1, 2, 3, 4, 5}

foo[[2]] = 100;
foo

{1, 100, 3, 4, 5}

But if the list reference is something more complex, then Part fails:

bar[1] = {4, 3, 2, 1};
AppendTo[bar[1], 5]

{4, 3, 2, 1, 5}

bar[1][[2]] = 100;

Set::setps: bar[1] in the part assignment is not a symbol. >>

I know there have been work-arounds posted, but does anyone know the rationale for why Part precludes this behavior?

Answer

The reason is that AppendTo and PrependTo actually replace the whole list with a new one, rather than only changing a single element. In that sense, they are not really "in-place". They are in-place in the sense that the result gets assigned back to the same variable.

You can see using Trace, that AppendTo and PrependTo in fact expand:

bar[1]={4,3,2,1};
Trace[AppendTo[bar[1],5]]

(* 
   {AppendTo[bar[1],5],{bar[1],{4,3,2,1}},bar[1]=Append[{4,3,2,1},5],

    {Append[{4,3,2,1},5],{4,3,2,1,5}},bar[1]={4,3,2,1,5},{4,3,2,1,5}} 
*)

and are mostly syntactic sugar on top of Set and Append / Prepend.

With Part, the situation is different, when we try to assign to parts of an expression in-place. In such a case, Part is a lower-level command (compared to most others), and only supports such operations when the expression being modified in-place is stored in a Symbol (rather than an indexed variable, as in your example, or a more general expression). This limitation must have to do with expressions being based on arrays internally, and also with efficiency (because it makes massive part assignments quite efficient). I have discussed this issue in a little more detail here.

To put it slightly differently, AppendTo and PrependTo have the same O(n) complexity as Append and Prepend, so them being in-place is a syntactic convenience, but doesn't require any new non-trivial (or lower-level) behavior from the language - and thus, in particular, can be implemented in the top-level. Part[s, pos] = newelem is an operation that we expect to have O(1) complexity, however. The top-level can only provide O(n) by doing something along the lines of bar[1] = ReplacePart[bar[1], 2 -> 100], but this is unacceptable performance-wise.

So, the language needs to do some lower-level work to make the complexity right, but then it is natural that, for a quite general language based on immutable expressions, there will be limitations like that (only Symbols allowed). If you allow other expressions (like indexed variables) to be L-values for part assignments, you'll inevitably have to allow top-level evaluation during those assignments. That, and the absence of true pass-by-reference semantics, would make both implementation much more complex, and performance in general much harder to predict. My guess is that these are some of the reasons why those limitations exist.