variable definitions - How to Set parts of indexed lists?

I would like to assign a list to an indexed variable and then change it using Part and Set like this:

matM[i] = ConstantArray[1, {10, 10}];
matM[i][[1,10]] = 10

Unfortunately I will get the error:

Set::setps: "matM[i] in the part assignment is not a symbol. "

Using Subscript will not work either:

Subscript[matM,i] = ConstantArray[1, {10, 10}];
Subscript[matM,i][[1,10]] = 10

will give the same error.

How can this be done?

Answer

I can offer a sort of a solution, which has its shortcomings, but will allow you (more or less) to use the syntax you want. The idea is that you mark symbols you need, as "references", and then execute your code in a special dynamic environment where Part is overloaded.

Implementation

Here is the code. This function will mark you symbol as a reference.

ClearAll[makeReference];
makeReference[sym_Symbol] :=
  sym /: Set[sym[index_], rhs_] :=
     With[{index = index},
      With[{heldVal = Hold[sym[index]] /. DownValues[sym]},
        If[heldVal === Hold[sym[index]],

           Module[{ref},
              AppendTo[DownValues[sym], 
                    HoldPattern[sym[index]] :> ref]
           ]
        ];
        Hold[sym[index]] /. DownValues[sym] /. Hold[s_] :> (s = rhs);
      ]];

What happens is that we "softly" overload Set on this particular symbol via UpValues, so that an intermediate symbol is inserted where the actual data will be stored, and our symbol (for a given index) refers to that intermediate symbol. Since the latter has no restrictions on part assignments, we can assign parts of it directly at O(1) time.

However, the subtlety is that when we call Set[Part[s[ind],1,2],something], Set holds its first argument, and therefore, s can not communicate to Set that this is special (UpValues won't work here since the s is too deep inside an expression - on level 2 - while UpValues are only looked at at level 1). To solve this problem, we will overload Part, but do it locally within a local dynamic environment, to make this operation safer. This is a dynamic environment:

ClearAll[withModifiedPart];
SetAttributes[withModifiedPart, HoldAll];
withModifiedPart[code_] :=
   Internal`InheritedBlock[{Part},
      Unprotect[Part];
      Part /: Set[Part[sym_Symbol[index_], inds__], rhs_] :=
        With[{index = index},
          Hold[sym[index]] /. DownValues[sym] /. 
               Hold[s_] :> (s[[inds]] = rhs);
        ];

      Protect[Part];
      code];

Tests

Now, we can test this:

ClearAll[a];
makeReference[a];

and then

withModifiedPart[

   a[1] = Range[10];
   a[1][[2]] = 100;
   a[1]
]

 {1, 100, 3, 4, 5, 6, 7, 8, 9, 10}

Let's now measure some timings:

withModifiedPart[

    a[1] = Range[100000];
    Do[a[1][[i]] = a[1][[i]] + 1, {i, a[1]}];
    a[1]
] // Short // AbsoluteTiming

  {1.5126953,{2,3,4,5,6,7,8,9,<<99984>>,99994,99995,99996,99997,
    99998,99999,100000,100001}}

We can compare this to the time it takes for direct assignments:

aa = Range[100000];
Do[aa[[i]] = aa[[i]] + 1, {i, aa}]; // Short // AbsoluteTiming

 {0.2470703,Null}

So, for massive assignments, we are about 6 times slower (which I think is OK). We can also see how costly is the overloaded Part for normal assignments:

withModifiedPart[
  aa=Range[100000];
  Do[aa[[i]]=aa[[i]]+1,{i,aa}]

 ];//Short//AbsoluteTiming

  {0.2822266,Null}

from where it looks that those are slower by about 15 percents.

Conclusions

The suggested solution requires 2 modifications to your code:

call makeReference on symbols which you wish to use as indexed symbols with part assignment, prior to assigning to them.

Execute all your code containing such assignments, inside the withModifiedPart environment.

So, your original code will be changed to

ClearAll[matM];
makeReference[matM];

withModifiedPart[
  matM[i] = ConstantArray[1, {10, 10}];
  matM[i][[1, 10]] = 10
]

What about safety? I would argue that, for this particular case, modifying Part is safe enough. This is because, first, Part is overloaded softly, via UpValues. This means that, when Part is not inside some head which holds arguments, it will execute normally before it would even "think" of a new definition. Now, when it is inside some head which holds arguments, the new definition will only work if that head is Set. Note that no ne rules were added to the Set itself. And since normally, assignments to indexed variables are not allowed anyway, we don't risk breaking some internal behavior.

The performance here is worse than for direct assignments to symbol's parts, but overall seems acceptable.

Blog

Search This Blog