Skip to main content

performance tuning - Efficiency problem with least square algorithm (Error in both variables)


I was informing myself about the least square algorithms with errors in x and y. I found this post and the top answer wasn't working for magnitudes around 10^-20. Because I couldn't figure out why, I completed the code from belisarius to also produce errors on slope and intercept. But my code is really, really slow, if I take 10 or more {x, y} pairs with errors it takes way too long. I think it's the Reduce part that slows it down so much, but I don't know how to make it more efficient. Also the function output isn't really elegant yet.



LinFit[xi_List, yi_List, errx_List, erry_List]:=
  Module[{n=Length@xi,wi,ui,vi,wmean,d,g,a,b,set,least,c,wxi,wyi},
  wxi=errx^-2;
  wyi=erry^-2;
  wi[i_,m_]:=wxi[[i]] wyi[[i]]/(m^2 wyi[[i]]+wxi[[i]]);
  ui[i_,m_]:=xi[[i]]-wmean[xi,m];
  vi[i_,m_]:=yi[[i]]-wmean[yi,m];
  wmean[q_List,m_]:=Sum[wi[i,m] q[[i]],{i,n}]/Sum[wi[i,m],{i,n}];
  d[m_]:=Sum[wi[i,m]^2 ui[i,m]^2/wxi[[i]],{i,n}];
  g[m_]:=-Sum[wi[i,m] ui[i,m] vi[i,m],{i,n}]/d[m];

  a[m_]:=2 Sum[wi[i,m]^2 ui[i,m] vi[i,m]/wxi[[i]],{i,n}]/(3 d[m]);
  b[m_]:=(Sum[wi[i,m]^2 vi[i,m]^2/wxi[[i]],{i,n}]-Sum[wi[i,m] ui[i,m]^2{i,n}])/(3d[m]);

  set={ToRules@Reduce[\[FormalM]^3-3 a[\[FormalM]] \[FormalM] \[FormalM]+
        3 b[\[FormalM]] \[FormalM]-g[\[FormalM]]==0&&\[FormalC]==wmean[yi,\[FormalM]]-
        \[FormalM]wmean[xi,\[FormalM]]&&\[FormalA]==Sqrt[1/(n-2) Sum[wi[i,\[FormalM]]
        ( \[FormalM] ui[i,\[FormalM]]-vi[i,\[FormalM]])^2,{i,n}]/
        Sum[wi[i,\[FormalM]] ui[i,\[FormalM]]^2,{i,n}]]&&\[FormalB]==Sqrt[(
        Sum[wi[i,\[FormalM]] xi[[i]]^2,{i,n}]/Sum[wi[i,\[FormalM]],{i,n}])*
        \[FormalA]^2],{\[FormalM],\[FormalC],\[FormalA],\[FormalB]}, 

        Backsubstitution->True]};
  least=Sum[wxi[[i]] (xi[[i]]-(yi[[i]]-\[FormalC])/\[FormalM])^2+wyi[[i]] (yi[[i]]-
           (\[FormalM] xi[[i]]+\[FormalC]))^2,{i,Length@xi}]/.set[[Flatten@Position[
           \[FormalM]/.set,_Real]]];
  c=Flatten@set[[Flatten@Position[\[FormalM]/.set,_Real]]][[Position[
      least,Min@least][[1]]]];
  {Function[(\[FormalM]/.c[[1]])#+\[FormalC]/.c[[2]]][x],
      {\[FormalA]/.c[[3]],\[FormalB]/.c[[4]]}}
]

Answer




"worals" is an acronym for Weighted Orthogonal Regression by Alternating Least Squares.
Arguments: x and y are lists of measured values,
sx and sy are lists of the corresponding standard errors of measurement.
The returned values are {chisquare, {intercept, slope}}.


worals[x_, y_, sx_, sy_] := Block[{a,b,f,z, u = 1/sx, v = 1/sy, w = (sy/sx)^2},
{a,b} = (y*v).PseudoInverse@{v,x*v}; f = #.#&[(a+b*x-y)v];
While[f > (z = (x*w + (y-a)b)/(b^2 + w);
           {a,b} = (y*v).PseudoInverse@{v,z*v};
           f = #.#&@Join[(z-x)u,(a+b*z-y)v])];
{f,{a,b}}]


If the true relation is linear, and if the errors are independent normal with zero means and standard deviations as given in sx & sy, (or, more to the point, if the foregoing are not too far from truth) then the returned chisquare will have a Chi-Square distribution with n-2 degrees of freedom, where n = Length@x. The corresponding p-value is GammaRegularized[(n-2)/2, chisquare/2].


EDIT, responding to questions.


The model is {x = z + d, y = a + b*z + e}, where d & e are random errors, and a, b, & z are unknowns for which values are to be found that minimize the weighted sum of squares f = #.# & @ Join[(z-x)/sx,(a+b*z-y)/sy]. We start with z = x, then minimize f alternately with respect to either {a,b} or z with the other held constant, until f no longer changes.


To estimate the error in {a,b}, I usually jackknife the solution:


n = Length@x; {f,ab} = worals[x,y,sx,sy];
{jab,jc} = {ab + (n-1)(ab-Mean@#), (n-1)^2/n Covariance@#}& @
           Table[Last[worals@@(Delete[#,i]&/@{x,y,sx,sy})],{i,n}]

jab is the jackknifed estimate of {a,b}, and jc is the jackknifed estimate of its covariance matrix; Sqrt@Diagonal@jc gives the estimates of the two standard errors.



Comments

Post a Comment

Popular posts from this blog

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more

How to thread a list

I have data in format data = {{a1, a2}, {b1, b2}, {c1, c2}, {d1, d2}} Tableform: I want to thread it to : tdata = {{{a1, b1}, {a2, b2}}, {{a1, c1}, {a2, c2}}, {{a1, d1}, {a2, d2}}} Tableform: And I would like to do better then pseudofunction[n_] := Transpose[{data2[[1]], data2[[n]]}]; SetAttributes[pseudofunction, Listable]; Range[2, 4] // pseudofunction Here is my benchmark data, where data3 is normal sample of real data. data3 = Drop[ExcelWorkBook[[Column1 ;; Column4]], None, 1]; data2 = {a #, b #, c #, d #} & /@ Range[1, 10^5]; data = RandomReal[{0, 1}, {10^6, 4}]; Here is my benchmark code kptnw[list_] := Transpose[{Table[First@#, {Length@# - 1}], Rest@#}, {3, 1, 2}] &@list kptnw2[list_] := Transpose[{ConstantArray[First@#, Length@# - 1], Rest@#}, {3, 1, 2}] &@list OleksandrR[list_] := Flatten[Outer[List, List@First[list], Rest[list], 1], {{2}, {1, 4}}] paradox2[list_] := Partition[Riffle[list[[1]], #], 2] & /@ Drop[list, 1] RM[list_] := FoldList[Transpose[{First@li...
Post a Comment
Read more

front end - keyboard shortcut to invoke Insert new matrix

I frequently need to type in some matrices, and the menu command Insert > Table/Matrix > New... allows matrices with lines drawn between columns and rows, which is very helpful. I would like to make a keyboard shortcut for it, but cannot find the relevant frontend token command (4209405) for it. Since the FullForm[] and InputForm[] of matrices with lines drawn between rows and columns is the same as those without lines, it's hard to do this via 3rd party system-wide text expanders (e.g. autohotkey or atext on mac). How does one assign a keyboard shortcut for the menu item Insert > Table/Matrix > New... , preferably using only mathematica? Thanks! Answer In the MenuSetup.tr (for linux located in the $InstallationDirectory/SystemFiles/FrontEnd/TextResources/X/ directory), I changed the line MenuItem["&New...", "CreateGridBoxDialog"] to read MenuItem["&New...", "CreateGridBoxDialog", MenuKey["m", Modifiers-...
Post a Comment
Read more