Skip to main content

performance tuning - Efficiency problem with least square algorithm (Error in both variables)


I was informing myself about the least square algorithms with errors in x and y. I found this post and the top answer wasn't working for magnitudes around 10^-20. Because I couldn't figure out why, I completed the code from belisarius to also produce errors on slope and intercept. But my code is really, really slow, if I take 10 or more {x, y} pairs with errors it takes way too long. I think it's the Reduce part that slows it down so much, but I don't know how to make it more efficient. Also the function output isn't really elegant yet.



LinFit[xi_List, yi_List, errx_List, erry_List]:=
  Module[{n=Length@xi,wi,ui,vi,wmean,d,g,a,b,set,least,c,wxi,wyi},
  wxi=errx^-2;
  wyi=erry^-2;
  wi[i_,m_]:=wxi[[i]] wyi[[i]]/(m^2 wyi[[i]]+wxi[[i]]);
  ui[i_,m_]:=xi[[i]]-wmean[xi,m];
  vi[i_,m_]:=yi[[i]]-wmean[yi,m];
  wmean[q_List,m_]:=Sum[wi[i,m] q[[i]],{i,n}]/Sum[wi[i,m],{i,n}];
  d[m_]:=Sum[wi[i,m]^2 ui[i,m]^2/wxi[[i]],{i,n}];
  g[m_]:=-Sum[wi[i,m] ui[i,m] vi[i,m],{i,n}]/d[m];

  a[m_]:=2 Sum[wi[i,m]^2 ui[i,m] vi[i,m]/wxi[[i]],{i,n}]/(3 d[m]);
  b[m_]:=(Sum[wi[i,m]^2 vi[i,m]^2/wxi[[i]],{i,n}]-Sum[wi[i,m] ui[i,m]^2{i,n}])/(3d[m]);

  set={ToRules@Reduce[\[FormalM]^3-3 a[\[FormalM]] \[FormalM] \[FormalM]+
        3 b[\[FormalM]] \[FormalM]-g[\[FormalM]]==0&&\[FormalC]==wmean[yi,\[FormalM]]-
        \[FormalM]wmean[xi,\[FormalM]]&&\[FormalA]==Sqrt[1/(n-2) Sum[wi[i,\[FormalM]]
        ( \[FormalM] ui[i,\[FormalM]]-vi[i,\[FormalM]])^2,{i,n}]/
        Sum[wi[i,\[FormalM]] ui[i,\[FormalM]]^2,{i,n}]]&&\[FormalB]==Sqrt[(
        Sum[wi[i,\[FormalM]] xi[[i]]^2,{i,n}]/Sum[wi[i,\[FormalM]],{i,n}])*
        \[FormalA]^2],{\[FormalM],\[FormalC],\[FormalA],\[FormalB]}, 

        Backsubstitution->True]};
  least=Sum[wxi[[i]] (xi[[i]]-(yi[[i]]-\[FormalC])/\[FormalM])^2+wyi[[i]] (yi[[i]]-
           (\[FormalM] xi[[i]]+\[FormalC]))^2,{i,Length@xi}]/.set[[Flatten@Position[
           \[FormalM]/.set,_Real]]];
  c=Flatten@set[[Flatten@Position[\[FormalM]/.set,_Real]]][[Position[
      least,Min@least][[1]]]];
  {Function[(\[FormalM]/.c[[1]])#+\[FormalC]/.c[[2]]][x],
      {\[FormalA]/.c[[3]],\[FormalB]/.c[[4]]}}
]

Answer




"worals" is an acronym for Weighted Orthogonal Regression by Alternating Least Squares.
Arguments: x and y are lists of measured values,
sx and sy are lists of the corresponding standard errors of measurement.
The returned values are {chisquare, {intercept, slope}}.


worals[x_, y_, sx_, sy_] := Block[{a,b,f,z, u = 1/sx, v = 1/sy, w = (sy/sx)^2},
{a,b} = (y*v).PseudoInverse@{v,x*v}; f = #.#&[(a+b*x-y)v];
While[f > (z = (x*w + (y-a)b)/(b^2 + w);
           {a,b} = (y*v).PseudoInverse@{v,z*v};
           f = #.#&@Join[(z-x)u,(a+b*z-y)v])];
{f,{a,b}}]


If the true relation is linear, and if the errors are independent normal with zero means and standard deviations as given in sx & sy, (or, more to the point, if the foregoing are not too far from truth) then the returned chisquare will have a Chi-Square distribution with n-2 degrees of freedom, where n = Length@x. The corresponding p-value is GammaRegularized[(n-2)/2, chisquare/2].


EDIT, responding to questions.


The model is {x = z + d, y = a + b*z + e}, where d & e are random errors, and a, b, & z are unknowns for which values are to be found that minimize the weighted sum of squares f = #.# & @ Join[(z-x)/sx,(a+b*z-y)/sy]. We start with z = x, then minimize f alternately with respect to either {a,b} or z with the other held constant, until f no longer changes.


To estimate the error in {a,b}, I usually jackknife the solution:


n = Length@x; {f,ab} = worals[x,y,sx,sy];
{jab,jc} = {ab + (n-1)(ab-Mean@#), (n-1)^2/n Covariance@#}& @
           Table[Last[worals@@(Delete[#,i]&/@{x,y,sx,sy})],{i,n}]

jab is the jackknifed estimate of {a,b}, and jc is the jackknifed estimate of its covariance matrix; Sqrt@Diagonal@jc gives the estimates of the two standard errors.



Comments

Post a Comment

Popular posts from this blog

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more

mathematical optimization - Minimizing using indices, error: Part::pkspec1: The expression cannot be used as a part specification

I want to use Minimize where the variables to minimize are indices pointing into an array. Here a MWE that hopefully shows what my problem is. vars = u@# & /@ Range[3]; cons = Flatten@ { Table[(u[j] != #) & /@ vars[[j + 1 ;; -1]], {j, 1, 3 - 1}], 1 vec1 = {1, 2, 3}; vec2 = {1, 2, 3}; Minimize[{Total@((vec1[[#]] - vec2[[u[#]]])^2 & /@ Range[1, 3]), cons}, vars, Integers] The error I get: Part::pkspec1: The expression u[1] cannot be used as a part specification. >> Answer Ok, it seems that one can get around Mathematica trying to evaluate vec2[[u[1]]] too early by using the function Indexed[vec2,u[1]] . The working MWE would then look like the following: vars = u@# & /@ Range[3]; cons = Flatten@{ Table[(u[j] != #) & /@ vars[[j + 1 ;; -1]], {j, 1, 3 - 1}], 1 vec1 = {1, 2, 3}; vec2 = {1, 2, 3}; NMinimize[ {Total@((vec1[[#]] - Indexed[vec2, u[#]])^2 & /@ R...
Post a Comment
Read more

plotting - Plot 4D data with color as 4th dimension

I have a list of 4D data (x position, y position, amplitude, wavelength). I want to plot x, y, and amplitude on a 3D plot and have the color of the points correspond to the wavelength. I have seen many examples using functions to define color but my wavelength cannot be expressed by an analytic function. Is there a simple way to do this? Answer Here a another possible way to visualize 4D data: data = Flatten[Table[{x, y, x^2 + y^2, Sin[x - y]}, {x, -Pi, Pi,Pi/10}, {y,-Pi,Pi, Pi/10}], 1]; You can use the function Point along with VertexColors . Now the points are places using the first three elements and the color is determined by the fourth. In this case I used Hue, but you can use whatever you prefer. Graphics3D[ Point[data[[All, 1 ;; 3]], VertexColors -> Hue /@ data[[All, 4]]], Axes -> True, BoxRatios -> {1, 1, 1/GoldenRatio}]
Post a Comment
Read more