Skip to main content

performance tuning - Can I generate a "piecewise" list from a list in a fast and elegant way?



Suppose we have a number list a:


n = 1000000;
a = RandomReal[{0, 1}, {n}];

We want to generate a list c with the same dimension of a and the relationship between the elements of a and c is:



$c[[i]] == 2 a[[i]] +3$



I think there will be no doubt that the best way to get this c is:


c = 2 a + 3


However, what if c is a "piecewise" list i.e. elements in different part of c are generated by different formula? For example, n/2 of the elements right in the middle of c are 0. while others still follow the formula above.


The most direct way I can think out is:


c1 = Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3 ], {i, n}]; // AbsoluteTiming


{7.7380000, Null}



It's simple, but too slow.


While these 2 approaches are fast:



(c2 = 2 a + 3 ; 
c2[[n/4 + 1 ;; 3 n/4]] = ConstantArray[0., {n/2}];) // AbsoluteTiming
(c3 = ConstantArray[0., {n}];
c3[[1 ;; n/4]] = 2 a[[1 ;; n/4]] + 3 ;
c3[[3 n/4 + 1 ;; -1]] = 2 a[[3 n/4 + 1 ;; -1]] + 3;) // AbsoluteTiming


{0.0760000, Null}


{0.0580000, Null}




But they are so dirty… And things will be worse when the dimension of the list becomes higher:


n=100;
b = RandomReal[{0, 1}, {n, n, n}]

d1 = Table[
If[n/4 + 1 <= i <= 3 n/4 && n/4 + 1 <= j <= 3 n/4 &&
n/4 + 1 <= k <= 3 n/4, 0., 2 b[[i, j, k]] + 3 ], {i, n}, {j,
n}, {k, n}]; // AbsoluteTiming
(d3 = ConstantArray[0., {n, n, n}];
d3[[1 ;; n/4]] = 2 b[[1 ;; n/4]] + 3;

d3[[3 n/4 + 1 ;; -1]] = 2 b[[3 n/4 + 1 ;; -1]] + 3;
d3[[n/4 + 1 ;; 3 n/4, 1 ;; n/4]] =
2 b[[n/4 + 1 ;; 3 n/4, 1 ;; n/4]] + 3;
d3[[n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] =
2 b[[n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] + 3;
d3[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 1 ;; n/4]] =
2 b[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 1 ;; n/4]] + 3;
d3[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] =
2 b[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] + 3;); // AbsoluteTiming



{12.8440000, Null}


{0.0930000, Null}



And a nightmare when the region are irregular:


f1 = Table[If[i^2 + j^2 + k^2 <= n^2, 0., 2 b[[i, j, k]] + 3 ], 
{i, n}, {j, n}, {k, n}]; // AbsoluteTiming


{7.4530000, Null}




A fast f3 with the method shown in c3 and d3 is of course possible, but I'd like to skip it now.


So my question is, as the title said, is there an approach that's both elegant as c1 and fast as c3? Or at least not that slow as c1 and not that dirty as c3? Or, I can't have my cake and eat it too?




Well, to be honest, I knew that Compile together with Module can somewhat help (thanks for the help of @chyaong!):


c11 = Compile[{}, Module[{a = a, n = n}, 
Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3], {i, n}]]][]; // AbsoluteTiming


{0.3590000, Null}




But I still look forward to a better solution: maybe a little greed?




I'd like to point out that, though I chose "$f(a)$ and $0$" to generate list in the example above for the sake of simplicity, this question is more general, the generated list may be formed by "$f(a)$ and $g(a)$", or even by "$f(a1,a2)$ and $g(a1,a2)$". Still, I appreciated and will appreciate those answers that focus on special cases since the "$f(a)$ and $0$" cases are simple but also common.


I added the tag "difference-equations" because this question generated when exploring finite difference method (FDM). I believe it's a inevitable issue when programming FDM.


Here are two existed post that can be optimized or answered with the outcome of this discussion:


How to discretize a nonlinear PDE fast?


How do I solve a PDE with a strange boundary condition?



Answer



OK, now let me make use of the experience got from this answer.



With the help of CompilationTarget -> "C" and RuntimeOptions -> "Speed" (both are indispensable), the Table approach turns out to be the most elegant and fast and universal. The 1D case:


n = 10^6;
a = RandomReal[1, n];

pxz = Compile[{{n, _Integer}, {a, _Real, 1}},
Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3], {i, n}],
CompilationTarget -> "C", RuntimeOptions -> "Speed"];

pxz[n, a]; // AbsoluteTiming



{0.0430000, Null}

Timings of other solutions in my machine:


(* RunnyKine's solution *)
pp[n, a]; // AbsoluteTiming
(* chyaong's two solutions *)
r1 = (2 a + 3) BitXor[k, 1] // Hash // AbsoluteTiming
r2 = (2 a + 3) k2 // Hash // AbsoluteTiming



{0.0970000, Null}
{0.0890000, 65118127}
{0.0850000, 65118127}

The Compile`GetElement trick doesn't show its power in this simple 1D case, but when it comes to the 3D irregular region:


o = 2 10^2;
b = RandomReal[1, {o, o, o}];

(* ybeltukov's approach, notice the tiny change *)

(2 b + 3)(1 - UnitStep[Outer[Plus,Range@o^2, Range@o^2, Range@o^2]- o^2]); // AbsoluteTiming

pxz3d1 = Compile[{{n, _Integer}, {a, _Real, 3}},
Table[If[i^2 + j^2 + k^2 > n^2, 0., 2 a[[i, j, k]] + 3],
{i, n}, {j, n}, {k, n}],
CompilationTarget -> "C", RuntimeOptions -> "Speed"];

pxz3d2 = Compile[{{n, _Integer}, {a, _Real, 3}},
Table[If[i^2 + j^2 + k^2 > n^2, 0., 2 Compile`GetElement[a, i, j, k] + 3],
{i, n}, {j, n}, {k, n}],

CompilationTarget -> "C", RuntimeOptions -> "Speed"];

AbsoluteTiming[pxz3d1[o, b];]
AbsoluteTiming[data = pxz3d2[o, b];]


{1.314000, Null}
{0.3870000, Null}
{0.2650000, Null}


Image3D[data]

enter image description here


Comments

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...

equation solving - Invert and fit implicitly defined curve

I need to fit an implicitly defined curve. I thought I could get some data out of Solve , and then using FindFit . Therefore, I would like to find the relation the parametric curve defined by $F(x,y)=0$: Solve[-(1/2) + 1/2 (0.41202 BesselK[0, 0.1 Sqrt[x^2 + y^2]] + (0.101483 x BesselK[1, 0.1 Sqrt[x^2 + y^2]])/Sqrt[x^2 + y^2]) == 0, y] But I can't get an output: Solve was unable to solve the system with inexact coefficients or the system obtained by direct rationalization of inexact numbers present in the system. Since many of the methods used by Solve require exact input, providing Solve with an exact version of the system may help. >> Edit: In particular, I would like to fit the data coming from the curve with the expression of another curve, and not with a function $f(x)$. In particular, since this clearly looks like a cardioid , I would like it to fit to something like it. What other strategies could I try?

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...