Skip to main content

performance tuning - Can I generate a "piecewise" list from a list in a fast and elegant way?



Suppose we have a number list a:


n = 1000000;
a = RandomReal[{0, 1}, {n}];

We want to generate a list c with the same dimension of a and the relationship between the elements of a and c is:



$c[[i]] == 2 a[[i]] +3$



I think there will be no doubt that the best way to get this c is:


c = 2 a + 3


However, what if c is a "piecewise" list i.e. elements in different part of c are generated by different formula? For example, n/2 of the elements right in the middle of c are 0. while others still follow the formula above.


The most direct way I can think out is:


c1 = Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3 ], {i, n}]; // AbsoluteTiming


{7.7380000, Null}



It's simple, but too slow.


While these 2 approaches are fast:



(c2 = 2 a + 3 ; 
c2[[n/4 + 1 ;; 3 n/4]] = ConstantArray[0., {n/2}];) // AbsoluteTiming
(c3 = ConstantArray[0., {n}];
c3[[1 ;; n/4]] = 2 a[[1 ;; n/4]] + 3 ;
c3[[3 n/4 + 1 ;; -1]] = 2 a[[3 n/4 + 1 ;; -1]] + 3;) // AbsoluteTiming


{0.0760000, Null}


{0.0580000, Null}




But they are so dirty… And things will be worse when the dimension of the list becomes higher:


n=100;
b = RandomReal[{0, 1}, {n, n, n}]

d1 = Table[
If[n/4 + 1 <= i <= 3 n/4 && n/4 + 1 <= j <= 3 n/4 &&
n/4 + 1 <= k <= 3 n/4, 0., 2 b[[i, j, k]] + 3 ], {i, n}, {j,
n}, {k, n}]; // AbsoluteTiming
(d3 = ConstantArray[0., {n, n, n}];
d3[[1 ;; n/4]] = 2 b[[1 ;; n/4]] + 3;

d3[[3 n/4 + 1 ;; -1]] = 2 b[[3 n/4 + 1 ;; -1]] + 3;
d3[[n/4 + 1 ;; 3 n/4, 1 ;; n/4]] =
2 b[[n/4 + 1 ;; 3 n/4, 1 ;; n/4]] + 3;
d3[[n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] =
2 b[[n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] + 3;
d3[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 1 ;; n/4]] =
2 b[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 1 ;; n/4]] + 3;
d3[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] =
2 b[[n/4 + 1 ;; 3 n/4, n/4 + 1 ;; 3 n/4, 3 n/4 + 1 ;; -1]] + 3;); // AbsoluteTiming



{12.8440000, Null}


{0.0930000, Null}



And a nightmare when the region are irregular:


f1 = Table[If[i^2 + j^2 + k^2 <= n^2, 0., 2 b[[i, j, k]] + 3 ], 
{i, n}, {j, n}, {k, n}]; // AbsoluteTiming


{7.4530000, Null}




A fast f3 with the method shown in c3 and d3 is of course possible, but I'd like to skip it now.


So my question is, as the title said, is there an approach that's both elegant as c1 and fast as c3? Or at least not that slow as c1 and not that dirty as c3? Or, I can't have my cake and eat it too?




Well, to be honest, I knew that Compile together with Module can somewhat help (thanks for the help of @chyaong!):


c11 = Compile[{}, Module[{a = a, n = n}, 
Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3], {i, n}]]][]; // AbsoluteTiming


{0.3590000, Null}




But I still look forward to a better solution: maybe a little greed?




I'd like to point out that, though I chose "$f(a)$ and $0$" to generate list in the example above for the sake of simplicity, this question is more general, the generated list may be formed by "$f(a)$ and $g(a)$", or even by "$f(a1,a2)$ and $g(a1,a2)$". Still, I appreciated and will appreciate those answers that focus on special cases since the "$f(a)$ and $0$" cases are simple but also common.


I added the tag "difference-equations" because this question generated when exploring finite difference method (FDM). I believe it's a inevitable issue when programming FDM.


Here are two existed post that can be optimized or answered with the outcome of this discussion:


How to discretize a nonlinear PDE fast?


How do I solve a PDE with a strange boundary condition?



Answer



OK, now let me make use of the experience got from this answer.



With the help of CompilationTarget -> "C" and RuntimeOptions -> "Speed" (both are indispensable), the Table approach turns out to be the most elegant and fast and universal. The 1D case:


n = 10^6;
a = RandomReal[1, n];

pxz = Compile[{{n, _Integer}, {a, _Real, 1}},
Table[If[n/4 + 1 <= i <= 3 n/4, 0., 2 a[[i]] + 3], {i, n}],
CompilationTarget -> "C", RuntimeOptions -> "Speed"];

pxz[n, a]; // AbsoluteTiming



{0.0430000, Null}

Timings of other solutions in my machine:


(* RunnyKine's solution *)
pp[n, a]; // AbsoluteTiming
(* chyaong's two solutions *)
r1 = (2 a + 3) BitXor[k, 1] // Hash // AbsoluteTiming
r2 = (2 a + 3) k2 // Hash // AbsoluteTiming



{0.0970000, Null}
{0.0890000, 65118127}
{0.0850000, 65118127}

The Compile`GetElement trick doesn't show its power in this simple 1D case, but when it comes to the 3D irregular region:


o = 2 10^2;
b = RandomReal[1, {o, o, o}];

(* ybeltukov's approach, notice the tiny change *)

(2 b + 3)(1 - UnitStep[Outer[Plus,Range@o^2, Range@o^2, Range@o^2]- o^2]); // AbsoluteTiming

pxz3d1 = Compile[{{n, _Integer}, {a, _Real, 3}},
Table[If[i^2 + j^2 + k^2 > n^2, 0., 2 a[[i, j, k]] + 3],
{i, n}, {j, n}, {k, n}],
CompilationTarget -> "C", RuntimeOptions -> "Speed"];

pxz3d2 = Compile[{{n, _Integer}, {a, _Real, 3}},
Table[If[i^2 + j^2 + k^2 > n^2, 0., 2 Compile`GetElement[a, i, j, k] + 3],
{i, n}, {j, n}, {k, n}],

CompilationTarget -> "C", RuntimeOptions -> "Speed"];

AbsoluteTiming[pxz3d1[o, b];]
AbsoluteTiming[data = pxz3d2[o, b];]


{1.314000, Null}
{0.3870000, Null}
{0.2650000, Null}


Image3D[data]

enter image description here


Comments

Popular posts from this blog

plotting - Plot 4D data with color as 4th dimension

I have a list of 4D data (x position, y position, amplitude, wavelength). I want to plot x, y, and amplitude on a 3D plot and have the color of the points correspond to the wavelength. I have seen many examples using functions to define color but my wavelength cannot be expressed by an analytic function. Is there a simple way to do this? Answer Here a another possible way to visualize 4D data: data = Flatten[Table[{x, y, x^2 + y^2, Sin[x - y]}, {x, -Pi, Pi,Pi/10}, {y,-Pi,Pi, Pi/10}], 1]; You can use the function Point along with VertexColors . Now the points are places using the first three elements and the color is determined by the fourth. In this case I used Hue, but you can use whatever you prefer. Graphics3D[ Point[data[[All, 1 ;; 3]], VertexColors -> Hue /@ data[[All, 4]]], Axes -> True, BoxRatios -> {1, 1, 1/GoldenRatio}]

plotting - Filling between two spheres in SphericalPlot3D

Manipulate[ SphericalPlot3D[{1, 2 - n}, {θ, 0, Pi}, {ϕ, 0, 1.5 Pi}, Mesh -> None, PlotPoints -> 15, PlotRange -> {-2.2, 2.2}], {n, 0, 1}] I cant' seem to be able to make a filling between two spheres. I've already tried the obvious Filling -> {1 -> {2}} but Mathematica doesn't seem to like that option. Is there any easy way around this or ... Answer There is no built-in filling in SphericalPlot3D . One option is to use ParametricPlot3D to draw the surfaces between the two shells: Manipulate[ Show[SphericalPlot3D[{1, 2 - n}, {θ, 0, Pi}, {ϕ, 0, 1.5 Pi}, PlotPoints -> 15, PlotRange -> {-2.2, 2.2}], ParametricPlot3D[{ r {Sin[t] Cos[1.5 Pi], Sin[t] Sin[1.5 Pi], Cos[t]}, r {Sin[t] Cos[0 Pi], Sin[t] Sin[0 Pi], Cos[t]}}, {r, 1, 2 - n}, {t, 0, Pi}, PlotStyle -> Yellow, Mesh -> {2, 15}]], {n, 0, 1}]

plotting - Mathematica: 3D plot based on combined 2D graphs

I have several sigmoidal fits to 3 different datasets, with mean fit predictions plus the 95% confidence limits (not symmetrical around the mean) and the actual data. I would now like to show these different 2D plots projected in 3D as in but then using proper perspective. In the link here they give some solutions to combine the plots using isometric perspective, but I would like to use proper 3 point perspective. Any thoughts? Also any way to show the mean points per time point for each series plus or minus the standard error on the mean would be cool too, either using points+vertical bars, or using spheres plus tubes. Below are some test data and the fit function I am using. Note that I am working on a logit(proportion) scale and that the final vertical scale is Log10(percentage). (* some test data *) data = Table[Null, {i, 4}]; data[[1]] = {{1, -5.8}, {2, -5.4}, {3, -0.8}, {4, -0.2}, {5, 4.6}, {1, -6.4}, {2, -5.6}, {3, -0.7}, {4, 0.04}, {5, 1.0}, {1, -6.8}, {2, -4.7}, {3, -1....