Skip to main content

graphics - ListPlot: Plotting large data fast


Mathematica produces fantastic-looking graphics, but it can be slow on large data sets. Here is an example for a (random) time series:


rv = RandomVariate[ExponentialDistribution[2], 10^5];


Plotting this takes quite some time:


t = AbsoluteTime[]; ListLinePlot[rv, PlotRange -> All]
AbsoluteTime[] - t (* Put this line into the NEXT cell, and evaluate both cells together*)

Mathematica graphics


The option PerformanceGoal->"Speed" has no effect. Turning off antialiasing makes it much faster (but if you increase the data size to 10^6 instead of 10^5, it is still VERY slow. A time series of 10^6 points is quite reasonable in my applications):


t = AbsoluteTime[]; 
Style[ListLinePlot[rv, PlotRange -> All], Antialiasing -> False]
AbsoluteTime[] - t (* This line in separate cell! *)


Mathematica graphics


Reducing the number of MaxPlotPoints makes it much faster, but completely distorts the shape of the data:


t = AbsoluteTime[]; ListLinePlot[rv, PlotRange -> All, MaxPlotPoints -> 1000]
AbsoluteTime[] - t (* This line in separate cell! *)

Mathematica graphics


Question: I am interested in tricks to show the data quickly without distorting the shape. I am showing here my own solution, which is quite a bit of a hack, but it works. Are there more elegant solutions?


Edit:


My own answer, previously posted as part of the question, is now an independent answer (see below)




Answer



My own solution:


Note: I have earlier posted this answer as part of the question itself. I have now removed it there and put it as independent answer.


(see also here for a simpler version of this)


Options[fastListPlot] = {plotPoints -> 1000, AspectRatio -> Full, PlotRange -> All, Options[ListLinePlot]} // Flatten;
fastListPlot[data_, opts:OptionsPattern[]] := 
    Module[{plotData=data, lengths, range = Automatic, points = OptionValue[plotPoints]},
        While[Depth[plotData] <= 2, plotData= {plotData}];
        lengths = Length/@plotData;
        If[NumericQ[points],

           plotData = Partition[#, Floor[Min[lengths]/points]]& /@ plotData;
           plotData = Flatten[{Min /@ #, Max /@ #}\[Transpose]] & /@ plotData;
           range = {1, Max[lengths]}
                   ];
        ListLinePlot[plotData, FilterRules[{opts, DataRange -> range,Options[fastListPlot]},Options[ListLinePlot]]
        ]
    ]

The trick is within the If statement: I partition the data into a number of blocks corresponding roughly to the resolution of my screen (usually 1000 or less, option plotPoints). Then I determine the Min and Max of each block, and draw a zig-zag line from min to max to min to max...


My solution, as presented, works for simple lists (i.e. of Depth 2), and also for lists containing more than 1 data set (Depth 3).



Examples:


fastListPlot[rv, PlotLabel -> "Fast"]
fastListPlot[rv, plotPoints -> All, PlotLabel -> "Normal, Slow"] 
fastListPlot[{rv,rv/2}, PlotLabel -> "Fast, more than 1 dataset"]

Mathematica graphics


So, very nicely, the visual structure of the data is preserved, in particular the peaks in the data are preserved and still visible - despite only plotting 2000 points (first plot) instead of 100.000 points (middle plot).


The following doesn't quite work correctly (problem with DataRange):


fastListPlot[{rv,Take[rv/2,10000]}, plotPoints -> All, PlotLabel->"Original"]
fastListPlot[{rv,Take[rv/2,10000]}, plotPoints -> 1000, PlotLabel -> "Incorrect scaling"]


Mathematica graphics


Comments

Post a Comment

Popular posts from this blog

front end - keyboard shortcut to invoke Insert new matrix

I frequently need to type in some matrices, and the menu command Insert > Table/Matrix > New... allows matrices with lines drawn between columns and rows, which is very helpful. I would like to make a keyboard shortcut for it, but cannot find the relevant frontend token command (4209405) for it. Since the FullForm[] and InputForm[] of matrices with lines drawn between rows and columns is the same as those without lines, it's hard to do this via 3rd party system-wide text expanders (e.g. autohotkey or atext on mac). How does one assign a keyboard shortcut for the menu item Insert > Table/Matrix > New... , preferably using only mathematica? Thanks! Answer In the MenuSetup.tr (for linux located in the $InstallationDirectory/SystemFiles/FrontEnd/TextResources/X/ directory), I changed the line MenuItem["&New...", "CreateGridBoxDialog"] to read MenuItem["&New...", "CreateGridBoxDialog", MenuKey["m", Modifiers-...
Post a Comment
Read more

How to thread a list

I have data in format data = {{a1, a2}, {b1, b2}, {c1, c2}, {d1, d2}} Tableform: I want to thread it to : tdata = {{{a1, b1}, {a2, b2}}, {{a1, c1}, {a2, c2}}, {{a1, d1}, {a2, d2}}} Tableform: And I would like to do better then pseudofunction[n_] := Transpose[{data2[[1]], data2[[n]]}]; SetAttributes[pseudofunction, Listable]; Range[2, 4] // pseudofunction Here is my benchmark data, where data3 is normal sample of real data. data3 = Drop[ExcelWorkBook[[Column1 ;; Column4]], None, 1]; data2 = {a #, b #, c #, d #} & /@ Range[1, 10^5]; data = RandomReal[{0, 1}, {10^6, 4}]; Here is my benchmark code kptnw[list_] := Transpose[{Table[First@#, {Length@# - 1}], Rest@#}, {3, 1, 2}] &@list kptnw2[list_] := Transpose[{ConstantArray[First@#, Length@# - 1], Rest@#}, {3, 1, 2}] &@list OleksandrR[list_] := Flatten[Outer[List, List@First[list], Rest[list], 1], {{2}, {1, 4}}] paradox2[list_] := Partition[Riffle[list[[1]], #], 2] & /@ Drop[list, 1] RM[list_] := FoldList[Transpose[{First@li...
Post a Comment
Read more

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more