I have a large data set consisting of $\mathcal{O}(10^9)$ two-dimensional points. In order to save memory and time I have pre-binned these into a uniform grid of $500 \times 500$ bins using Fortran. When imported into Mathematica 8.0 as a table the resulting data look like:
data = {{0.388348, 0.388349, 9},{0.388348, 0.776699, 23},...},
where the first two items of each entry correspond to the $x$-$y$-coordinates of the upper-right-hand corner of the bin and the third is the count.
Edit:
For a sample of the raw data,
raw=RandomReal[1,{1000000000,2}]is a good approximation. This is obviously unworkable.For the binned data:
binned=Table[{.01*Ceiling[raw[[i,1]]/.01],.01*Ceiling[raw[[i,2]]/.01],RandomInteger[1000]},{i,1,250000}].
I would like to plot this pre-binned data set in the form of a DensityHistogram, but my data format doesn't fit into what this function is expecting. I have reviewed a similar question for one-dimensional histograms at Histograms with pre-counted data, however I'm at a loss as to how to apply this to 2-D. I have also looked at doing
Image[Rescale[data]]
on the raw data. However, this crashes immediately with a SIGSEGV error that has the Wolfram Support team puzzled. Consequently, I haven't gone very far down this road.
Edit:
- I have also tried
ListDensityPlot[data,InterpolationOrder->0]. For the full data set, Mathematica hangs for over 10 minutes, at which point it runs out of memory and the kernel shuts down. For a subset of the data, I get something more reasonable, but I would need some way to scale this up to $500^2$ data points.
Making these plots seem to be something that is fairly easily done in Matplotlib, but I have already made some other plots in Mathematica and don't want to mess with different styles. I'm fairly new to Mathematica and don't have a good knowledge of all the functionality, unfortunately.
So, how can I make a DensityHistogram when the bins and counts have already been calculated?
Comments
Post a Comment