Skip to main content

time series - Export TimeSeries[] to CSV or Excel


I am planning to do a very specific prediction model using some statistical learning techniques in R. I am using some weather data that I obtain using Mathematica. However, I haven't been able to export it properly and every question over here seems either too advanced or too basic to apply to my problem. I have a location (lat, long), say for this example equal to (10,-10) that I need to get the maximum and minimum temperatures per day and the total precipitation as well.


MaxTemp = WeatherData[{10, -10}, "MaxTemperature", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]
MinTemp = WeatherData[{10, -10}, "MinTemperature", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]
TotalPrec = WeatherData[{10, -10}, "TotalPrecipitation", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]

Now that I have these three variables, how can I export them in an easy to use format (.csv, .xslx) for R? Is this straightforward? Ideally I would like a dataset with the three of them, but I have no idea how to produce it.



Thanks!



Answer



I haven't looked at other questions/answers on this site because you stated that what you found didn't help you with your issue at hand; instead I will show you how to treat practically any issue in Mathematica: starting small and building from there


Let's look at something representative of the kind of data you try to deal with


example = WeatherData[{10, -10}, "MaxTemperature", {{2016, 1, 1}, {2016, 1, 31}, "Day"}]

The output of which is


TimeSeriesObject


From the Details and Options-section in the documentation of TimeSeries one can learn that the time series property "DatePath" gives a list of date-value pairs


example["DatePath"]


DatePath


Note the special formatting of the dates as little panels and the brownish font color of the temperatures. The formatting indicates that these are not mere numbers but build-in objects. To export those in a format appropriate for your needs we need to transform them somehow.


Typing "date" and "unit" into documentation search should eventually lead you to DateObject and Quantity and their related functions.


I wrote these two simple function definitions for converting dates and units with a little help from the documentation


format[date_DateObject] := DateValue[date, {"Day", "Month", "Year" }]
format[value_Quantity] := QuantityMagnitude[value]

format


and added another definition for {date, temperature}-pairs



format[{date_DateObject, value_Quantity}] := {format@date, format@value} //Flatten



Assembling the helper functions and methods


The steps above can be packaged in a function like


convertToList[timeSeries_TemporalData] := timeSeries["DatePath"] // Map@format 

that converts TimeSeries-objects into a list of lists of the form {day, month, year, value}


Exporting this is as easy as


Export["example.csv", example //convertToList]




Treatment for missing data


I noticed that the temperature values for some dates are missing; take for example the 7. of August 1982. Luckily TimeSeries has a build-in method for handling missing data called MissingDataMethod.


This can be packaged in another helper function such as


treatMissing[data_TemporalData] := TimeSeries[data, 
MissingDataMethod -> {"Interpolation", InterpolationOrder -> 1}]

The export command then becomes


Export["example.csv", example //treatMissing //convertToList]




Result


We wrote our own little domain specific language for transforming time series into .csv-files. Especially the Postfix-Syntax makes this really shine since you can combine arbitrary complex operations into sentence-like syntax


example //treatMissing  (* //doSomeStuff *) //convertToList (* //doSomeMore *)

Comments

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...

list manipulation - Selecting multiple columns from a matrix?

Sample data: data = { {{2013, 1, 1}, 24.13, 167.67, 231.82}, {{2013, 1, 2}, 32.15, 170.92, 225.99}, {{2013, 1, 3}, 35.43, 172.68, 221.67}, {{2013, 1, 4}, 36.73, 173.05, 218.32}, {{2013, 1, 5}, 58.19, 165.96, 197.05}, {{2013, 1, 6}, 69.99, 163.50, 187.52}, {{2013, 1, 7}, 71.37, 154.21, 175.58}, {{2013, 1, 8}, 72.51, 149.66, 163.25}}; I want a DateListPlot with three graphs, so for a matrix formed by columns 1 and 2, one for columns 1 and 3, and 1 for columns 1 and 4. At the moment I'm using this code: data2 = Transpose[{data[[All, 1]], data[[All, 2]]}]; data3 = Transpose[{data[[All, 1]], data[[All, 3]]}]; data4 = Transpose[{data[[All, 1]], data[[All, 4]]}]; DateListPlot[{data2, data3, data4}, Joined -> True, Filling -> {3 -> {1}}] but I have a hunch that this can be done more efficiently. I don't like the Transpose s in particular. Any ideas? edit (for extra credit) What if I need to multiply the second column by 2, which in my solution is simp...