Skip to main content

time series - Export TimeSeries[] to CSV or Excel


I am planning to do a very specific prediction model using some statistical learning techniques in R. I am using some weather data that I obtain using Mathematica. However, I haven't been able to export it properly and every question over here seems either too advanced or too basic to apply to my problem. I have a location (lat, long), say for this example equal to (10,-10) that I need to get the maximum and minimum temperatures per day and the total precipitation as well.


MaxTemp = WeatherData[{10, -10}, "MaxTemperature", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]
MinTemp = WeatherData[{10, -10}, "MinTemperature", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]
TotalPrec = WeatherData[{10, -10}, "TotalPrecipitation", {{1990, 1, 1}, {2016, 12, 31}, "Day"}]

Now that I have these three variables, how can I export them in an easy to use format (.csv, .xslx) for R? Is this straightforward? Ideally I would like a dataset with the three of them, but I have no idea how to produce it.



Thanks!



Answer



I haven't looked at other questions/answers on this site because you stated that what you found didn't help you with your issue at hand; instead I will show you how to treat practically any issue in Mathematica: starting small and building from there


Let's look at something representative of the kind of data you try to deal with


example = WeatherData[{10, -10}, "MaxTemperature", {{2016, 1, 1}, {2016, 1, 31}, "Day"}]

The output of which is


TimeSeriesObject


From the Details and Options-section in the documentation of TimeSeries one can learn that the time series property "DatePath" gives a list of date-value pairs


example["DatePath"]


DatePath


Note the special formatting of the dates as little panels and the brownish font color of the temperatures. The formatting indicates that these are not mere numbers but build-in objects. To export those in a format appropriate for your needs we need to transform them somehow.


Typing "date" and "unit" into documentation search should eventually lead you to DateObject and Quantity and their related functions.


I wrote these two simple function definitions for converting dates and units with a little help from the documentation


format[date_DateObject] := DateValue[date, {"Day", "Month", "Year" }]
format[value_Quantity] := QuantityMagnitude[value]

format


and added another definition for {date, temperature}-pairs



format[{date_DateObject, value_Quantity}] := {format@date, format@value} //Flatten



Assembling the helper functions and methods


The steps above can be packaged in a function like


convertToList[timeSeries_TemporalData] := timeSeries["DatePath"] // Map@format 

that converts TimeSeries-objects into a list of lists of the form {day, month, year, value}


Exporting this is as easy as


Export["example.csv", example //convertToList]




Treatment for missing data


I noticed that the temperature values for some dates are missing; take for example the 7. of August 1982. Luckily TimeSeries has a build-in method for handling missing data called MissingDataMethod.


This can be packaged in another helper function such as


treatMissing[data_TemporalData] := TimeSeries[data, 
MissingDataMethod -> {"Interpolation", InterpolationOrder -> 1}]

The export command then becomes


Export["example.csv", example //treatMissing //convertToList]




Result


We wrote our own little domain specific language for transforming time series into .csv-files. Especially the Postfix-Syntax makes this really shine since you can combine arbitrary complex operations into sentence-like syntax


example //treatMissing  (* //doSomeStuff *) //convertToList (* //doSomeMore *)

Comments

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...

equation solving - Invert and fit implicitly defined curve

I need to fit an implicitly defined curve. I thought I could get some data out of Solve , and then using FindFit . Therefore, I would like to find the relation the parametric curve defined by $F(x,y)=0$: Solve[-(1/2) + 1/2 (0.41202 BesselK[0, 0.1 Sqrt[x^2 + y^2]] + (0.101483 x BesselK[1, 0.1 Sqrt[x^2 + y^2]])/Sqrt[x^2 + y^2]) == 0, y] But I can't get an output: Solve was unable to solve the system with inexact coefficients or the system obtained by direct rationalization of inexact numbers present in the system. Since many of the methods used by Solve require exact input, providing Solve with an exact version of the system may help. >> Edit: In particular, I would like to fit the data coming from the curve with the expression of another curve, and not with a function $f(x)$. In particular, since this clearly looks like a cardioid , I would like it to fit to something like it. What other strategies could I try?

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...