Skip to main content

output formatting - How to export data files using specific number format


Very often, I have to export data files from Mathematica. These output files contain several columns and hundreds of lines with data. For the sake of the question let's create a sample list:


data = Table[{RandomReal[i], RandomReal[i^10]}, {i, 0, 100, 0.1}];

Then, I export the data to an external file using the following command:


Export["data.out", data, "Table"];


Now let's take a look at the output file. I present here only the first ten lines of data, but of course the whole file is easily reproducible:


0.  0.
0.01599700768300292 9.144414172338937e-12
0.05264806593747054 5.191535295682516e-8
0.26762136311267287 2.6644991730537336e-6
0.01121545335019451 0.000014373686962031977
0.05282872336357303 0.0005555775821525191
0.39492299878320175 0.0005639987736085206
0.6483050598753638 0.016687867077835535

0.5871008451356092 0.029518547448633307
0.2613882661983659 0.046494918800056996
.................. ....................

My problem is that the format of the data is very bad. When I create external output files in FORTRAN I can control the exact format. So, I am wondering: Is this possible in Mathematica? Can I control the number of decimal digits of the data, using exponential form or even the gap between the two columns? In FORTRAN, I would use this format 2(4X,E16.8), meaning that the data would be in exponential form (E), having total 16 digits in total from which the 8 are decimal and finally the gap between the two columns is 4 blank characters.


EDIT:


Following cormullion's instructions the exported data files look like this:


0.0000000000000000E     0.0000000000000000E
0.0292206329521133E 3.5660981240075680E-12
0.0110286543632250E 7.1403083420409010E-8

0.1512525645232069E 3.4764180261918010E-8
0.1802104544376126E 0.0000150833977857E
0.1909866023262902E 0.0004442874170494E
0.1977662892883431E 0.0047809317792338E
0.1043698170574586E 0.0116230093104291E
0.2452055184319113E 0.0756524674940767E
0.4203148306922685E 0.0682570346613104E
0.4640953468992068E 0.6204923184434734E
0.4361142873366830E 0.8423302837693910E
0.3853270077185619E 1.9278084597534120E

0.9173409732335570E 1.9391384326481460E
0.2309798126928349E 6.9819944783354480E
0.9064707972602210E 55.0683061788128700E
1.1871007725662080E 47.5205793705051100E
0.0657321188104301E 165.0478398165293000E
0.6501851799068231E 261.9422532326231000E
0.1976942785179729E 83.6983691851117000E
0.1781317179022484E 877.1531182356240000E
0.9603071406080800E 610.9760718811238000E
1.2678697679679910E 2107.5934446883950000E

1.2555519112242100E 695.2293658225135000E
1.4503489860688380E 6300.5931889760220000E
2.3698084216652830E 9336.5466490215000000E
0.9495735185410430E 8100.1159435400300000E
2.3382755917778770E 18471.7912516294500000E
1.3595555295319900E 27021.6788394801700000E
1.1450366450215240E 20223.3331979412600000E
0.6496852979125343E 14269.5720554042100000E
1.4074545584871190E 22413.5903375179000000E
1.8790499719570280E 85437.5672819188000000E

3.0597774075552670E 37024.4278765415400000E
3.2694586701530210E 99567.9453873553000000E
2.7064643220362330E 226071.8308196532000000E
................... ........................

As everyone can observe, there are some inconsistencies regarding the format


(i). The exponential part (E) should always have the same pattern; two digits (i.e. E-08 not just E-8) and when for zero it should be E+00


(ii). In some cases, the second column contains some additional (more than 16) decimal digits (in fact they are unwanted zeros).


I filtered this data file using a very simple FORTRAN code and here is the corresponding output:


0.0000000000000000E+00    0.0000000000000000E+00

0.2922063295211330E-01 0.3566098124007568E-11
0.1102865436322500E-01 0.7140308342040901E-07
0.1512525645232069E+00 0.3476418026191801E-07
0.1802104544376126E+00 0.1508339778570000E-04
0.1909866023262902E+00 0.4442874170494000E-03
0.1977662892883431E+00 0.4780931779233800E-02
0.1043698170574586E+00 0.1162300931042910E-01
0.2452055184319113E+00 0.7565246749407670E-01
0.4203148306922685E+00 0.6825703466131040E-01
0.4640953468992068E+00 0.6204923184434734E+00

0.4361142873366830E+00 0.8423302837693910E+00
0.3853270077185619E+00 0.1927808459753412E+01
0.9173409732335570E+00 0.1939138432648146E+01
0.2309798126928349E+00 0.6981994478335448E+01
0.9064707972602210E+00 0.5506830617881287E+02
0.1187100772566208E+01 0.4752057937050511E+02
0.6573211881043010E-01 0.1650478398165293E+03
0.6501851799068231E+00 0.2619422532326231E+03
0.1976942785179729E+00 0.8369836918511170E+02
0.1781317179022484E+00 0.8771531182356240E+03

0.9603071406080800E+00 0.6109760718811237E+03
0.1267869767967991E+01 0.2107593444688395E+04
0.1255551911224210E+01 0.6952293658225135E+03
0.1450348986068838E+01 0.6300593188976022E+04
0.2369808421665283E+01 0.9336546649021500E+04
...................... ......................

Now, everything is OK! Well, as Xerxes pointed out, why care what the file looks like as long as the next computer program knows how to parse it? The answer is very simple. Because scientists love neatness and order in their work and they hate sloppy jobs! Moreover, I refuse to believe, that a sophisticated and modern software like Mathematica cannot export data using predefined format.



Answer



This is a variant on Andre's solution (worked out before I noticed!). Anyway this takes the FORTRAN descriptor values as arguments.



f77Eform[x_?NumericQ, fw_Integer, ndig_Integer] := Module[{sig, s, p, ps},
{s, p} = MantissaExponent[x];
{sig, ps} = {ToString[Round[10^ndig Abs[s]]], ToString[Abs[p]]};
StringJoin @@ Join[
Table[" ", {fw - ndig - 7}],
If[x < 0, "-", " "], {"0."}, {sig},
Table["0", {ndig - StringLength[sig]}], {"E"},
If[p < 0, {"-"}, {"+"}],
Table["0", {2 - StringLength[ps]}], {ps}]]


f77Eform[Pi, 16, 8] -> " 0.31415927E+01"
f77Eform[-Sqrt[2], 16, 8] -> " -0.14142136E+01"
f77Eform[10^-4, 16, 8] -> " 0.10000000E-03"
f77Eform[-10^6, 16, 8] -> " -0.10000000E+07"

sanity check:


Max[Table[(Abs[{Read[StringToStream[f77Eform[#, 22, 14]], Number]/# - 
1}]) &[RandomReal[{-1, 1}] 10^RandomReal[{-5, 5}]] , {1000}]] -> 4.64073*10^-14

I must say as someone who works with Fortran quite a lot I have never needed to do this. Fortran can perfectly well read more generally formatted data. Unless you are stuck with legacy Fortran code that uses fixed field formatting for input don't bother.



Comments

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...

equation solving - Invert and fit implicitly defined curve

I need to fit an implicitly defined curve. I thought I could get some data out of Solve , and then using FindFit . Therefore, I would like to find the relation the parametric curve defined by $F(x,y)=0$: Solve[-(1/2) + 1/2 (0.41202 BesselK[0, 0.1 Sqrt[x^2 + y^2]] + (0.101483 x BesselK[1, 0.1 Sqrt[x^2 + y^2]])/Sqrt[x^2 + y^2]) == 0, y] But I can't get an output: Solve was unable to solve the system with inexact coefficients or the system obtained by direct rationalization of inexact numbers present in the system. Since many of the methods used by Solve require exact input, providing Solve with an exact version of the system may help. >> Edit: In particular, I would like to fit the data coming from the curve with the expression of another curve, and not with a function $f(x)$. In particular, since this clearly looks like a cardioid , I would like it to fit to something like it. What other strategies could I try?

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...