4
$\begingroup$

I'm trying to import CSV file with the following format:

20150803 00:00:01.649,1.09675,1.09678

using this code:

Import[filename, "DateStringFormat" -> {"Year", "Month", "Day", " ", "Hour", ":", "Minute", ":", "Second", ".", "Millisecond"}]

But this does not import Time part - it only imports the Date:

{{DateObject[{2015, 8, 4}], 1.09675, 1.09682}, {DateObject[{2015, 8, 3}], 1.09675, 1.09678}, ...

How can I import all of the information from my file?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
2
$\begingroup$

Import does not support the time portion of DateStringFormat. (This is a deficiency in the docs if not a bug ) If you know the specific column(s) where your date appears you can map datelist like this:

MapAt[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", "Minute", ":", "Second", ".", "Millisecond"}}] & , Import[filename.csv], {All, 1}]

In case you do not know where the strings might occur you can do this:

Map[Quiet@ Check[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", "Minute", ":", "Second", ".", "Millisecond"}}], #] & , Import[filename.csv], {-1}]

I would guess this will really kill performance on a large file though.

Improve this answer
$\endgroup$
0
$\begingroup$

I found solution at this answer:

And also improved loading speed of my 250Mb CSV file

Improve this answer
$\endgroup$
3
  • $\begingroup$ 250 Gb, would that mean Giga-Bit or was it a type and would read like GB (Giga-Byte) or are it rather Mega-Bytes really? I would wonder on what machine Mathematica would be able to handle a 250 Gb (30GB ?) CSV file :-) $\endgroup$ Commented Sep 17, 2015 at 14:31
  • $\begingroup$ Oh, I'm mistaked - my CSV file 250Mb, not 250Gb :) $\endgroup$ Commented Sep 19, 2015 at 11:23
  • $\begingroup$ OK, that sounds somewhat less demanding :-) $\endgroup$ Commented Sep 19, 2015 at 13:11
0
$\begingroup$ 
in = StringToStream[StringJoin[ "20150803 00:00:01.649,1.09675,1.09678\r\n", "20170930 23:59:59.649,1.19675,1.29678\r\n"]] file = (StringSplit[#1, ","] & ) /@ StringSplit[ReadString[in], RegularExpression[ "[\r\n]+"]] Close[in] getNumber = Interpreter["Number"] data = ({FromDateString[#1[[1]]], getNumber[#1[[2]]], getNumber[ #1[[3]]]} & ) /@ file 
InputStream[Name: String Unique ID: 3 ] {{20150803 00:00:01.649,1.09675,1.09678},{20170930 23:59:59.649,1.19675,1.29678}} String Interpreter[Number] {{Mon 3 Aug 2015 00:00:01GMT-4,1.09675,1.09678},{Sat 30 Sep 2017 23:59:59GMT-4,1.19675,1.29678}}

The first cell would be replaced normally by an OpenRead to the CSV file.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.