58

I am receiving twitter messages that are sent at a certain date in the following format from twitter:

Tue Mar 29 08:11:25 +0000 2011

I want to store these dates in 'timestamp with time zone' field in postgresql with djangos DateTimeField field. When I store that string however I get this error:

ValidationError: [u'Enter a valid date/time in YYYY-MM-DD HH:MM[:ss[.uuuuuu]] format.']

can I automatically transform the twitter datetype to a python datetime time (that does work elsewhere in my app for saving dates).

Improve this question

10 Answers 10

Reset to default
99

Writing something like this should convert a twitter date to a timestamp.

import time ts = time.strftime('%Y-%m-%d %H:%M:%S', time.strptime(tweet['created_at'],'%a %b %d %H:%M:%S +0000 %Y'))

UPDATE

For Python 3, as per 2020, you can do it in this way:

from datetime import datetime # dtime = tweet['created_at'] dtime = 'Fri Oct 09 10:01:41 +0000 2015' new_datetime = datetime.strftime(datetime.strptime(dtime,'%a %b %d %H:%M:%S +0000 %Y'), '%Y-%m-%d %H:%M:%S') print((new_datetime))
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

This gives me the following error, 'expected string or buffer'. Can you help me why?
@ZairaZafar I added an answer below which may help. I had similar errors that you had.
15

Give this a go. It assumes the date format from twitter is RFC822 compliant (see the question linked to by @Adrien).

A naive datetime object is constructed (i.e. no timezone info). It is adjusted according to the timezone offset to UTC. Unless you have a need to keep the original timezone, I'd store the date time as UTC and format to local time when you display it.

from datetime import datetime, timedelta from email.utils import parsedate_tz s = 'Tue Mar 29 08:11:25 +0000 2011' def to_datetime(datestring): time_tuple = parsedate_tz(datestring.strip()) dt = datetime(*time_tuple[:6]) return dt - timedelta(seconds=time_tuple[-1])
Improve this answer

1 Comment

I prefer this method because it even works with timezones different from +0000 (despite the fact that Twitter always uses +0000). Django side is also better to make the resulting datetime timezone-aware using the utc timezone ( see this question )
8

A little bit old but using parse really help me with this issue

from datetime import datetime from dateutil.parser import parse date = 'Fri May 10 00:44:04 +0000 2019' dt = parse(date) print(dt) # 2019-05-10 00:44:04+00:00
Improve this answer

2 Comments

This seems the best answer in 2019. Any downsides to doing it this way?
Nope. This is STILL the most pythonic approach in 2021. Use a common parser.
3

To get datetime with timezone you can simple use datetime.strptime as follow:

from datetime import datetime s = 'Wed Jun 05 05:34:02 +0000 2019' created_at = datetime.strptime(s, '%a %b %d %H:%M:%S %z %Y') print(created_at) #2019-06-05 05:34:02+00:00
Improve this answer

Comments

3

Twitter API V2 sends date strings that look like this:

2020-12-15T20:17:10.000Z

This worked, to convert from string to datetime:

datetime.datetime.strptime(THE_STRING,"%Y-%m-%dT%H:%M:%S.%fZ")

The end looks like a timezone, but it's milliseconds, hence the %f. The final character, "Z" is a timezone code that means UTC, as explained, here.

Improve this answer

Comments

2

you can convert the date using datetime.strptime(), or time.strptime(). however, those two functions cannot parse the timezone offset (see this bug).

so, the only solution i see is to split the date yourself, remove the timezone offset, feed the rest to strptime(), and process the offset manually...

have a look at this question, where you will find some hints on how to parse the offset yourself.

Improve this answer

Comments

1

The following code will print a nice date (local time) from a Twitter date (UTC).

from datetime import datetime from datetime import timezone datetime.strptime(mydata["created_at"], '%a %b %d %H:%M:%S %z %Y').replace( tzinfo=timezone.utc).astimezone(tz=None).strftime('%Y-%m-%d %H:%M:%S'))
Improve this answer

Comments

1

The initial problem I was having was converting from the datetime that the twitter api gives to String.

The following works which addresses different comments people seem to have for above solutions which are a little unclear as to whether the starting date is already in string format or not. This works for Python 2.7

With a tweet from the API, tweet.created_at gives the date in datetime format. At the top of your file, add from datetime import datetime

then use the following to get the corresponding string. 

datetime.strftime(tweet.created_at,'%a %b %d %H:%M:%S %z %Y').

You can then use this string as described in other comments to manipulate it.

Improve this answer

Comments

0

Using a similar strategy as SoFolichon proposed, in Python 3.x you can also use pytz like:

from datetime import datetime, timezone import pytz datetime.strptime(tweets["created_at"], '%a %b %d %H:%M:%S %z %Y').replace( tzinfo=timezone.utc).astimezone(pytz.timezone('US/Eastern')).strftime( '%Y-%m-%d %H:%M:%S')
Improve this answer

3 Comments

Hey. this gives me the following error: strptime() argument 1 must be string, not Series.
@ZairaZafar, Which version of Python do you use? In Python 3.x, it works fine for me.
im using 2.7 python
0

How about this? It doesn't need any formatting strings.

import datetime from email.utils import mktime_tz, parsedate_tz def parse_datetime(value): time_tuple = parsedate_tz(value) timestamp = mktime_tz(time_tuple) return datetime.datetime.fromtimestamp(timestamp) print(parse_datetime('Tue Mar 29 08:11:25 +0000 2011')) #2011-03-29 10:11:25

My system is at GMT +2 hence the difference included.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.