6

In Python, I can find the Unix time stamp of a local time, knowing the time zone, like this (using pytz):

>>> import datetime as DT >>> import pytz >>> mtl = pytz.timezone('America/Montreal') >>> naive_time3 = DT.datetime.strptime('2013/11/03', '%Y/%m/%d') >>> naive_time3 datetime.datetime(2013, 11, 3, 0, 0) >>> localized_time3 = mtl.localize(naive_time3) >>> localized_time3 datetime.datetime(2013, 11, 3, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>) >>> localized_time3.timestamp() 1383451200.0

So far, so good. naive_time is not aware of the time zone, whereas localized_time knows its midnight on 2013/11/03 in Montréal, so the (UTC) Unix time stamp is good. This time zone is also my local time zone and this time stamp seems right:

$ date -d @1383451200 Sun Nov 3 00:00:00 EDT 2013

Now, clocks were adjusted one hour backward November 3rd at 2:00 here in Montréal, so we gained an extra hour that day. This means that there were, here, 25 hours between 2013/11/03 and 2013/11/04. This shows it:

>>> naive_time4 = DT.datetime.strptime('2013/11/04', '%Y/%m/%d') >>> localized_time4 = mtl.localize(naive_time4) >>> localized_time4 datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EST-1 day, 19:00:00 STD>) >>> (localized_time4.timestamp() - localized_time3.timestamp()) / 3600 25.0

Now, I'm looking for an easy way to get the localized_time4 object from localized_time3, knowing I want to get the next localized day at the same hour (here, midnight). I tried timedelta, but I believe it's not aware of time zones or DST:

>>> localized_time4td = localized_time3 + DT.timedelta(1) >>> localized_time4td datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>) >>> (localized_time4td.timestamp() - localized_time3.timestamp()) / 3600 24.0

My purpose is to get informations about log entries that are stored with their Unix timestamp for each local day. Of course, if I use localized_time3.timestamp() and add 24 * 3600 here (which will be the same as localized_time4td.timestamp()), I will miss all log entries that happened between localized_time4td.timestamp() and localized_time4td.timestamp() + 3600.

In other words, the function or method I'm looking for should know when to add 25 hours, 24 hours or 23 hours sometimes to a Unix time stamp, depending on when DST shifts happen.

Improve this question
6
  • Does it have to be std lib modules or can you use pip modules? Commented Nov 29, 2013 at 0:31
  • I'm willing to use lightweight PyPI modules if they provide a really easy solution. For instance, I'm planning on using pytz. Commented Nov 29, 2013 at 0:37
  • 1
    May I recommend you take a look at the excellent Arrow (crsmithdev.com/arrow). Commented Nov 29, 2013 at 0:38
  • @rdodev: Arrow seems to work great (just tried it)! Do you want to provide an answer (that I will accept) with a working example, or should I answer my own question using Arrow? Thank you anyway. Commented Nov 29, 2013 at 1:00
  • glad to help! Go ahead and answer it yourself. I'm a bit short on time. Commented Nov 29, 2013 at 1:02

4 Answers 4

Reset to default
3

Without using a new package:

def add_day(x): d = x.date()+DT.timedelta(1) return mtl.localize(x.replace(year=d.year, month=d.month, day=d.day, tzinfo=None))

Full script:

import datetime as DT import pytz import calendar mtl = pytz.timezone('America/Montreal') naive_time3 = DT.datetime.strptime('2013/11/03', '%Y/%m/%d') print repr(naive_time3) #datetime.datetime(2013, 11, 3, 0, 0) localized_time3 = mtl.localize(naive_time3) print repr(localized_time3) #datetime.datetime(2013, 11, 3, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>) print calendar.timegm(localized_time3.utctimetuple()) #1383451200.0 def add_day(x): d = x.date()+DT.timedelta(1) return mtl.localize(x.replace(year=d.year, month=d.month, day=d.day, tzinfo=None)) print repr(add_day(localized_time3)) #datetime.datetime(2013, 11, 4, 0, 0, tzinfo=<DstTzInfo 'America/Montreal' EST-1 day, 19:00:00 STD>)

(calendar is for Python2.)

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

I actually like this solution! It's not as trivial as with using Arrow, which really has this feature built-in, but it's still doing the work without external deps. Thanks!
it doesn't handle ambiguous or non-existent time. You could fix it using .dst(), .normalize()` methods
Thanks for pointing this out, J.F. Sebastian. However, in my specific case, I only need to support midnight (i.e. the user only specifies a date, not a date/time). Therefore, ambiguous/non-existent times will never be a problem. It's interesting to know there's also a solution for this in case one needs it, given in your answer to my question, though.
@eepp: Timezone utc offset may change at midnight (it is not common unless you are in Brasilia but it happens). If you want to assert that your code doesn't handle it then use is_dst=None to detect it. To get midnight: datetime.combine(some_date, time(0, 0)) could be used instead of x.replace(year=d.year, ....
2

I gradually provide several solutions with the most robust solution at the very end of this answer that tries to handle the following issues:

  • utc offset due to DST
  • past dates when the local timezone might have had different utc offset due to reason unrelated to DST. dateutil and stdlib solutions fail here on some systems, notably Windows
  • ambiguous times during DST (don't know whether Arrow provides interface to handle it)
  • non-existent times during DST (the same)

To find POSIX timestamp for tomorrow's midnight (or other fixed hour) in a given timezone, you could use code from How do I get the UTC time of “midnight” for a given timezone?:

from datetime import datetime, time, timedelta import pytz DAY = timedelta(1) tz = pytz.timezone('America/Montreal') tomorrow = datetime(2013, 11, 3).date() + DAY midnight = tz.localize(datetime.combine(tomorrow, time(0, 0)), is_dst=None) timestamp = (midnight - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

dt.date() method returns the same naive date for both naive and timezone-aware dt objects.

The explicit formula for timestamp is used to support Python version before Python 3.3. Otherwise .timestamp() method could be used in Python 3.3+.

To avoid ambiguity in parsing input dates during DST transitions that are unavoidable for .localize() method unless you know is_dst parameter, you could use Unix timestamps stored with the dates:

from datetime import datetime, time, timedelta import pytz DAY = timedelta(1) tz = pytz.timezone('America/Montreal') local_dt = datetime.fromtimestamp(timestamp_from_the_log, tz) tomorrow = local_dt.date() + DAY midnight = tz.localize(datetime.combine(tomorrow, time(0, 0)), is_dst=None) timestamp = (midnight - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

To support other fixed hours (not only midnight):

tomorrow = local_dt.replace(tzinfo=None) + DAY # tomorrow, same time dt_plus_day = tz.localize(tomorrow, is_dst=None) timestamp = dt_plus_day.timestamp() # use the explicit formula before Python 3.3

is_dst=None raises an exception if the result date is ambiguous or non-existent. To avoid exception, you could choose the time that is closest to the previous date from yesterday (same DST state i.e., is_dst=local_dt.dst()):

from datetime import datetime, time, timedelta import pytz DAY = timedelta(1) tz = pytz.timezone('America/Montreal') local_dt = datetime.fromtimestamp(timestamp_from_the_log, tz) tomorrow = local_dt.replace(tzinfo=None) + DAY dt_plus_day = tz.localize(tomorrow, is_dst=local_dt.dst()) dt_plus_day = tz.normalize(dt_plus_day) # to detect non-existent times timestamp = (dt_plus_day - datetime(1970, 1, 1, tzinfo=pytz.utc)).total_seconds()

.localize() respects given time even if it is non-existent, therefore .normalize() is required to fix the time. You could raise an exception here if normalize() method changes its input (non-existent time detected in this case) for consistency with other code examples.

Improve this answer

4 Comments

Why would you do is_dst=local_dt.dst() to ensure the "time that is closest to the previous date from yesterday"? The is_dst parameter of localize accepts True, False or None, while the dst() method of a datetime.datetime returns a datetime.timedelta. Thus, mtl.localize(DT.datetime(2013, 11, 4)).dst() == True is False, and so is mtl.localize(DT.datetime(2013, 11, 4)).dst() == True, whereas the first has DST and the second does not.
@eepp: Avoid some_python == True, try bool(some_python_obj) instead. Read Truth value testing in Python
Good to know! I'm fairly new to Python, but getting better thanks to guys like you. Here's what the doc says: and in Boolean contexts, a timedelta object is considered to be true if and only if it isn’t equal to timedelta(0).. This makes sense now. You really shed light on something however, thank you.
@eepp: just to be clear: you rarely need to call bool() explicitly, if some_python_obj: ... works as is.
1

(Thanks to @rdodev for pointing me to Arrow).

Using Arrow, this operation becomes easy:

>>> import arrow >>> import datetime as DT >>> lt3 = arrow.get(DT.datetime(2013, 11, 3), 'America/Montreal') >>> lt3 <Arrow [2013-11-03T00:00:00-04:00]> >>> lt4 = arrow.get(DT.datetime(2013, 11, 4), 'America/Montreal') >>> lt4 <Arrow [2013-11-04T00:00:00-05:00]> >>> lt4.timestamp - (lt3.replace(days=1).timestamp) 0 >>> (lt3.replace(days=1).timestamp - lt3.timestamp) / 3600 25.0

Using Arrow's replace method, singular unit names replace that property while plural adds to it. So lt3.replace(days=1) is November 4th, 2013 while lt3.replace(day=1) is November 1st, 2013.

Improve this answer

3 Comments

How does Arrow deal with ambiguous or non-existent times?
Well, November 3rd, 2013 at 1:30 in Montréal seems to default to no DST because it outputs <Arrow [2013-11-03T01:30:00-05:00]> (while 0:59 has DST: <Arrow [2013-11-03T00:59:00-04:00]>).
what about non-existent time such as 2014-03-09 02:45:00?
1

Here an alternative based on dateutil:

>>> # In Spain we changed DST 10/26/2013 >>> import datetime >>> import dateutil.tz >>> # tzlocal gets the timezone of the computer >>> dt1 = datetime.datetime(2013, 10, 26, 14, 00).replace(tzinfo=dateutil.tz.tzlocal()) >>> print dt1 2013-10-26 14:00:00+02:00 >>> dt2 = dt1 + datetime.timedelta(1) >>> print dt2 2013-10-27 14:00:00+01:00 # see if we hace 25 hours of difference >>> import time >>> (time.mktime(dt2.timetuple()) - time.mktime(dt1.timetuple())) / 3600.0 25.0 >>> (float(dt2.strftime('%s')) - float(dt1.strftime('%s'))) / 3600 # the same 25.0
Improve this answer

3 Comments

dateutil might fail during DST transitions. It might fail for past dates if OS' historical timezone database is not used or absent (Windows). It doesn't handle ambiguous, non-existent times.
I tested DST transitions with my timezone (CEST -> CET) as the example demonstrates. I'm using it for recent and future dates only. Thanks for the tip.
timezone info changes all the time. There are recent changes in 2013. I'm sure the changes won't stop in the future.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.