datetime module has issue with year 2038 problem

[gregwa1953]

When the time.gmtime() reaches January 19, 2038 03:14:08 UTC, a program using datetime.now() will crash with an

OverflowError: overflow converting long int to machine word

The line in question is in datetime.py line 638 in fromtimestamp

Here is a small program demonstrating the issue...

# ==================================
#           test2038.py
# ----------------------------------
# Written by G.D. Walters
# ==================================

import machine
import utime
import time
from time import sleep, localtime, gmtime
import sys
import gc

from datetime import MAXYEAR, MINYEAR, datetime, date, timedelta, timezone, tzinfo
    
# Simulate datetime to be January 19, 2038 03:14:00 UTC
def settime2038():
    tm=utime.gmtime(timelong)
    machine.RTC().datetime((tm[0], tm[1], tm[2], tm[6] + 1, tm[3], tm[4], tm[5], 0))
                    
# Return a local time based using datetime module datetime.datetime.now(timezone)                    
def showMyTime():
    import datetime
    my_timezone=timezone(timedelta(hours=-5))
    current_time = datetime.datetime.now(my_timezone)
    return current_time
    
timelong=2147483639  # January 19, 2038 03:14:00 UTC

# Do a garbage collect
gc.collect()
# Set the machine.RTC to Jan 19, 2038 03:14:00 UTC
settime2038()
print(f"{gmtime()=} - {showMyTime()=}")
#print(showMyTime())

while True:

    print(f"{gmtime()=} - {showMyTime()=}")
    #print(showMyTime())    
    time.sleep(1)

And the REPL output is ...

%Run -c $EDITOR_CONTENT

MPY: soft reboot
gmtime()=(2038, 1, 19, 3, 14, 0, 1, 19) - showMyTime()=2038-01-18 22:14:00-05:00
gmtime()=(2038, 1, 19, 3, 14, 0, 1, 19) - showMyTime()=2038-01-18 22:14:00-05:00
gmtime()=(2038, 1, 19, 3, 14, 1, 1, 19) - showMyTime()=2038-01-18 22:14:01-05:00
gmtime()=(2038, 1, 19, 3, 14, 2, 1, 19) - showMyTime()=2038-01-18 22:14:02-05:00
gmtime()=(2038, 1, 19, 3, 14, 3, 1, 19) - showMyTime()=2038-01-18 22:14:03-05:00
gmtime()=(2038, 1, 19, 3, 14, 4, 1, 19) - showMyTime()=2038-01-18 22:14:04-05:00
gmtime()=(2038, 1, 19, 3, 14, 5, 1, 19) - showMyTime()=2038-01-18 22:14:05-05:00
gmtime()=(2038, 1, 19, 3, 14, 6, 1, 19) - showMyTime()=2038-01-18 22:14:06-05:00
gmtime()=(2038, 1, 19, 3, 14, 7, 1, 19) - showMyTime()=2038-01-18 22:14:07-05:00
Traceback (most recent call last):
File "", line 39, in
File "", line 25, in showMyTime
File "datetime.py", line 644, in now
File "datetime.py", line 638, in fromtimestamp
OverflowError: overflow converting long int to machine word

This was tested on a RPi Pico-W running MicroPython v1.23.0-preview.322.g5114f2c1e and using Thonny as the IDE.
test2038.zip

[ekspla]

Looking at the last line number in the error trace, it is caused by the time.gmtime() in the time module.

            dt = cls(*_tmod.gmtime(ts)[:6], microsecond=us, tzinfo=tz)

The simplest code to reproduce this is as follows:

>>> import time
>>> time.gmtime(2147483647)
(2068, 1, 19, 3, 14, 7, 3, 19)
>>> time.gmtime(2147483648)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: overflow converting long int to machine word
>>>

[ekspla]

Additional comment:

I am not sure if this is an issue in datetime module though. This is because 32-bit timestamp and rollover in time module would be solved before 2038 (or 2068, depending on the epoch).

It seems that we have at least three choices of datetime module: adafruit's circuitpython, pycopy-lib and micropython-lib. However, all of them seems to use time module when converting from timestamp to tuple. A workaround would be to write the converter, which does not seems to be worth the effort.