Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date display wrong, homepage timeline is using UTC timezone however the photo time is correct in info page #10417

Closed
2 of 3 tasks
MFYDev opened this issue Jun 17, 2024 · 39 comments

Comments

@MFYDev
Copy link

MFYDev commented Jun 17, 2024

The bug

Hello, just upgraded to the latest v1.106.4, I just uploaded a photo which was taken on GMT-4 on June 16 however on the homepage this picture is displaying under tomorrow's date which is Jun 17, in the photo info page it is correct

The OS that Immich Server is running on

ubuntu 22.04

Version of Immich Server

1.106.4

Version of Immich Mobile App

1.106.4

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    ports:
      - 2283:3001
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  backup:
    container_name: immich_db_dumper
    image: prodrigestivill/postgres-backup-local:14
    env_file:
      - .env
    environment:
      POSTGRES_HOST: database
      POSTGRES_CLUSTER: 'TRUE'
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      SCHEDULE: "0 */4 * * *"
      POSTGRES_EXTRA_OPTS: '--clean --if-exists'
      BACKUP_DIR: /db_dumps
      BACKUP_KEEP_DAYS: 30
    volumes:
      - ./db_dumps:/db_dumps
    depends_on:
      - database

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=****

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=sjahvysifdgnwer87sjdhgkja
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=****
DB_USERNAME=postgres
DB_DATABASE_NAME=****

REDIS_HOSTNAME=immich_redis

PUBLIC_LOGIN_PAGE_MESSAGE="****"

Reproduction steps

1.Upgrade to the latest version
2.My timezone is America/New_York, upload an image taken around 10pm
3.The image is displaying under tomorrow's date
...

Relevant log output

No response

Additional information

No response

@MFYDev
Copy link
Author

MFYDev commented Jun 17, 2024

image
photo info is correct
However it is in tomorrow's date
image

@bo0tzz
Copy link
Member

bo0tzz commented Jun 17, 2024

Can you try a force refresh/clearing your browser cache?

@MFYDev
Copy link
Author

MFYDev commented Jun 17, 2024

Can you try a force refresh/clearing your browser cache?

Hi, thank you for the response, I tried both incognito and normal model with cache cleared, still the same which is really weird

@snuq
Copy link

snuq commented Jun 17, 2024

Just installed Immich and running into the same problem (images are displayed in different 'day folder' than they should be).

In my case tho, the displayed dates are weird as well - for instance, one photo shows "Nov 23, 2023 Thu, 7:18 AM GMT-08:00", the photo was taken at 3:18 pm PST, and this is the time that the file modified date displays.
The photo's original date and time exif data shows: 2023:11:23 15:18:57
so... it thinks that the exif/modified date is gmt and its subtracting 8 hours from that to get what it thinks is my time?

Also, some files that are in the wrong 'date folder' are showing in UTC timezone when i view the info. These files appear to be mostly videos, and dont have any exif data. like the other files, the date and time are displayed correctly when viewing the files in a file explorer.

@snuq
Copy link

snuq commented Jul 9, 2024

First off, my timezone is PDT, all photos were taken in that timezone, and the server is in that timezone. GPS was disabled on my phone for all the photos and videos that were taken (i only enable it when im needing navigation).

First example:
I have an image dated Nov 23, 2023, 4:09 PM GMT-08:00, it shows this correctly in the info pane.
In the photo timeline, this image is placed in Nov 24, 2023.
Another image taken on same device from the same day - Nov 23, 2023 at 3:52 PM GMT-08:00 is in the correct day on the timeline.

Second example:
Even worse are the videos, I have one actually taken on December 30th, 2023 at 8:47PM (as indicated by the date modified in the file) which shows in the timeline under December 31st, it thinks was taken at "Dec 31, 2023, 4:47 AM UTC" in the photo info.
In the Immich android app, this shows up in the correct date in the timeline, but when I check the image date it shows the (incorrect) decmber 31st at 4:47am.

One of the things I tried to make this work better was setting the 'TZ' environment variable to my timezone... but somehow this makes it WORSE in some ways... it seems to have corrected the time display of images in the info pane (tho videos are still incorrect), but those images are now displayed in the wrong date on the timeline when they were correctly displayed before...

I also ran into another bug that appears related - when I tried to back up some photos from the android app, they ended up in folders with the incorrect date (in the same way as the timeline shows the incorrect date).
UPDATE: After changing the timezone on my server with the TZ environmental variable, the android app backup folders are correct.

@chodthewacko
Copy link

just encountered this as well. 1.107.2
Like snuq, I am in new york. many ipad videos are showing, on the web page, in UTC, causing any photos taken after 8pm to appear on the next day in the timeline.

On the android app, the videos appear in EST.
What really threw me off is that the arrow keys/cursor keys on the keyboard follow "EST order".
So if you have pic1, pic2, vid3, pic4, vid5 in chronological order,
on the web page you see:
July 5:
vid3 -> vid5
July 4:
pic1, pic2, pic4

which would make you think, if you hit 'right' when viewing vid3, you should go to vid5.
You don't - it goes to pic4.

I originally hit right arrow 5 times expecing to go to the picture 5 "spaces" to the right, and it didn't.

@MFYDev
Copy link
Author

MFYDev commented Aug 15, 2024

same issue is still happening now even if I set the TZ in the .env in the latest version today

@Thinkscape
Copy link

Thinkscape commented Aug 15, 2024

Example

  • My system time (and browser time) is AEST time zone 16 Aug 2024, 9:52 GMT+10:00.
  • The photo is 15 Aug 2024 Thu, 09:12 GMT+10:00
  • All immich services are running with TZ=Australia/Sydney (which is GMT+10:00)
  • Immich is showing the photo under Today

Metadata

Metadata for a file with yesterday's date
{
    "id": "0160009b-a8f9-42a9-b56b-9d23ed00be82",
    "deviceAssetId": "web-Journal Display Aug 15 2024 (9).jpg-1723763136464",
    "ownerId": "ef0c6f2b-87bc-4be0-b236-bb885652168b",
    "deviceId": "WEB",
    "libraryId": null,
    "type": "IMAGE",
    "originalPath": "upload/library/admin/2024/2024-08-15/Journal Display Aug 15 2024 (9).jpg",
    "originalFileName": "Journal Display Aug 15 2024 (9).jpg",
    "originalMimeType": "image/jpeg",
    "resized": true,
    "thumbhash": "5vcJDQJ/Z12Hd5eVdYVnqa+z5QgL",
    "fileCreatedAt": "2024-08-14T23:12:00.000Z",
    "fileModifiedAt": "2024-08-15T23:05:36.464Z",
    "localDateTime": "2024-08-15T09:12:00.000Z",
    "updatedAt": "2024-08-15T23:12:57.372Z",
    "isFavorite": false,
    "isArchived": false,
    "isTrashed": false,
    "duration": "0:00:00.00000",
    "exifInfo": {
        "make": null,
        "model": null,
        "exifImageWidth": 768,
        "exifImageHeight": 1024,
        "fileSizeInByte": 99624,
        "orientation": null,
        "dateTimeOriginal": "2024-08-14T23:12:00.000Z",
        "modifyDate": "2024-08-15T23:05:36.464Z",
        "timeZone": "UTC+10",
        "lensModel": null,
        "fNumber": null,
        "focalLength": null,
        "iso": null,
        "exposureTime": null,
        "latitude": null,
        "longitude": null,
        "city": null,
        "state": null,
        "country": null,
        "description": "",
        "projectionType": null,
        "rating": null
    },
    "livePhotoVideoId": null,
    "people": [],
    "checksum": "7Q7PzOJuuzXsDgx3Pc9pQSdiG4k=",
    "stackCount": null,
    "isOffline": false,
    "hasMetadata": true,
    "duplicateId": null
}

How the file shows up in the UI

Screenshot 2024-08-16 at 9 38 40 AM Screenshot 2024-08-16 at 9 38 46 AM

Browser locale

new Date()
// Fri Aug 16 2024 09:52:47 GMT+1000 (Australian Eastern Standard Time)

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

I am experiencing this as well.

Version info

Immich
v1.112.1

ExifTool
12.91

Node.js
v20.16.0

Libvips
8.15.2

ImageMagick
7.1.1-24

FFmpeg
6.0.1-6

Repository
immich-app/immich

Source
v1.112.1@f7bfde6a3

Build
10396590839

Build Image
v1.112.1

The below screenshot was taken at August 17, 2024 at 8:33 PM Pacific Time. All of these photos were taken/uploaded on the same day in Pacific Time.

image

Browser date: Sat Aug 17 2024 20:33:34 GMT-0700 (Pacific Daylight Time)

TZ=America/Los_Angeles is set on all containers.

Metadata for the first image in "Yesterday"
{
    "id": "0e6de382-b684-42b1-9411-fda5fa3f8265",
    "deviceAssetId": "E5EDE398-69F4-4FAA-94D8-5541E131328A/L0/001",
    "ownerId": "c7c204f4-5a70-4074-aed6-6a09ef09f9a6",
    "owner": {
        "id": "c7c204f4-5a70-4074-aed6-6a09ef09f9a6",
        "email": "", // redacted
        "name": "TJ Horner",
        "profileImagePath": "",
        "avatarColor": "green"
    },
    "deviceId": "85a4bcaa692a512c691f95968385f81e97c9c824305cccfeabe6b740d5e0bdb2",
    "libraryId": null,
    "type": "IMAGE",
    "originalPath": "upload/upload/c7c204f4-5a70-4074-aed6-6a09ef09f9a6/f9/69/f9699162-4000-4abe-8ec5-136038a16725.HEIC",
    "originalFileName": "IMG_9848.HEIC",
    "originalMimeType": "image/heic",
    "resized": true,
    "thumbhash": "UQgKDQBFao9oGbhmhoiIqCzPovM8",
    "fileCreatedAt": "2024-08-18T03:02:29.732Z",
    "fileModifiedAt": "2024-08-18T03:02:29.000Z",
    "localDateTime": "2024-08-17T20:02:29.732Z",
    "updatedAt": "2024-08-18T03:16:20.266Z",
    "isFavorite": false,
    "isArchived": false,
    "isTrashed": false,
    "duration": "0:00:00.000000",
    "exifInfo": {
        "make": "Apple",
        "model": "iPhone 15 Pro",
        "exifImageWidth": 4032,
        "exifImageHeight": 3024,
        "fileSizeInByte": 1337326,
        "orientation": "6",
        "dateTimeOriginal": "2024-08-18T03:02:29.732Z",
        "modifyDate": "2024-08-18T03:02:29.000Z",
        "timeZone": "UTC-7",
        "lensModel": "iPhone 15 Pro back triple camera 6.765mm f/1.78",
        "fNumber": 1.8,
        "focalLength": 6.764999866,
        "iso": 3200,
        "exposureTime": "0.5",
        "latitude": 0, // redacted
        "longitude": 0, // redacted
        "city": "Redmond",
        "state": "Washington",
        "country": "United States of America",
        "description": "",
        "projectionType": null,
        "rating": null
    },
    "livePhotoVideoId": null,
    "tags": [],
    "people": [],
    "unassignedFaces": [],
    "checksum": "aN1aV91G7ndP+oCYuwA2SPZGVtY=",
    "stackCount": null,
    "isOffline": false,
    "hasMetadata": true,
    "duplicateId": null
}
Metadata for the photo in "Sun, Aug 18"
{
    "id": "4a5336c5-614f-4dde-b4b3-835a89e6308f",
    "deviceAssetId": "E3998847-BA66-4DBE-8898-9EC93621F863/L0/001",
    "ownerId": "c7c204f4-5a70-4074-aed6-6a09ef09f9a6",
    "owner": {
        "id": "c7c204f4-5a70-4074-aed6-6a09ef09f9a6",
        "email": "", // redacted
        "name": "TJ Horner",
        "profileImagePath": "",
        "avatarColor": "green"
    },
    "deviceId": "85a4bcaa692a512c691f95968385f81e97c9c824305cccfeabe6b740d5e0bdb2",
    "libraryId": null,
    "type": "IMAGE",
    "originalPath": "upload/upload/c7c204f4-5a70-4074-aed6-6a09ef09f9a6/94/35/94354eab-3a1b-455e-b94d-e9a7e3b97a69.jpg",
    "originalFileName": "camphoto_1254324197.jpg",
    "originalMimeType": "image/jpeg",
    "resized": true,
    "thumbhash": "CPgFDQCQiE+bqKfXk2h5aIZPbPjF",
    "fileCreatedAt": "2024-08-18T03:01:41.000Z",
    "fileModifiedAt": "2024-08-18T03:01:41.000Z",
    "localDateTime": "2024-08-18T03:01:41.000Z",
    "updatedAt": "2024-08-18T03:13:37.510Z",
    "isFavorite": false,
    "isArchived": false,
    "isTrashed": false,
    "duration": "0:00:00.000000",
    "exifInfo": {
        "make": null,
        "model": null,
        "exifImageWidth": 3024,
        "exifImageHeight": 4032,
        "fileSizeInByte": 2245290,
        "orientation": null,
        "dateTimeOriginal": "2024-08-18T03:01:41.000Z",
        "modifyDate": "2024-08-18T03:01:41.000Z",
        "timeZone": null,
        "lensModel": null,
        "fNumber": null,
        "focalLength": null,
        "iso": null,
        "exposureTime": null,
        "latitude": null,
        "longitude": null,
        "city": null,
        "state": null,
        "country": null,
        "description": "",
        "projectionType": null,
        "rating": null
    },
    "livePhotoVideoId": null,
    "tags": [],
    "people": [],
    "unassignedFaces": [],
    "checksum": "1ssXAqBaqKOJLV4wZi9N+X55mgk=",
    "stackCount": null,
    "isOffline": false,
    "hasMetadata": true,
    "duplicateId": null
}

The correct time and time zone appears for each of the photos when viewing them individually:

image image

PS: Immich is really great software. Very impressed by the breadth of features and how well it matches competitors like Google Photos. I just purchased an individual license to help keep it alive 😄

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

Ok, I did some investigation and it appears to be related to how localDateTime is calculated. It depends on if the EXIF data has a timezone offset attached to it.

Here's the problem area:

const dateTimeOriginal = exifData.dateTimeOriginal;
let localDateTime = dateTimeOriginal ?? undefined;
const timeZoneOffset = tzOffset(firstDateTime(tags as Tags)) ?? 0;
if (dateTimeOriginal && timeZoneOffset) {
localDateTime = new Date(dateTimeOriginal.getTime() + timeZoneOffset * 60_000);
}
await this.assetRepository.update({
id: asset.id,
duration: asset.duration,
localDateTime,
fileCreatedAt: exifData.dateTimeOriginal ?? undefined,
});

I haven't verified this by stepping through it in a debugger, but here's what I think is happening. This snippet is intended to apply the time zone from the EXIF data to the dateTimeOriginal (assuming it is a UTC timestamp) in order to get the timestamp in the time zone that it was taken.

However, it looks like this step is unnecessary since the dateTimeOriginal appears already adjusted to be in the correct time zone (by exiftool, I think?). As an example, the photo taken on August 17, 2024 at 8:02 PM Pacific Time that I posted above has the following dateTimeOriginal: 2024-08-18T03:02:29.732Z. This is the correct UTC timestamp, already adjusted for the time zone it was taken in:

// Convert `dateTimeOriginal` to local time zone
luxon.DateTime.fromISO("2024-08-18T03:02:29.732Z").toLocaleString(luxon.DateTime.DATETIME_FULL)
'August 17, 2024 at 8:02 PM PDT'

However, since the EXIF data also specifies a time zone of UTC-7, the metadata processing job applies this offset to the dateTimeOriginal in order to get the localDateTime (which is 2024-08-17T20:02:29.732Z). But since the offset was already accounted for, instead of a timestamp at 8:02 PM PDT, it is a timestamp at 8:02 PM UTC:

// Convert `localDateTime` to local time zone
luxon.DateTime.fromISO("2024-08-17T20:02:29.732Z").toLocaleString(luxon.DateTime.DATETIME_FULL)
'August 17, 2024 at 1:02 PM PDT'

I think the assumption is made that all dates are TZ-less, since the client code for rendering the photos in the timeline specifies UTC as the timezone for localDateTime rather than obtaining it from the ISO8601 string (this is also why the UTC dates are displayed as headers instead of your local time zone's):

export const fromLocalDateTime = (localDateTime: string) =>
DateTime.fromISO(localDateTime, { zone: 'UTC', locale: get(locale) });

But since the column type in the database is TIMESTAMP WITH TIME ZONE, the timezone metadata is getting stored alongside the timestamp, and it appears to assume UTC for all dates, as indicated by the ISO8601 timestamps specifying no offset.

I'll try to get a local dev instance spun up and fix the issue. Hopefully maybe a PR if I am able to!

@tjhorner
Copy link

I can confirm that omitting zone: 'UTC' from the localDateTime parsing and removing the snippet which incorrectly offsets the dateTimeOriginal fixes the issue; both photos now show up as being taken today:

image

I'll make a PR shortly.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

Localdatetime is used to put it into the correct time bin. It is intended to be the local time for where the photo is taken, so the utc conversation is intended.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

See #4072 for an explanation

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

Time zone logic is very messy and is filled with edge cases. Sorry if you already mentioned this, but what camera took the photos, and were they processed in some way? The TZ variable is a bit of a hack where we tell immich the default time zone to apply to photos that lack the information, usually DSLR photos. Modern phone cameras usually have this information correct. The TZ hack fails if you go on a trip to another time zone. The real fix is usually to edit the TZ information of the photos with exiftool. Maybe I should make a guide...

It seems like localdatetime stores the wrong date of the photos, in my experience this usually means that the issue is within the metadata of the photos, but I could be wrong.

Fun fact, Google photos has terrible support for DSLR photos because they just ignore this issue altogether. If I upload my d700 photos to Google photos and have phone pictures taken of the same event, they will be offset by several hours! Each photo must be manually corrected in the web ui.

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

The photos are from an iPhone 15 Pro, directly from the Immich mobile app with no processing. This comment includes the full JSON metadata of these photos (at the bottom of the comment) if you want to take a look. The dateTimeOriginal stores the correct UTC timestamp of when the photo was taken, taking into account the time zone offset. Immich is performing an unnecessary additional offset.

The conversion to UTC in the frontend should not be happening at all; this will make all date formatting happen relative to the current UTC time rather than the user's local time zone. The timestamps are ISO8601 so there is no need to specify the time zone.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

That does sound strange. I'm still on vacation for another week or two so I'm only on my phone for now. I think we'll need to add some e2e tests for TZ logic, that infrastructure was not available to us when we made #4072 .

Any change to the logic needs extensive testing to make sure all the edge cases are treated correctly.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

The photos are from an iPhone 15 Pro, directly from the Immich mobile app with no processing. This comment includes the full JSON metadata of these photos (at the bottom of the comment) if you want to take a look. The dateTimeOriginal stores the correct UTC timestamp of when the photo was taken, taking into account the time zone offset. Immich is performing an unnecessary additional offset.

The conversion to UTC in the frontend should not be happening at all; this will make all date formatting happen relative to the current UTC time rather than the user's local time zone. The timestamps are ISO8601 so there is no need to specify the time zone.

I'm curious. Are the photos taken with the iphone with gps on or off?

@tjhorner
Copy link

On. The second one was taken with Telegram's in-app camera, which is why much of the EXIF information is missing. However, the dateTimeOriginal is still correct despite the time zone not being present.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

Ok. As another test, what happens if you unset the TZ variable, rerun metadata extraction for that image and check the info pane?

I want to know if it changes. If it does, the photo itself doesn't contain TZ data, it's only assumed from the default time zone set by the TZ variable.

@tjhorner
Copy link

Neither photo changes timestamp in the info pane with TZ unset and metadata extraction re-run. Date, time, and time zone are all correct.

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

Thanks for verifying

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

It appears exiftool-vendored already does some processing on the raw DateTimeOriginal tag to return a TZ-corrected timestamp, which is why we are seeing a correct dateTimeOriginal, and why Immich's processing of the same tag is redundant: https://github.com/photostructure/exiftool-vendored.js/blob/d6fa63469aebb911a7ac929ab287e7d6fe503bec/src/ExifToolOptions.ts#L156-L168

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

You mentioned that DSLR photos will often have incorrect EXIF data and will differ by hours from a phone's photo taken at the same time. Do you have examples of those on hand?

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

(Ignore the below, see next comment for why I am wrong here lol)

So, separately from how localDateTime is calculated, I'm pretty sure we can safely make the client-side change to remove the zone: 'UTC' parameter from timeline-util. This parameter isn't intended to tell Luxon what the time zone of the input is; ISO8601 is a TZ-aware format, so that is already specified within the string. It is only relevant when formatting and transforming the time — from the Luxon docs:

Specifically, a DateTime's zone affects its behavior in these ways:

  1. Times will be formatted as they would be in that zone.
  2. Transformations to the DateTime (such as plus or startOf) will obey any DSTs in that zone that affect the calculation (see "Math across DSTs" below)

We don't want to format the times in UTC, so that one is clearly a bug. This change will slightly mitigate the impact of the incorrect localDateTimes.

I totally understand waiting until a comprehensive test suite etc is written to make the change to localDateTime, though.

@tjhorner
Copy link

tjhorner commented Aug 18, 2024

Ok, I'm reading over that original PR again and I think I get the gist of the change. The goal is for assets to be sorted, grouped, etc. by the local time of capture, e.g. if two photos were taken at 00:00 in any time zone, they will still both show up in the same day no matter what. I understand why the times are being formatted as UTC now, since that is intended to be the "local" TZ-less capture date.

The issue, then, is happening when the EXIF data has no datetime tags and asset.fileCreatedAt is used instead. This means that dateTimeOriginal will still be TZ-corrected, but there will be no timeZoneOffset because the date did not originate from the tags themselves, but instead the system. So localDateTime will still be the correct UTC timestamp, but incorrect in the context of the local capture date.

(Sorry for the thread spam! Just trying to understand the issue and context better.)

@etnoy
Copy link
Contributor

etnoy commented Aug 18, 2024

Thanks for your help in looking into this. As you can probably tell, this is a can of worms indeed.

When I'm back at my computer, I think we should start to add more e2e tests and add relevant TZ cases to our test asset repository. Then we can start changing code.

@tjhorner
Copy link

As you can probably tell, this is a can of worms indeed.

Totally. Working with dates is such a pain 😅

When I'm back at my computer, I think we should start to add more e2e tests and add relevant TZ cases to our test asset repository. Then we can start changing code.

Sounds good to me. Let me know if you would like me to send over any of the photos I'm experiencing issues with to assist in writing those test cases.

@tjhorner
Copy link

Was doing some more research on the rendering/formatting part — I know we're waiting for those E2E test cases before doing any implementation work, but thought I'd share some notes here as they may be helpful in the future. Plus, if I try to keep all this ephemeral knowledge about time and time zones in my head I think it may explode.

One part of the issue is that the dates are being formatted to the relative time in UTC, since that's how localDateTime is stored. We want the formatting to happen relative to the current local time, and it seems we can use date.setZone("default", { keepLocalTime: true }) to change the time zone but not actually change the date/time to match. This means the local time at date of capture will be preserved and the correct relative terms will be used based on the user's current time zone.

Example:

Photo taken on August 18, 2024 at 12:00 in any time zone has localDateTime of 2024-08-18T12:00:00.000Z. This is parsed as UTC then changed to the user's current time zone with .setZone("default", { keepLocalTime: true }). Basically, it will format it as if the photo was taken at 12:00 in the user's current local time no matter what time zone it was originally taken in, which is what is expected.

const localDateTime = luxon.DateTime.fromISO("2024-08-18T12:00:00.000Z", { zone: "UTC" })

localDateTime.toRelativeCalendar()
> 'yesterday'

// Relative time is correct (this was run in Pacific Time at time of comment)
localDateTime.setZone("default", { keepLocalTime: true }).toRelativeCalendar()
> 'today'

// Local time is preserved (12:00 PM UTC turns into 12:00 PM PDT)
localDateTime.setZone("default", { keepLocalTime: true }).toLocaleString(luxon.DateTime.DATETIME_FULL)
> 'August 18, 2024 at 12:00 PM PDT'

Hopefully this makes sense. It's difficult to illustrate/explain 😅

@C-Otto
Copy link
Contributor

C-Otto commented Aug 30, 2024

I'll have a look at this. Personal note: issue surfaces in album c539fe5-5ae3-4275-b27a-48eba9050341.

@C-Otto
Copy link
Contributor

C-Otto commented Aug 30, 2024

Some insights (only web, I haven't looked at the app). I'm in Germany and my browser is configured to use Europe/Berlin, which currently is at offset +2. The Immich server container is also using Europe/Berlin.

  • From https://immich.app/docs/install/environment-variables/: TZ is used by exiftool as a fallback in case the time zone cannot be determined from the image metadata. I think this is confusing, as the time zone is not the same thing as the offset relative to UTC. For example, the time zone Europe/Berlin means +1 in winter and +2 in summer (which is working fine, see below). As the date and offset (usually) is part of some asset's metadata, this suffices to place it on a timeline, or convert the information to some (local, browser specific) representation, though.
  • The details view shows the date and offset correctly for assets where it is configured (exif, XMP, ...), with the exception noted in the next bullet point. As (usually?) no time zone information is stored, this is also not shown (and impossible to deduce automatically). One of my early morning New Zealand photos taken with a camera configured for the offset +13 shows the local time and date with the offset information. A photo just taken a few minutes earlier (still early morning in NZ) , where I manually tweaked the offset to be +2 (changing the time and date accordingly!), shows up at an evening time the day before it was actually taken. This may happen if you forget to set the offset (or, more fancy, time zone) in your camera and Immich can't do anything about it, but at least the shown "absolute" time is correct (you just don't know the correct LOCAL time of the location where the picture was taken). This time may be used to sort photos chronologically, i.e. "which was taken first?".
  • The details view does not show the correct offset if it is zero, for example when taken in Iceland (Reykjavík). If you configure any valid location with an offset of +0 has the same effect as configuring UTC (also +0, sometimes called "Z"): Immich assumes there is no offset information and always shows the (+0) time converted to the local time zone. An image taken in January at midnight in Iceland for me shows up as 1am (+1), an image taken in July (still in Iceland, at midnight) shows up as 2am (+2). This is due to different offsets (daylight saving time) used in my configured time zone (Europe/Berlin). UTC (Z) should be treated differently than explicitly configured offsets (+00:00). Currently, Immich writes out Z instead of +00:00 when explicitly selecting the Iceland timezone. To me, this is a bug. The cause might be in a downstream library: Zero time zone offset +00:00 is parsed as UTC photostructure/exiftool-vendored.js#203
  • The album overview (below the title) shows the date range of all assets in the album, with all times converted to the browser time zone. An image taken on 31 December 1999 at 23:59 in Iceland is treated as 01 January 2000 (at 00:59) as my browser is running using the Europe/Berlin timezone. This is somehow correct, but might be unexpected. Configuring my system/browser to use this time zone "fixes" the issue, i.e. the album overview now shows 1999.
  • The (album) timeline always shows the local time of the assets, ignoring all offset information. In the aforementioned early morning New Zealand example, the photo with the correct offset information (+13) is shown for the date of the morning, while the photo with +2 as the configured offset shows up on the day before (even though it's supposed to be the same morning). This seems to be intentional, showing the respective local time and date (assuming the time zone / offset information in the asset's metadata is correct), so that finding those assets doesn't depend on the time zone of your browser. A picture taken on December 31st in New Zealand always shows up on December 31st. The same holds for the other extreme (Samoa Islands with -11).

@sirweazel
Copy link

sirweazel commented Sep 9, 2024

There are a lot of similar time issue topics. I'm trying to follow them as i have this issue or similar like others. Here i posted some screenshots showing the date of the DB container not using the environmental variable. Could that be an issue, and possible why the photos show as yesterday (like i'm running a few hours in the future)? I show time i took screenshots, and showing "yesterday" but it was still today. Conainer was running a few hours ahead not using eastern or new york time zone.

I didn't want to repost all my screenshots, so i'll link to this other thread.
#12322 (comment)

@sirweazel
Copy link

There are multiple threads with this issue. In one of the threads I'm following, which I mentioned at previous post, It looks like someone might have posted a fix. Link below. I'm on my mobile so I'm having difficulty formatting things correctly if someone wants to tidy up my post feel free.

#12612

@danieldietzler
Copy link
Member

I'd like to close this in favor of #12650. Any objections?

@danieldietzler
Copy link
Member

Oh and actually, it may have been fixed by the PR @sirweazel referenced, yea. Could someone verify?

@snuq
Copy link

snuq commented Sep 26, 2024

If the fix mentioned above has been put into the latest release, then its not fixed yet. The issue still exists in 1.116, in fact if anything its gotten WORSE, now the times/dates in the info panel are incorrect as well.

to illustrate -
Clipboard01
all those photos were taken on the 10th, which immich seems to at least have an idea of internally or it wouldnt be putting them BELOW the other photos.... in previous versions, the info panel for the lower photos would say the correct date of the 10th, but now the info panel shows the 11th

my server still has the TZ environment variable set to "America/Los_Angeles"

@danieldietzler
Copy link
Member

Did you re-extract metadata for those again?

@snuq
Copy link

snuq commented Sep 26, 2024

i selected them and did 'refresh metadata' from the hamburger menu. i havnt done a full db refresh tho

@danieldietzler
Copy link
Member

That should be enough. I'd like to close this in favor of #12650 though, as that's supposed to bundle all those issues. I'd appreciate it if you could post a message summarizing the issue there, and also provide a sample file (as a zip archive).

@Tjsyl
Copy link

Tjsyl commented Sep 29, 2024

Most of the images from today show "Yesterday" and one shows in the future "Sun, Sep 29".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests