immich-go

Release 0.8.6

fix for #68: A lot of images skipped from Google Photos Takeout

The Google takeout archive is full of traps. The difficulty is to associate all images with a JSON.
Now more files are now imported. There still few missing files, but they are now listed.

The program now reports how files are handled, or discarded.

Upload report:
 53998 scanned files
 53993 handled files
 26937 metadata files
   535 uploaded files on the server
    49 upgraded files on the server
  1540 duplicated files in the input
  8382 files already on the server
    77 discarded files because in folder failed videos
     1 discarded files because of options
 16470 discarded files because server has a better image
     1 files type not supported
     1 errors
     5 files without metadata file
7 files can't be handled
File: Takeout/Google Photos/Photos from 2019/1556189729458-8d2e2d13-bca5-467e-a242-9e4cb238e(1).jpg
        File unhandled, missing JSON
File: Takeout/Google Photos/Photos from 2022/original_1d4caa6f-16c6-4c3d-901b-9387de10e528_P(1).jpg
        File unhandled, missing JSON
File: Takeout/Google Photos/Photos from 2022/original_af12c386-e334-4c57-88be-fdfadea71f16_P(1).jpg
        File unhandled, missing JSON
File: Takeout/Google Photos/Photos from 2022/original_ec8d7b93-cbec-49c8-8707-38841db5e37d_P(1).jpg
        File unhandled, missing JSON
File: Takeout/Google Photos/Photos from 2023/original_d3671642-c937-49c0-917a-8ef9cbb449c5_P(1).jpg
        File unhandled, missing JSON
File: Takeout/Google Photos/user-generated-memory-titles.json
        Error , json: cannot unmarshal array into Go struct field GoogleMetaData.title of type string
File: Takeout/archive_browser.html
        File type not supported
Done.

The plenty of rules for associating image to JSON are somewhat contradictory. I have to rethink the system for applying
rules from the most common to the strangest ones.

Still lot of work to deliver.

Changelog

• fde9232 #68: MP files
• 1e7e316 #68: better error report
• ef9958a #68: handling edge cases
• 9a09501 Merge branch 'Fix-for-#78-mp4-files-do-not-get-imported' into #68-Lot-of-images-is-missing
• 50c5bdf WIP #68
• 68d4d40 edit release.md
• 6f26ece wip: better upload report

Changelog

• b8827c8 fix for #78: mp4-files do not get imported

Changelog

• edc40a3 doc: edit release.md
• 071b52d fix #67: Live photos files are stacked and not recognized as live photos
• fd62fa8 implement include / exclude options
• f36888b wip #67
• b93ceab wip #67: implement Live photos for folders
• d959c75 wip #67: implement live photos for google takeouts
• c214a49 wip: rename package assets in browser

Changelog

• bc6f066 Merge branch 'feature-#58---add-filter-on-directories-during-the-import' into Fix-for-#69----Panic--runtime-error--slice-bounds-out-of-range---29--
• c62e7a0 add trimpath to goreleaser.ymal
• d4442af fix for #69: Panic: runtime error: slice bounds out of range
• a5bed00 implement nologger
• d69c11a move the logger package at root of the project
• df77d9b wip #69
• 8df7277 wip #69

Changelog

• 0792ade doc: edit releases.md
• 20e3422 fix for #61: album must have name error
• fb9b7fb fix for #64 segfault
• cc58567 wip: fix for segfault

Changelog

• e258497 workaround for #62

Release 0.8.0

New feature: create stacks when uploading images

The option -create-stacks <bool> drive the creation of stack of images for couples JPG/RAW or bursts of photos. The option is enabled by default.

Your asset must have the date of capture in the metadata.

Example:

./immich-go -server=http://mynas:2283 -key=zzV6k65KGLNB9mpGeri9n8Jk1VaNGHSCdoH1dY8jQ upload
 ~/mnt/sdcard/           

Server status: OK
Ask for server's assets...
....
Done, total 12 uploaded
Creating stacks
  Stacking 3H2A0018.CR3, 3H2A0018.JPG...
  Stacking 3H2A0019.CR3, 3H2A0019.JPG...
  Stacking 3H2A0020.CR3, 3H2A0020.JPG...
  Stacking 3H2A0021.CR3, 3H2A0021.JPG...
  Stacking 3H2A0022.CR3, 3H2A0022.JPG...
  Stacking 3H2A0023.CR3, 3H2A0023.JPG...
12 media scanned, 12 uploaded.
Done.

New feature: command tool album delete [regexp pattern]

Changelog

• 5d1c902 Merge branch 'Fix--CR3-date-of-catpture-TZ-error-#56' into implement--stack-jpg+raw-wen-uploading-photos-#54
• 2dcefa9 add feature delete album
• 9987597 add: stack jpg+raw when uploading photos #54
• 84d5432 doc: edit readme
• b1fcb87 doc: release note
• 4ff4432 fix #57 cr2
• 9bb6d53 fix DNG files #59, CR3 TZ error #56, CR2 Exif #57
• d232e3c fix e2e test
• ea4dd0d readme.md typo
• 64d44f4 simplication of stack detection
• 45baf5e wip: staking while import

Changelog

• 59f9630 Fix #52: Duplicate command fails with 504 timeout
• e467c02 Fix #52: Duplicate command fails with 504 timeout
• 2b695ff Merge branch 'main' into fix--Duplicate-detection-fails-when-timezone-of-both-images-differs-#50
• 2d67e08 Merge branch 'main' of github.com:simulot/immich-go
• e0ca792 fix #50: Duplicate detection fails when timezone of both images differs
• 76e29ab fix output message

Changelog

• 4c9000a Fix #51 Import a single file doesn't work
• 4a7e2ae Multiple fix around google photos albums
• 6a6b6a2 edit readme.md
• 9f96552 edit readme.md

This release introduce add the command `stack' to group jpg/raw images together or all images of a burst.
Now freebsd binaries are released.

Changelog

• 3234a17 #42: edit readme
• 5aa7f53 Merge branch 'Fix-#42--better-handling-of-google-photos-albums' into Use-the-new-stacking-feature-to-group-jpg-and-raw-images,-same-for-burst-#45
• 9308e5a Merge branch 'Use-the-new-stacking-feature-to-group-jpg-and-raw-images,-same-for-burst-#45'
• 2d01ebb Merge branch 'better-handling-of-getAssets' into Fix-#42--better-handling-of-google-photos-albums
• bda9c2d Merge branch 'dev'
• 55cb025 Merge branch 'fix-for--Error-when-importing-#47'
• c87b0fa Merge branch 'main' into dev
• 00ded11 Merge branch 'readme-struture'
• bd74a51 WIP on stack command
• 1194ad6 add e2e tests for upload command
• 68ab3df add freebsd to releases
• 478cb38 add freebsd to releases
• 80322da add support for .jpe files
• 436ab7d discard trashed assets
• 4bd4ec5 fix #47 error when importing from a folder
• 6f806e0 fix for #45 Use tof the stacking functionality
• 657dd94 merge #47
• e9e2f35 responseJSON is now generic
• 3ee0295 restructure the readme.md
• 74e3218 restructure the readme.md

Changelog

• a5d2edf Add isTrashed field in Asset
• ef0a893 Edot readme.md
• 5522ed2 Fix #44: duplicate is not working
• fabe0d4 Fix SEGDEV error on GP
• 511b3b7 edit readme.md
• 4d2a606 fix #39: another problems with Takeout archives

Changelog

• 7290792 assume that time in asset name is UTC
• a3d39b6 fix #40: Don't decode the response when the status is 204

Changelog

• 20a39f7 fix #37 only one asset gets uploaded / unnecessary album gets created

Changelog

• 6073872 Fix for #35: weird name cases in google photos takeout: truncated name or jp.json
• eaf59f5 Merge branch 'main' into fix-#32--Albums-contains-album's-images-and-all-images-having-the-same-name
• 9fc19c5 edit readme.md
• ebf8152 edit readme.md
• 4bb0ad8 fix #32: Albums contains album's images and all images having the same name
• 6b4c945 fix TakeTimeFromName tests

Changelog

• 700f0ae Merge branch 'Fix--#30-time--missing-Location-in-call-to-Time.In'
• 892c89e edit readme.md
• 0a4285f fix #30: tzdata are now embeded within binaies

Changelog

• 4a2a9a0 edit readme.md
• b41011c go mode tidy

Changelog

• 8d36fff fix #30 panic: time: missing Location in call to Time.In

The takeout archive has flaws making the import task difficult and and error prone.
I have rewritten this part of the program to fix most of encountered error.

google photos: can't find image of album #11

Some image may miss from the album's folder. Those images files are located into the year folder.
This fix looks for album images in the whole archive.

photos with same name into the year folder #12

Iphones and digital cameras produce images with the sequence number of 4 digits. This leads inevitably to have several images with the same number in the the year folder.

Google Photos disambiguates the files name by adding a counter at the end of the image file:

• IMG_3479.JPG
• IMG_3479(1).JPG
• IMG_3479(2).JPG

Surprisingly, matching JSON are named as

• IMG_3479.JPG.json
• IMG_3479.JPG(1).json
• IMG_3479.JPG(2).json

This special case is now handled.

Untitled albums are now handled correctly

Untitled albums now are named after the album's folder name.

This address partially the issue #19.

can't find the image with title "", pattern: ".": file does not exist: "___" #21

The refactoring of the code don't use anymore a file pattern to find files in the archive.
The image and the JSON file are quite identical, except for duplicate image (see #12) or when the file name is too long (how long is too long?).

Now, the program takes the image name, check if there is a JSON that matches, open it and use the title of the image to name the upload.

If the JSON isn't found, the image is uploaded with it's name in the archive, and with no date. Now all images are uploaded to immich, even when the JSON file is not found.

MPG files not supported. #20

Immich-go now accepts the same list of extension as the immich-server. This list is taken from the server source code.

immich-go detects raw and jpg as duplicates #25

The duplicate checker now uses the file name, its extension and the date of take to detect duplicates.
So the system doesn't signal IMG_3479.JPG and IMG_3479.CR2 as duplicate anymore.

fix duplicate check before uploading #29

The date parsing now takes into account the time zone of the machine (ex: Europe/Paris). This handles correctly summer time and winter time.
This isn't yet tested on Window or Mac machines.

Changelog

• 77f5f0c Merge branch 'main' of github.com:simulot/immich-go
• 61bb736 fix #29 date comparison before uploading file
• 47283c3 fix darwin compilation
• fd63325 fix release link in README.md (#22)
• d9b0165 google photos: disregard album metadata and use dir name as album title
• ee4ef82 refactor google photo fix for #11 #12 #21 #20 #25
• 49b387d update readme.md

Changelog

• 1737963 Merge branch 'main' of https://github.com/simulot/immich-go
• 83e8ba7 add partner sharing album (#14)

Changelog

• c3439aa Merge branch 'main' into mrwulf-partner-sharing-album
• d7428c8 add partner sharing album
• bf59715 ducplicate: option for Confirm delete
• 17f3b6b duplicate: better handling of ^C
• 5911886 edit readme.md
• 4b064c9 go.mod
• 156a312 upload: fix duplicate assets when creating an album
• a736b6a upload: update the list of supported file (issue #15)