immich-go

Many thanks to @maybeanerd for their meticulous proofreading of the documentation files.

New feature: Use the full image path as album name

Thanks to @giejay for their contribution
When the -use-full-path-album-name option is enabled, photos are added to a new album named after their full file path.
The path separator can be replaced using the -album-name-path-separator=CHAR

New feature: google photos archived photos are imported as immich archive by default

Thanks to @Alex1607 for their contribution
Use the option -auto-archive=FALSE to disable this feature.

What's Changed

• fix Takeout zip is unsupported file type #357 by @simulot in https://github.com/simulot/immich-go/pull/415
• docs: fix typos in readme by @maybeanerd in https://github.com/simulot/immich-go/pull/421
• Program errors out due to no ping API response despite API responding by @simulot in https://github.com/simulot/immich-go/pull/431
• remove "GetJobs" call from API traces by @simulot in https://github.com/simulot/immich-go/pull/442
• Add support for -use-full-path-album-name to be able to use the full path to the file as album name/title by @giejay in https://github.com/simulot/immich-go/pull/444
• Documentation-update by @simulot in https://github.com/simulot/immich-go/pull/446
• Add new AutoArchive option by @Alex1607 in https://github.com/simulot/immich-go/pull/450
• Update README.md, google-takeout.md, and motivation.md by @aaronjrodrigues in https://github.com/simulot/immich-go/pull/454

New Contributors

• @maybeanerd made their first contribution in https://github.com/simulot/immich-go/pull/421
• @giejay made their first contribution in https://github.com/simulot/immich-go/pull/444
• @Alex1607 made their first contribution in https://github.com/simulot/immich-go/pull/450

Full Changelog: https://github.com/simulot/immich-go/compare/0.21.0...0.22.0

Changelog

• d58d2669ae6f13bbb32ff45fc787e92f0ae4d166 "Added return nil statement when encountering a directory in passOneFsWalk function" (#447)
• 8adfd32a18c1221407ff7000c23a6cb14e5363d1 Add support for -use-full-path-album-name to be able to use the full path to the file as album name/title (#444)
• ab7ee9f2c88d19185e52336f3cc7687100638a10 Documentation-update (#446)

Changelog

• 3396f6f24d7624e076469195759fe315134a14fb Program errors out due to no ping API response despite API responding (#431)
• 1a4de53f812a58f7e4b312f5313552760570323e chore(deps): bump golang.org/x/sync from 0.7.0 to 0.8.0 (#425)
• fad0f570edfb9796c9cf337f54b886796c5c0acb google photo: file is uploaded with the title VID_20180811_184034.mp4.MP4 (#441)
• c9bfd9758a9772eaa29d82338db9e7c80e34c9aa remove "GetJobs" call from API traces (#442)

Changelog

• ca96b7543cf52e3f6262d029557fe79dc2951ac0 Improve get debug data (#413)
• c6377178797bd12f63305877f5903417f9791417 edit readme (#420)
• ff81fd0a80fde6984562d330ce3bcd8ec380102b fix Takout zip is unsupported file type #357 (#415)
• 1d3016eccb3790c0fcd4ea7eb89c4e9ff6a0a892 fix motion photo files with MP~2 extension marked unsupported and skipped #405 (#419)

Refactoring the Google Photos import another time

Lot of users have reported inconsistencies in upload counters. Each user case a different, and the takeout structure varies a bit.
In order to debug those cases, I have developed a way to simulate the takeout import using only the the file list. Read how to send debug data without sharing photos.

Option to force the upload of images despite the lack of JSON

Each image in a takeout is supposed to come with A JSON file giving the date of capture and the GPS coordinate. There a few reason for this:

1. The original file is copied, modified... and sometime there ins't a JSON for all versions
2. JSON aren't in the same ZIP file than the image, and only one part of the takeout is processed
3. The takeout misses a bunch of JSON

When asking another takeout isn't an option, it's possible to force the upload of photos with no JSON. Use the option -upload-when-missing-JSON

The stack function is disabled

The stack function need to be improved #399, #345, #235
Meanwhile, it is disabled by default. You can enable it using the option -create-stacks=TRUE.

fixes:

• #376 errors when uploading are disturbing the the % of the progression
• files with same path and name, but in different part of the takeout file set was forgotten in duplicate counters
• iPhone's Live photos recognition when the name is duplicated: ex IMG_2710(1).MP4 and IMG_2710(1).HEIC
• Missing a file when a directory contain several files with the same name, but of a different type. Ex: IMG_0170.HEIC, IMG_0170.JPG
• Live videos attached to duplicated photos are now counted as duplicate as well, making the final report more relevant
• #402 Wrong album assignment for images with the same name
• #390 Question: report shows way less images uploaded than scanned
• #376 errors when uploading are disturbing the the % of the progression
• #401 Add an option to import images/movies even if there is no JSON file in the takeout

Release 0.20.1

changes

• add git action to build and release

fixes:

• #380 not all GP duplicates are detected correctly, counters are wrong

Feature: exclude files based on a pattern

Use the -exclude-files=PATTERN to exclude certain files or directories from the upload. Repeat the option for each pattern do you need. The following directories are excluded automatically:

• @eaDir/
• @__thumb/
• SYNOFILE_THUMB_*.*
• Lightroom Catalog/
• thumbnails/
• .DS_Store/

Example, the following command excludes any files in directories called backup or draft and any file with name finishing with "copy)" as PXL_20231006_063121958 (another copy).jpg:

immich-go -sever=xxxxx -key=yyyyy upload -exclude-files=backup/ -exclude-files=draft/ -exclude=copy).*  /path/to/your/files

Fixes:

• #365 missing associated metadata file isn't correct
• #299 Real time GUI log only shows 4 lines
• #370 ui: clearly mention when the upload in completed
• #232 Exclude based on filename / glob
• #357 clarify error message when a zip file is corrupted

A big thank you to @jrasm91 and @bo0tzz for their help

Changelog

• 00f3c1c Getting stuck at 75% - server assets to delete #343
• 3b8fc3d Merge branch 'main' into simulot/issue359
• 5b7dafd Unexpected Discrepancy in 'Server has same quality' Metric After Re-uploading Images Fixes #359
• a1ba280 fix -api-trace error

Full Changelog: https://github.com/simulot/immich-go/compare/0.19.0...0.19.1

You can sponsor the project immich-go ! Thank you for your feedback!

In this release:

• Github sponsor
• Use Google Photos date and location instead of EXIF data
• Use description and text enrichment from Google Photos albums
• Use Google Photos albums location when album's photos don't have GPS coordinates

Changelog

• 67c50ec Api-trace (#351)
• 1d41803 Create FUNDING.yml
• dafec49 Metadata (#337)
• e009356 add sponsor links
• bca9618 fix: Do not require value for version call (#352)

What's Changed

• Api-trace by @simulot in https://github.com/simulot/immich-go/pull/351
• fix: Do not require value for version call by @kai-tub in https://github.com/simulot/immich-go/pull/352
• Metadata by @simulot in https://github.com/simulot/immich-go/pull/337

Full Changelog: https://github.com/simulot/immich-go/compare/0.18.2...0.19.0

Changelog

• 30afeed Denied access to admin only route: /api/job (#349)
• 11e6f58 Simulot/issue347 (#348)
• 2248746 chore(deps): bump github.com/thlib/go-timezone-local from 0.0.2 to 0.0.3 (#341)
• b3b2334 edit release.md

fix #336 Processing stops with "context canceled" all the time

feat: #322 Add -version to get the immich version

The option -version return the version of the executable.

feat: #289 Server's activity graph

The UI mode now show the current activity of the immich server. After 10 seconds of zero activity, the program stops

feat: generate a CSV files with the fate of each file

Use the option -debug-counters to generate a CSV beside the log file

feat: #308 Immich-go gets photos date from filename or path

Immich-go tries to determine the date of capture with the file name, or the file path.

Ex:

Thanks to @erkexzcx for his contribution.

fix: #326, #303 Live Photo / Motion pictures

Since a recent release of Immich, the live photos and motion picture were seen as a picture and a small movie.
The code has been refactored to be sure that the movie part is uploaded before the photo, and attached to the photo.

fix: #304 Error when uploading images with a wild card without path .JPG

Immich-go accepts "*.jpg" as parameter.

fix: #317 Explicit message when the call to /api/server-info/ping fails

The message is now explicit:

The ping API end point doesn't respond at this address: http://localhost:2283/api/server-info/ping

fix: #235,#240 Stack detection issue

Thanks to @matteolomba for his contribution

fix: Path of temporary files

Temporary files are created in the system's temporary folder.

fix: #311 Readme spelling

fix: report unsupported files as unsupported

fix: report actual error instead of "context canceled"

fix: stop all task on error in the no-ui mode

Released 0.17.1

Fix: UpdateAsset new API endpoint

Fix: Typo in motivation.md file

Changelog

• abd793e fix #336 (#338)

feat: #322 Add -version to get the immich version

The option -version return the version of the executable.

feat: #289 Server's activity graph

The UI mode now show the current activity of the immich server. After 10 seconds of zero activity, the program stops

feat: generate a CSV files with the fate of each file

Use the option -debug-counters to generate a CSV beside the log file

feat: #308 Immich-go gets photos date from filename or path

Immich-go tries to determine the date of capture with the file name, or the file path.

Ex:

Thanks to @erkexzcx for his contribution.

fix: #326, #303 Live Photo / Motion pictures

Since a recent release of Immich, the live photos and motion picture were seen as a picture and a small movie.
The code has been refactored to be sure that the movie part is uploaded before the photo, and attached to the photo.

fix: #304 Error when uploading images with a wild card without path .JPG

Immich-go accepts "*.jpg" as parameter.

fix: #317 Explicit message when the call to /api/server-info/ping fails

The message is now explicit:

The ping API end point doesn't respond at this address: http://localhost:2283/api/server-info/ping

fix: #235,#240 Stack detection issue

Thanks to @matteolomba for his contribution

fix: Path of temporary files

Temporary files are created in the system's temporary folder.

fix: #311 Readme spelling

fix: report unsupported files as unsupported

fix: report actual error instead of "context canceled"

fix: stop all task on error in the no-ui mode

Released 0.17.1

Fix: UpdateAsset new API endpoint

Fix: Typo in motivation.md file

New Contributors

• @aaronjrodrigues made their first contribution in https://github.com/simulot/immich-go/pull/311
• @matteolomba made their first contribution in https://github.com/simulot/immich-go/pull/320

Full Changelog: https://github.com/simulot/immich-go/compare/0.17.1...0.18.0

Release 0.17.1: bug fix

Fix: UpdateAsset new API endpoint

Fix: Typo in motivation.md file

Changelog

• c491a22 Merge pull request #293 from d4vidsha/d4vidsha/motivation-fixes
• 9c46903 Merge pull request #301 from dreautall/patch-2
• 2d9ce24 edit release.md file
• 6e59260 fix assets API url
• 6391f77 fix: typo on "own"; remove space

⚠️ Immich has changed its API

This version of immich-go is compatible with immich v.1.106 and later. Use immich-go version 0.16.0 with older immich servers.

feature: [#284] Be ready for the next immich version

See https://github.com/immich-app/immich/pull/9831 and https://github.com/immich-app/immich/pull/9667 for details

fix: log upload errors with error level error in JSON logs

fix: [#287] Prevent the Upload of photos that are trashed (Google Photos)

Trashed server's assets are excluded from the duplicate detection before uploading the same asset.

Changelog

• 5488cef Be ready for the next immich version Fixes #284
• 1e7b4ca Merge branch 'main' into simulot/issue275
• ccfeb98 Merge commit '7579f25'
• 742df19 Merge pull request #278 from simulot:simulot/issue277
• 488db84 Merge pull request #279 from simulot:simulot/issue275
• 9f750f7 Merge pull request #282 from simulot:simulot/issue276
• 824254e Merge pull request #285 from simulot:simulot/issue284
• 3f0dfe0 Merge pull request #288 from simulot:simulot/issue287
• 6bbed93 Optimizations Fixes #276
• bb9fa66 Prevent the Upload of photos that are trashed (Google Photos) Fixes #287
• bd61bad Remove -force-sidecar option
• 475ec59 edit readme and release files
• 0c60659 edit readme and releases.md files
• 6c694f6 edit readme.md
• 9b37596 edit release.md
• 257451b log upload errors with error level error in JSON logs
• 602d0d1 no-colors-log option removed as not used any more

Release 0.16.0

feature: [#261] Fallback to no-gui mode when the UI can't be created

When the terminal can't handle the UI mode, the program falls back to non gui mode automatically

feature: The log can be written with the JSON format (JSONL)

Use the -log-json option to enable JSON logging (JSONL format). This allows using ./jq to explore large logs.

feature: [#277] Adjust client side timeouts

The immich client timeout is set with the option -client-timeout=duration.
The duration is a decimal numbers with a unit suffix, such as "300ms", "1.5m" or "45m". Valid time units are "ms", "s", "m", "h".

fix: [#270] Missing associated metadata file counter is not updated after the performance improvement

The counter missing associated metadata is broken since 0.15.0

fix: [#266] Better handling of archive name with wildcards that matches with no file

When the file name pattern returns no files, a message is printed, and the program ends.

fix: [#273] Missing upload files

Any error is counted as upload error, and reported in the log file.

fix: Error handling during multitasking

Any error occurred during parallelized tasks cancels other as well.

fix: Processed files count is displayed in no-ui mode

The processed files counter is updated whenever a file for the source is processed.

fix: Unsupported files are now counted as unsupported files

There were previously counted as discarded files.

fix: The name of the sidecar file is correctly written in the log

fix: [#272] Wrong release downloaded for 0.15.0

Oops!

Changelog

• de0b458 Better handling of archive name with wildcards that matches with no file Fixes #266
• 3549bd8 Edit releases.md
• c98b3f9 Edit releases.md
• 11cfe5f Edit releases.md
• 6ffb62a Edit releases.md
• 6a4fdaa Fallback to no-gui mode when the UI can't be created Fixes #261
• dc65281 Fix: report the xmp file name in the log
• 1007705 Merge pull request #264 from simulot:simulot/issue261
• e0f0321 Merge pull request #268 from simulot:simulot/issue266
• c80201b Merge pull request #271 from simulot:simulot/issue267
• ef8d972 Merge pull request #274 from simulot:simulot/issue273
• b58becb add the possibility to write a JSON log
• 1bc587c better log handler initialisation
• 8b0b2aa change the wording "server has a better asset" instead of "server has a better photo"
• 62175d1 edit readme and release note
• 2fb5950 edit release.md
• 118ee4f fix the error management when pulling assets from immich and parsing the archive
• 7e5bede fix the immich reading progression bar
• 34100e1 fix: Adjust client side timeouts #277
• eb275b6 fix: Count offile not assiciated with google JSON
• 7579f25 fix: e2e tests
• 80e102c fix: print the latest error when exiting
• 17b0caf fix: usupported files are counted as discared files
• 088a117 report upload error
• 26204a9 show procecessed in no-gui mode
• 31b32c3 write report

fix #255 Last percents of google puzzle solving are very slow when processing very large takeout archive

The google puzzle solving is now much faster for large takeout archives.

fix #215 Use XDG_CONFIG_HOME for storing config

The configuration file that contains the server and the key is now stored by default in following folder:

• Linux $HOME/.config/immich-go/immich-go.json
• Windows %AppData%\immich-go\immich-go.json
• Apple $HOME/Library/Application Support/immich-go/immich-go.json

Store the log files into sensible dir for user's system

The default log file is:

• Linux $HOME/.cache/immich-go/immich-go_YYYY-MM-DD_HH-MI-SS.log
• Windows %LocalAppData%\immich-go\immich-go_YYYY-MM-DD_HH-MI-SS.log
• Apple $HOME/Library/Caches/immich-go/immich-go_YYYY-MM-DD_HH-MI-SS.log

Feat: [#249] Fix Display the path of log file name

The log file name is printed when the program exits.

Changelog

• 87471f6 Edit releases.md
• d4269ee Fix Display the path of log file name (#251)
• 49bf673 Fix Display the path of log file name (#256)
• 73fde8d Fixes #255
• 9b70932 Merge branch 'main' into simulot/issue215
• 734bbeb Merge pull request #257 from simulot:simulot/issue255
• 8805ca7 Merge pull request #258 from simulot:simulot/issue215
• c63dc09 Merge pull request #260 from simulot:simulot/issue259
• e4d1643 Set a timeout for all http queries (#253)
• f968bd6 The log doesn't show discared files Fixes #259
• 37f4b83 Use XDG_CONFIG_HOME for storing config (#248)
• 755e1b7 Use XDG_CONFIG_HOME for storing config Fixes #215

Release 0.14.1 "A better UI"

This release is focussed the improvement of the user experience.

A shiny user interface

. _ _  _ _ . _|_  __  _  _ 
|| | || | ||(_| |    (_|(_)
      version dev     _)

Working with big database and big takeout files take some time. Users are now informed about the progression of different tasks:

The screen presents number of processed photos, how they have been processes, the messages log, and at the bottom, the progression of the 3 mains tasks.

A minimalist user interface

This shiny interface can be be disabled for quieter user interface (-no-ui).
The progression is visible. All details on operations are listed in the log file.

. _ _  _ _ . _|_  __  _  _ 
|| | || | ||(_| |    (_|(_)
      version dev     _)   

Server status: OK
Connected, user: [email protected]
Immich read 100%, Google Photos Analysis: 100%, Uploaded 100%  

Input analysis:
----------------------
scanned image file                      :   25420
scanned video file                      :    1447
scanned sidecar file                    :   26934
discarded file                          :     197
unsupported file                        :       0
file duplicated in the input            :    1706
associated metadata file                :   26867
missing associated metadata file        :       0

Uploading:
----------
uploaded                                :   25160
server error                            :       0
file not selected                       :       1
server's asset upgraded with the input  :       0
server has same photo                   :       0
server has a better asset               :       0

Immich-go runs simultaneously the collect of immich-server's assets and the analysis of the Google takeout

The code has been refactored to run several task simultaneously to animate the progression screen. The program runs now the reading of immich asset and the the takeout analysis in parallel.

Immich-go now always produces a log file

The default name for the log file is immich-go YYYY-MM-DD HH-MI-SS.log, located in the current directory.

It's possible to give a path and a name to the log file with the option -log-file=path/to/file.log.
If the file file exists already, the new messages will be added to its end.

The log level OK is removed.

Immich-go is published under the AGPL-3.0 license

I chose the same license as the immich project license to release immich-go.

Next steps

• Issues closing
• A shiny user interface for the command duplicate

Changelog

• e953522 Automatically determine commit info with buildinfo (#243)
• e4c6ba9 Google Takeout 403 Forbidden on non admin user (#247)
• f8e7b42 fix e2e test
• 1acd3c8 impove the banner
• 1f02717 A better user interface (#245)
• e13cee4 Create LICENSE (#220)
• 6634a2a Simulot/issue205 (#206)
• 674b165 Update ci.yml (#229)
• f79261d Update readme.md (#237)
• 9c62473 Version 0.13.0 not working as expected (#200)

Release 0.14.0 "A better UI"

This release is focussed the improvement of the user experience.

A shiny user interface

. _ _  _ _ . _|_  __  _  _ 
|| | || | ||(_| |    (_|(_)
      version dev     _)

Working with big database and big takeout files take some time. Users are now informed about the progression of different tasks:

The screen presents number of processed photos, how they have been processes, the messages log, and at the bottom, the progression of the 3 mains tasks.

A minimalist user interface

This shiny interface can be be disabled for quieter user interface (-no-ui).
The progression is visible. All details on operations are listed in the log file.

. _ _  _ _ . _|_  __  _  _ 
|| | || | ||(_| |    (_|(_)
      version dev     _)   

Server status: OK
Connected, user: [email protected]
Immich read 100%, Google Photos Analysis: 100%, Uploaded 100%  

Input analysis:
----------------------
scanned image file                      :   25420
scanned video file                      :    1447
scanned sidecar file                    :   26934
discarded file                          :     197
unsupported file                        :       0
file duplicated in the input            :    1706
associated metadata file                :   26867
missing associated metadata file        :       0

Uploading:
----------
uploaded                                :   25160
server error                            :       0
file not selected                       :       1
server's asset upgraded with the input  :       0
server has same photo                   :       0
server has a better asset               :       0

Immich-go runs simultaneously the collect of immich-server's assets and the analysis of the Google takeout

The code has been refactored to run several task simultaneously to animate the progression screen. The program runs now the reading of immich asset and the the takeout analysis in parallel.

Immich-go now always produces a log file

The default name for the log file is immich-go YYYY-MM-DD HH-MI-SS.log, located in the current directory.

It's possible to give a path and a name to the log file with the option -log-file=path/to/file.log.
If the file file exists already, the new messages will be added to its end.

The log level OK is removed.

Immich-go is published under the AGPL-3.0 license

I chose the same license as the immich project license to release immich-go.

Next steps

• Issues closing
• A shiny user interface for the command duplicate

Changelog

• 1f02717 A better user interface (#245)
• e13cee4 Create LICENSE (#220)
• 6634a2a Simulot/issue205 (#206)
• 674b165 Update ci.yml (#229)
• f79261d Update readme.md (#237)
• 9c62473 Version 0.13.0 not working as expected (#200)
• 2a7f962 chore(deps): bump golangci/golangci-lint-action from 4 to 5 (#227)
• d04b35f chore(deps): bump golangci/golangci-lint-action from 5 to 6 (#239)
• eeeb86b immich-go appears to retain/cache an API key (#214)

Changelog

• 6634a2a Simulot/issue205 (#206)
• 9c62473 Version 0.13.0 not working as expected (#200)
• 159e381 immich-go appears to retain/cache an API key Fixes #211

Changelog

• 332afea Version 0.13.0 not working as expected Fixes #199
• 6fb63f5 linter pass