If you have an Amazon Prime subscription you get unlimited photo storage in Amazon Photos.

If you don’t have a Google Drive subscription you get a 15GB storage limit for photos.

After a while you will hit the Google storage limit. If you don’t want to pay for a Google Drive subscription, you can do this to move everything.

  • Check you have about 30GB free on disk to do this comfortably
  • Request takeout from Google of all photos
  • Wait for Google to email a download link
  • Download archive
  • Inspect archive (takes several seconds to open in Ubuntu archive manager)

  • Extract archive

    • Don’t use the right-click and extract in Nautilus. It just creates an empty folder.
    • Use this command in the terminal: tar -xzf takeout-20210131T163243Z-001.tgz
  • Check that the size of the extracted folder is slightly more than the original archive (decompressed JSON metadata)
  • Open Amazon Photos in Firefox
  • Click “Add” then “Upload folder”
  • Try to upload the whole “Google Fotos” folder at once lol yolo
  • See upload counter “Uploading 8 of 18557…”
  • Wait for it to churn through them all!

  • Check the stats.

    • 9192 items uploaded
    • 9465 upload errors
    • Is that because of the JSON files or did some photos actually fail?
    • Install https://marketplace.visualstudio.com/items?itemName=mohd-akram.vscode-html-format
    • Inspect errors element of page <section class="error-list">
    • Copy outer HTML of element
    • Paste into VS Code and format

    • Filter to find any non-JSON errors. This works becuase VS Code formatting puts the span on the same line as the div.

      $ cat errors.html |grep "error-desc" | sort | uniq --count
    149             <div class="error-description"><a href="/photos/storage" class="over-quota-link"><span>Over quota</span></a>
   9316             <div class="error-description"><span>Invalid file type</span></div>

    • The file name is on the line before. We can use context lines or we can use XPath.

      $ xmllint --xpath "//div[@class='error']" errors.html
<div class="error">
            <div class="file-name">IMG_20201219_153540770.jpg.json</div>
            <div class="error-description"><span>Invalid file type</span></div>
        </div>
<div class="error">
            <div class="file-name">24167879.webp.json</div>
            <div class="error-description"><span>Invalid file type</span></div>
        </div>
[...]

    • Use XPath to list the invalid files names.

      $ xmllint --xpath "//div[@class='error' and div[@class='error-description']//span/text()='Invalid file type']/div[@class='file-name']/text()" errors.html > invalid-files.txt
$ cat invalid-files.txt
IMG_20201219_153540770.jpg.json
24167879.webp.json
22976173.webp.json
[...]

    • The next command returns no results, so all the invalid files were JSON files. That’s good!

      $ awk '!/\.json$/' invalid-files.txt

    • Use XPath to list the files over quota.

      $ xmllint --xpath "//div[@class='error' and div[@class='error-description']//span/text()='Over quota']/div[@class='file-name']/text()" errors.html > over-quota.txt
$ cat over-quota.txt
VID_20200123_011704_688.mp4
VID_20200315_074648_545.mp4
VID_20200122_002649_641.mp4
[...]

    • The next command shows that all the files over quota are mp4 video files. It’s unfortunate to exceed the quota, but it’s expected, because Amazon Photos has a 5GB limit for video files.

      $ cat over-quota.txt | awk -F '.' '{print $NF}' | sort | uniq --count
   149 mp4
  • Create a new album to add them to
  • Namne album Google Photos
  • Save
  • Done!

Not sure what to do about the excess videos for now. Probably most of them are WhatsApp junk that can be deleted, but I’ll need to go through them.

There are 801 MP4s in the Google Fotos export.

$ find ~/Descargas/Takeout/Google\ Fotos/ -name '*.mp4' | wc -l
801

The total size of all the videos is 6,3GB.

$ find ~/Descargas/Takeout/Google\ Fotos/ -name '*.mp4' -printf '%s\n' | datamash sum 1 | numfmt --to=iec
6,3G

I can find the over-quota videos in the export like this.

$ find ~/Descargas/Takeout/Google\ Fotos/ -name '*.mp4' | grep --file over-quota.txt
/home/isme/Descargas/Takeout/Google Fotos/Photos from 2020/VID_20200924_124818893.mp4
/home/isme/Descargas/Takeout/Google Fotos/Photos from 2020/VID_20200123_011704_688.mp4
/home/isme/Descargas/Takeout/Google Fotos/Photos from 2020/VID_20200315_074648_545.mp4
[...]

I disabled video uploads in the Amazon Photos app.

I kept video uploads enabled in the Onedrive app.

In the end I didn’t copy any of over-quota videos to Onedrive.

Instead I will sort all the files in Google Photos into broad categories, delete the ones I don’t need, and use Amazon Photos for everything from now on.