Sunday, May 03, 2015

Problems uploading to Apple Photos in iCloud

Update 2015-05-07: Example steps to get video uploading working again. And again.

Apple's Photos service can't upload my 39,000 photos and 2,000 videos. I'm using this page to document the reasons as I find them.

High Upload Rates, then High CPU Rates

About nine days ago, I enabled iCloud syncing of my photos and videos, and the process ran at high upload rates for quite a while. That made sense: it has to send the data to the Apple servers.

But then the High CPU load appeared, and with Activity Monitor and viewing the Open Files, I found that the CPU-intensive processes were reading my video files. Transcoding! I suspect Apple is transcoding my videos to a format they prefer before sending them to their servers.


I'm guessing the transcoding is called "modernization", because of this message.

May  3 17:00:04 crwth.local cloudphotosd[5552]: Failed to generate video derivative for  file:///Users/mark/PERSONAL/Pictures/Photos%20Library.photoslibrary/private/ public.mpeg-4 CPLResourceTypeVideoFullSize {0, 0} bytes=0 temp=1 (RDMaster(0x7f9b504ed530) modelId=45254 uuid=MdoAmLnJT46sd4v99Sl3Ew tableName=RKMaster state=persisted,local VID00009_20111021_152554EDT.AVI) status: 2 error: Error Domain=PAMediaConversionServiceErrorDomain Code=2 "Unable to initialize modernized asset, modernization timed out" UserInfo=0x7f9b507b3670 {NSLocalizedDescription=Unable to initialize modernized asset, modernization timed out}
Fortunately this log message gives a filename. In my case, the problem files seem to be .AVI files, so I got the list of problem filenames in Terminal this way:
# 2015-05-03 22:50:27 mark@crwth ~$ grep ",local.*modernization timed out" /var/log/system.log | sed 's/.*local //' | cut -d')' -f1 | sort -u
This gave me something I could deal with. In Photos, I could search for these videos, and found that they're just not in a format Apple likes.

This is a file I normally view with VLC. 

So my next job is to find each of these files and move it out of the Photos library. I used this method in Apple Terminal. The idea is that it each video appears in an iPhoto Masters directory, and that we want to get that one and copy it to a new directory.
mkdir ~/problem_videos_photo_import
(cat /var/log/system.log ; gzip -dc  /var/log/system.log.?.gz ) | 
grep ",local.*modernization timed out" | 
sed 's/.*local //' | 
cut -d')' -f1 | 
sort -u | 
while read f
  mdfind -name "$f" | 
  grep Masters | 
  while read p
    cp "$p" ~/problem_videos_photo_import

Having run this, I have a tidy directory of all my problem videos.

Now I can search for each one in Apple Photos, and delete it:

I noticed that each of the problem videos appear twice. I deleted both.

More problem .AVI files

Even after deleting this batch of problem .AVI files, I kept finding high-CPU processes that were chewing on .AVI files for hours. It doesn't take hours on a Late-2013 4-core MacBook Pro to transcode a 3-minute video. Ultimately, I searched ~/Pictures/Photos\ Library.photoslibrary/ for VID000*.AVI files and moved them to a new directory. These were files I had created on a Flip Video camera. This is what I did in Terminal:
mkdir ~/problem_videos_photos_import
find ~/Pictures/Photos\ Library.photoslibrary/ -name 'VID000*.AVI' | grep Masters |
while read p
  mv "$p" ~/problem_videos_photos_import
This isn't a clean method, because I'm deleting the data out from underneath Photos and all the associated software.  But I'm relying on error recovery for missing files to work better than transcoding unsupported file types works. It turns out this is a safe bet. A couple of reboots later (which might have helped it to re-index the available data and restart the iCloud upload process), high-speed uploads have resumed.

In Activity Monitor, you can see that nsurlsessiond and cloudd have sent the most data since bootup. And my sending speed is quite high, over 750 kbps i.e. 6.1 Mbps: that's the red area you see below the line.

"Put container ... url expired."

Another error in the logs is shown below.

May  3 20:13:19 crwth.local cloudd[65369]: Progress of MMCS item 2850: s:7 p:-1.00 error:Error Code=7 "Put container(s) failed." UserInfo=0x7f88c97c3960 {NSDescription=Put container(s) failed., NSLocalizedDescription=Put container(s) failed., NSUnderlyingError=0x7f88cbe23b60 "Put container 0x7f88c965b3e0 url expired. Retry immediately"}