Stephen Shankland from CNET has an interesting post that’s been doing the rounds. Lightroom Product Manager Tom Hogarty referred to it as “Not my favourite headline this year” on Twitter. The post relates to Export not being as quick as it could be and is partially based on miscommunication, which has since been updated. Of course the furore over the post is in full swing, and a lot of the earlier readers may not yet realise there’s been an update to the post.
Stephen indicates that it started with an older post from Lloyd Chambers on getting better performance in Lightroom. The post did the rounds a while back with Scott Kelby weighing in on it. One of the outcomes from Lloyds post was the revelation that running 3 smaller exports is faster than one larger export with the same total number of images. Seems odd that it should work, but it does.
Stephen approached Tom Hogarty for an explanation. Essentially Tom’s response equates to saying that they want Lightroom to be capable of doing other tasks while Export is running as a background task. And that, in my rather humble opinon, is a good idea. Other programs (Aperture for example-edit version 1 did this) give total control to the exporting process, rendering the program useless for other tasks. Often I proceed with new tasks, like image tagging or development on a different image set while an Export is happening. I’m rarely worried about how long it takes under these circumstances, and am more than happy to trade a slower export with program usablility.
There are obviously times when a speedy export is needed. In these circumstances I can use Lloyds speed trick, but it would be nice if it could be automatically set in the export dialog. As each task is alloted it’s own set of resources, the export can happen quicker. With Import, I’ve often had 3 cards on the going with an import session. Import one card, then when the previews are rendering, import the next and so on. I end up with an import and 2 preview renders running consecutively. It doesn’t seems to phase Lightroom and I’m happy not to be waiting any longer with card swapping.
On the performamce matter, Lloyd weighs in again: “Today is not 2006, Adobe. I have 32GB and 16 virtual cores in my Mac Pro Nehalem, I paid a ton of money for that potential, and I did so to save my valuable time, not to watch the machine idling (which is mostly what it does in Lightroom and Photoshop).”
It’s a more than fair point. When the user is doing task that are processor intensive, Lightroom should make more use of the machines ability. There will always be users on lower power machines, but this shouldn’t prevent those who have paid for the privilege of having a high powered machine getting full use out of it.
Here is a technical explanation of why 3 exports run faster than one, and why at some point even more exports aren't yet faster. This is based on standard programming techniques called multi-tasking or in this case multi-threading more likely.
Essentially every computer has a number of resources – disk, memory, processor, network. Every time an application does something it uses some of these resources, but usually it does not consume 100% of each, nor does it use them all at once. For example, Lightroom may be creating the exported file in memory, doing JPG compression which uses a lot of the memory and processor capacity, but does not use the disk at all. Once done, it has to save this to disk, which uses memory and disk, but not as much of the processor. You get the idea.
Essentially what Lightroom does, is by allowing exports to run in the background it enables more than one export to share resources at any given time. An application that runs exports as an exclusive task may actually not do that, and thus not be faster (plus you are force to watch, instead of working on the next thing).
What happens when you run a 2nd or 3rd export is that those exports run on whatever is left over of the resources. If one export is currently saving the already converted image to disk, while the first one is converting, they're not using the same resources, and can actually accomplish more in the same time. And even if they do the same thing all at once, they most likely use up some of the slack in the resources.
The reason more than 3 export don't keep running faster yet, is that there is no more slack in the system left. At least one or two of the resources are now totally maxed out at 100% all the time. The additional exports just keep sitting there waiting until it is their turn. In fact if you started 500 exports there is a chance it could even become slower as the overhead of switching back and forth actually adds work.
To understand why having 16 cores doesn't make things yet faster, one would have to analyze the exact resource utilization of Lightroom on that Mac Pro. The 16 cores don't help if the disk is at 100% after 5 exports running in parallel.
There is of course a lot more detail to this, but that paints the main picture.
Disclaimer: I'm not related to Adobe, have not worked on Lightroom code or am familiar with its particular design. But I have spent decades in the software industry. This is general knowledge, not anything unique to Lightroom or Adobe.