With Lightroom 2.7 and 3 Beta 2 both receiving a limit boost, now is a good time to talk about the Camera Raw Cache. So what is the the Camera Raw Cache anyway? Well in Lightroom, Develop uses a different preview system to Library. The Library Previews are created either on import via the Preview options, or as required (e.g. when you zoom into 1:1). In Develop, Lightroom goes back to the original file and creates the previews from scratch and applies whatever settings specified by the user. As you move to the next file and back, Lightroom again regenerates this preview. While this sounds tedious, you need it for accurate rendering, so that the changes you make in Develop will be what gets exported as your final version for further use.
Surely there must be some parts of this process that are the same each time? Well in fact there are. The early part of the process includes decoding, decompressing, linearizing, hot pixel filtering, and demosaicing an image. The results of this are the same each time, so for this reason, it’s perfectly legitimate to use a cached version of this as the basis for a preview. To help speed up the workflow in Develop, this partially processed version is stored in the Camera Raw cache.
How does this work in practicality? Well, the first time you open an image in Develop, it must do a full render. The partially processed version is then stored in the Camera Raw cache, for the next use, so when you return to the image, the creation of preview takes about half the time. The next question I would ask is, How can this benefit me? Well the default Camera Raw cache setting is only 1Gb. That’s equivalent to about 200 5DMkII Tiff files. The engineering team feels that it’s a good balance between space and speed. The maximum used to be 50Gb, but is now 200Gb, so obviously power users deemed a need for a much larger space. If you have spare space, I’d recommend increasing the cache size. Try 20Gb for a start, and if you are going back and forward over a lot of images in Develop, go even higher. Of course, it’s not just Lightroom that makes use of this cache, as the name implies, Camera Raw uses this also, so previews are shared between the 2 applications.
To change it open Preferences (Lightroom>Preferences on Mac, Edit>Preferences on PC) and click the File Handling tab. You should see a dialog like above at the bottom of the Preference dialog. You can see there’s 2 options. The 2nd one is what we’ve been talking about. If you have a spare internal drive, you can use the first preference to set the location of the cache. This way the catalog and cache (and possibly the images, depending on how you work) are on different drives, so Lightroom is not accessing the catalog and the cache from the same read head on one drive. It’s also a good idea to keep the cache off of your startup drive, if you’re using a large one. 200Gb is still a large chunk of space on a 1.5TB drive. The final option is the ‘Purge Cache’ button, which clears out the cache.
So what happens when the cache fills up? Easy, Lightroom just removes the oldest images from the cache, to make way for new ones.
Obviously you don’t need to use 200Gb, but I feel that even a modest increase from the default should make Develop zippier for most users. I know a few people on Twitter have thanked me for the recommendation.
Thanks to Eric Chan for information relating to what the cache stores exactly.
I’m revisiting the concept of Camera Raw Cache because moving from image to image in the develop module is frustratingly slow (though I admit I’m not a patient person staring at loading images hundreds to thousands of times in an editing session). I have my Camera Raw cache set to 25GB, and am currently using only under 5GB of it with 13,971 files in the folder (including the Index.dat file). That equates to an average file size of approx. 350KB per .dat file which is smaller than the expected size for 5DMkIII 21MP images even if they are develop previews. With the mentioned 200 images for 1GB of cache, each image .dat file should be about 5MB (assuming roughly that 1 GB = 1000 MB).
Is the small size of the individual .dat files a potential reason why my image-to-image speed in the Develop module so slow?
Probably not. Adobe made a change so the cache files became JPEG based rather than TIFF for faster loading. Your initial read of a file will generate the cache file, so the benefit is in returning to the file. If you’ve Smart Previews generated, it will be faster… and if the original files are offline, it’ll be faster again with smart previews- a little counter intuitive, but at the expense of a proper 1:1 view.
What’s your OS/RAM? Files and catalog are on internal drives? You’ll benefit from using an SSD for the catalog.
Hi Sean, appreciate the reply, esp. so many years after writing this article! Great to understand that the cache is now JPG so the small file size is explained. With that, I’d argue allocating over 10-15GB for the LR cache is overkill.
I’m running on a desktop, Win7 64-bit on an i2500k OC’d to 4.5GHz, 16GB of RAM, OS on an SSD, LR Catalogs and LR Cache on a separate SSD. There literally should not be reasons LR is so darn slow. I’ve been trying to figure it out for years now.
I can see that my CPU utilization goes up (almost to full) when going between images in the Develop module, even when scrolling back and forth between the same 2-3 images, which should totally be cached (I waited for the full render the first time on each photo so cache preview generation wasn’t interrupted).
It depends on how many files you process and how often you go back to said files as to how big a CRC you need. I revisit files all the time, so it benefits me to have it larger. The main bottleneck with Lightroom is disk read speed. The SSD takes away some of the pain, but ultimately when you develop a file you’re using the processor to the fullest. That’s the whole point of getting a better processor. The Cache holds the demosaic info, but all the settings are applied fresh-these are not cached; a single slider movement would render it invalid. Process Version 2012 is quite processor intensive, as are Lens Corrections, adjustment brushes, noise reduction and Spot Removal (Especially on lens corrected files).
That said, there are systems that are slower than expected and unfortunately there is no straight answer as to why. Have you tried creating Smart Previews or using DNG with Fast Load Data?
I’ve tried creating Smart Previews but haven’t spent too much time with them. I’m assuming Smart Previews are fastest when I purposely remove the original files from the picture (rename the folder, move the files, etc.). I’m constantly working with adjustment brushes at 100% though so this may not be a solution that benefits me though I’m sure it should be much faster. Smart Previews don’t seem to be useful if you don’t purposely remove the original files though, am I incorrect in this assumption?
I haven’t tried DNG with fast load data, this may be the next thing to revisit. It’s been a while since I’ve read up on the pros/cons of the DNG format. Thanks again!
I did change cache size to 8gb. However, when I go back to preference, the number went back to 1.0gb.
I still have 500gb of harddisk space.
Why can’t I change this number to higher than 1.0gb???
Honestly, I’ve never seen it do this. Maybe your preference file is corrupt?
Very clearly explained. Many thanks for that.
Although I have my Lightroom catalog and all photo files on my 2TB prime external drive, so that I can change over from pc to laptop for travelling very easily, I just somehow overlooked the option for an alternative location for the Camera Raw Cache. This means that (without realising it) at the moment, I have two different caches: one on the pc, and the other on the laptop.
I see the benefit, mentioned above, in not having the Camera Raw Cache on the same drive as the Lightroom Catalog, but in my case, I feel I’ll have to, with this constant switching back and forwards between computers.
I’ll give it a try, anyway.
I’m also a tad weary of the glossy screen. That said, If I could put an ssd in an imac and connect an external storage device via esata, I’d buy one right now.
Some people are reposting success with an SSD for the catalog. Someone else was asking about moving the previews file off an SSD (search for Symbolic Link to see how).
As to the difference with the iMac and MacPro? I’ve had an older iMac and the MacPro and the MacPro was worth it. My main issue the the iMac right now would be the glossy screen. I’d have no issues with the power. I am happy to have additional internal drives though.
Thanks for the quick reply, Sean.
I definitely want the slowdown during the import / preview creation phase. I do most of my importing / exporting overnight. Honestly, all I care about is how response LR is while I’m culling and editing.
I’ve been trying to do some research into what the best LR setup would be. Whether an SSD drive makes a worthwhile difference… whether a mac pro is worth the premium over the i7 imac. It seems surprisingly difficult to get a clear cut answer.
Anyways, thanks again for the answering my question. It’s definitely 1:1 previews for me from here on in.
Thanks for the great post! Quick question:
So if I understand correctly, LR doesn’t generate previews in the develop module until you actually navigate to the image (while in the develop module). Is there any way to get LR to generate all of these previews ahead of time? I’m often editing hundreds of photos in a sitting, and if i could cut down the loading time between photos I’d be one happy guy.
Thanks again!
I’ve just purged my Cache via prefernces. I’ve then gone to an older folder and selected all, then I’ve gone to Library>Previews>Render 1:1 Previews. Lightroom scans for previews. Then it starts creating previews. As I looking at my Camera Raw Cache, I can see that for every 1:1 Preview being created, I have a Camera Raw Cache .dat file created.
Now rendering 1:1 files is time consuming, but it also fills cache. The question is where you want the slowdown: As you go, or before you begin.
One thing I don’t understand is how come a demosaiced 5DII file be so small.
I thought 21000000 x 3(RGB) x 16(bpp) = 120 MB.
That is 8 files per 1 GB, yet a 1 GB cache fits 200 such files.
Where am I wrong?
Hi Dorin,
The files aren’t fully baked and are still previews, not TIFFs.
i got a 7400rpm…but sometimes above all when i use the brush in LR it’s a bit slow: start to load every single movements with the multicolour wheel :D…i think it sucks a lot of RAM…anyway thanks a lot for advice 😀
The issue becomes one of portability here rather than convenience. You’ve 2 options: put catalog and images on the external, or put catalog on internal and images on external. It depends on whether you always have the external connected, or don’t mind working with offline previews. Generally internal laptop disks are only 5,400rpm, so it’s going to be slower because of that anyway.
ok thanks a lot, but to be more fussy :D…i got a mac book pro with a 320 gb hd and i have just one partition. The other hd external by firewire800 it’s 1 tera….what do u suggest to do ? thanks a lot for your help 😀
hiya…so u suggest to have cache in an other Hard drive and pictures and catalog in other. Basically i got all in my Hard drive and after a while i notice that lightroom slows down. I raised the cache to 20 GB as u suggest, but what about the location of picture, cache and catalog. What is your best set up of this 3 elements> thanks a lot for the article very useful.
In general fast internals is the best choice, so using internal drives, have the cache on 1, Catalog/Previews on another and photos on the final. Really though it’s not critical if the cache is on the same drive as previews, but it does mean the read head of the drive is jumping around more.
When you said “The Library cache is unlimited in size and generally lower quality”, what you mean exactly for unlimited? Is that cache growing indefinitely until my HD is full? I hope not 😀
Actually my catalog preview cache is about to 5GB, should I delete all files and folder under it? or there is a parameter that control how it grow?
Unlimited=nothing to stop it getting bigger. As you add an image previews will be generated. The only parameters that control it are in the Library previews menu. All deleting the folder will do is force Lightroom to generate more replacement previews as you sit around waiting on it to complete the task. I don’t recommend it.
Hello Sean,
I am using Lightroom version 1.4.1. Is it possible to increase the size of the cache in this version? I cannot figure out where to do it!
Thank you so much!
I think it needs to be done via Camera Raw in 1.41. Open Bridge and click on Bridge>Camera Raw Preferences (Mac), or Edit>Camera Raw Preferences (PC).. just guessing it’s in Edit on PC..
Camera Raw Cache is the third section. I really don’t remember how much effect it has in LR1, feel free to report back!
Sean,
Thanks for the reply. I do appreciate the extra information you’ve provided.
Thanks for the tip Sean
Interesting post this explains why it takes a few seconds for the image to render on the screen, especially raw files from my Canon 7D. I do have a question or two though.
(1) In library mode when are previews actually used? Is it just when you zoom to 1:1 or are they used when you move from image to image? I’m thinking since I work primarily in grid view mode when in the library module that I may not need to generate previews when importing images.
(2) Why are there two different caches? Why couldn’t the library and development module share the same cache?
1) All the time in Library for everything. You’d find Library impossibly slow without them.
2) The Library cache is unlimited in size and generally lower quality. A full blown preview cache of all your images at 1:1 develop size would eat your hard drive.