cache

A basic problem in computer design is how to optimize the fetching of instructions or data so that it will be ready when the processor (CPU) needs it. One common solution is to use a cache. A cache is an area of relatively fast-access memory into which data can be stored in anticipation of its being needed for processing. Caches are used mainly in two contexts: the processor cache and the disk cache.

CPU Cache

The use of a processor cache is advantageous because instructions and data can be fetched more quickly from the cache (static memory chips next to or within the CPU) than they can be retrieved from the main memory (usu-ally dynamic RAM). An algorithm analyzes the instruc-tions currently being executed by the processor and tries to anticipate what instructions and data are likely to be needed in the near future. (For example, if the instructions call for a possible branch to one of two sets of instruc-tions, the cache will load the set that has been used most often or most recently. Since many programs loop over and over again through the same instructions until some condition is met, the cache’s prediction will be right most of the time.)

These predicted instructions and data are transferred from main memory to the cache while the processor is still executing the earlier instructions. If the cache’s predic-tion was correct, when it is time to fetch these instructions and data they are already waiting in the high-speed cache memory. The result is an effective increase in the CPU’s speed despite there being no increase in clock rate (the rate at which the processor can cycle through instructions).

The effectiveness of a processor cache depends on two things: the mix of instructions and data being processed and the location of the cache memory. If a program uses long sequences of repetitive instructions and/or data, caching will noticeably speed it up. A cache located within the CPU itself (called an L1 cache) is faster (albeit more expensive) than an L2 cache, which is a separate set of chips on the motherboard.

Changes made to data by the CPU are normally written back to the cache, not to main memory, until the cache is full. In multiprocessor systems, however, designers of pro-cessor caches must deal with the issue of cache coherency. If, for example, several processors are executing parts of the same code and are using a shared main memory to commu-nicate, one processor may change the value of a variable in memory but not write it back immediately (since its cache is not yet full). Meanwhile, another processor may load the old value from the cache, unaware that it has been changed. This can be prevented by using special hardware that can detect such changes and automatically “write through” the new value to the memory. The processors, having received a hardware or software “signal” that data has been changed, can be directed to reread it.

Disk Cache

A disk cache uses the same general principle as a proces-sor cache. Here, however, it is RAM (either a part of main memory or separate memory on the disk drive) that is the faster medium and the disk drive itself that is slower. When an application starts to request data from the disk, the cache reads one or more complete blocks or sectors of data from the disk rather than just the data record being requested. Then, if the application continues to request sequential data records, these can be read from the high-speed memory on the cache rather than from the disk drive. It follows that disk caching is most effective when an application, for example, loads a database file that is stored sequentially on the disk.

Similarly, when a program writes data to the disk, the data can be accumulated in the cache and written back to the drive in whole blocks. While this increases efficiency, if a power outage or other problem erases or corrupts the cache contents, the cache will no longer be in synch with the drive. This can cause corruption in a database.

Microsoft’s Windows Vista introduced an ingenious type of cache at the system level. The “ReadyBoost” features allows many inexpensive USB flash drives to be used auto-matically as disk caches to store recently used data that had been paged out of main RAM memory.

Network Cache

Caching techniques can be used in other ways. For exam-ple, most Web browsers are set to store recently read pages on disk so that if the user directs the browser to go back to such a page it can be read from disk rather than having to be retransmitted over the Internet (generally a slower pro-cess). Web servers and ISPs (such as cable services) can also cache popular pages so they can be served up quickly.

Further Reading

Nottingham, Mark. “Caching Tutorial for Web Authors and Web-masters.” Available online. URL: http://www.wdvl.com/ Internet/Cache/index._html. Accessed May 24, 2007.