The Cache

A cache is a chunk of faster memory that holds information for faster retrieval than would otherwise be possible. One analogy would be to the cash you have in your pocket - it's faster and easier to access and spend than money in the bank.

There are several places a computer will cache data. Some software caches information from CD-ROM disks to your hard drive. Since hard drives are typically faster than CD players, caching the CD-ROM's directory can speed access to data. Similarly, by setting aside a section of memory to store some data from your hard drive, caching can improve hard drive performance.

In the same way, compared with CPU speed, system memory is slow. By caching data either on the CPU itself or in a special card with high speed memory, your CPU won't be hobbled by slow system memory.

As CPU speeds passed 16 MHz, they began to outrun the computer's memory. One way to address this was to use faster system memory, which was always more expensive than slower RAM. Another way was to tell the CPU it would have to wait for the data it requested. (You may have heard the term "wait state" - that measures how many processor cycles the CPU has to wait before it received the data it has requested.)

Another solution is to cache a small amount of data. The 68020 and later CPUs were designed with a little high speed cache right on the CPU. This level-1 (L1) memory could hold several instructions and pieces of data so the CPU wasn't hobbled by slower system memory.

Still, despite the cache, the speed difference between L1 and system memory still meant every time the CPU needed to access system memory, it had to request the data, then wait for it. A small L1 cache was nice, but it wasn't enough.

A level-2 (L2) cache is usually fast (25ns or faster) RAM. Cache size can range from 32KB to 1 MB. There is no hard and fast rule about cache size, but the general rule is the faster the CPU relative to system memory, the larger your L2 cache should be.

Although the 68030 had two 256-byte caches, the speed differential between the 25 MHz CPU and the 80ns SIMMs kept the IIci from achieving its full potential. A relatively small 32KB L2 cache would boost performance 20-30%. Larger 64KB and 128KB caches were made, but each doubling of cache size only added about 2% more speed. With high speed, high cost memory, the larger caches were less cost effective than the smaller ones. A 32KB cache was enough to provide a significant boost without exorbiatnt cost.

Because they had a larger L1 cache (8KB v. 512 bytes), none of the 68040-based Macs came standard with L2 caches, although 128KB cards were offered by some manufacturers. As with L2 cards on the IIci, these provided a 20-30% performance boost.

The L2 cache really came into its own with the Power PC. Despite a 32KB L1 cache on the 601, a relatively inexpensive 256 KB cache would boost performance 15-30%. As CPU speeds increased, larger 512 KB and 1 MB caches became popular.

The 603 had two smaller caches (8KB for instructions, 8KB for data), making the L2 cache even more helpful. The souped up 603e had an improved cache (16KB each for instructions and data), but because of its still greater speeds (100 MHz and up), the L2 cache was still a big plus. As CPU speeds reach and exceed 200 MHz, a still larger L2 cache becomes important. It's not unusual to see 512 KB and 1 MB caches on 200 MHz and faster computers.

Like the 603e, the 604 had a dual L1 cache (16KB for instructions, 16KB for data) and benefitted greatly from a 256 KB or larger cache. The improved 604e doubled the size of the instruction and data caches, significantly improving performance. Still, a 512 KB or larger cache boosted performance.

Until this year, all Mac caches were located on the motherboard. But the motherboard runs at 50 MHz or slower, creating a real bottleneck for system RAM and the L2 cache. The hot new technologies are implemented on the new PPC 750 chip: inline and backside caching.

An inline cache sits between the motherboard and CPU; Umax is implementing this under the name Cache Doubler. At this point, the only inline cache operates at twice the speed of the motherboard. This give the CPU access to the L2 cache twice as quickly as a motherboard L2 cache.

A backside cache uses very fast memory that is essentially coupled to the CPU. The advantage is still greater speed, but at the cost of incredibly fast cache memory. This is being implemented on the fastest new models from Apple, Power Computing, and Motorola. The biggest advantage of the backside cache is that it is directly related to CPU speed, not motherboard speed.

Other Resources

• MacTips, The Disk Cache
• The fundamentals of cache, Paul Mazzucco, SystemLogic. More than you ever wanted to know about L1, L2, and L3 cache - and pretty accessible.