Part II: 4GB vs. 8GB vs. 16GB
We weren't satisfied with the results in our initial test. We've been running all our benchmarks on a 16GB system for the longest time - surely we did that for a reason, right? To get to the bottom of this question, we ran seven more benchmarks, on a higher-end test system, this time focusing on built-in timedemo benchmarks. We had a hunch that any minor hitches related to RAM shortages would be masked by test-to-test variability inherent in our in-game benchmark runs in Part I of this test. Our test system for this part of the test consisted of an Intel Core i7-3770K@4.0GHz, a Asus Maximus V Gene motherboard, an EVGA GeForce GTX 670 FTW 2GB video, and Samsung 4x4GB, 2x4GB, or 1x4GB DDR3@1866, running Windows 7 x64.
Again, we tested all games at a resolution of 1920x1080, and each test result was averaged over three runs of the built-in benchmarks. We collected results for all of our tests using 4x4GB, 2x4GB, and 1x4GB configurations. Note that the 4GB configuration is at a disadvantage because it is running in single-channel mode. This is a shortcoming of using 4GB sticks to test with, but that's what we had on hand, and it was necessary to reach 16GB in any event.
Here's what we tested, followed by the results:
- Batman: Arkham City (Benchmark) - DX11, Extreme Settings, 4xAA, no PhysX
- Dirt 3 (Benchmark) - Ultra Settings/4xAA
- Civilization V (Late Game View Benchmark) - High Settings, 4xAA
- Metro2033 (Benchmark Utility) - Very High Settings, 4xAA
- Hitman: Absolution (Benchmark) - Ultra Settings/4xAA
- Tomb Raider (Benchmark) - Ultimate Settings/FXAA
- Bioshock: Infinite (Welcome Center Benchmark) - Ultra DX11, DDoF
Well, no big surprises here, it seems. On the left, we have Batman: Arkham City, and it looks like the 16GB rig is actually slower than the other two. This benchmark tends to be somewhat imprecise, so we'll just chalk that up to testing variability. Dirt3 shows a delta of less than 1 percent for all tests, so we'll call it a three-way tie. Of more concern is the result on the right - Civilization V. We're pretty sure that the slower speed of the 4GB test rig is not an anomoly. The average using 4GB was 100.3fps, compared to 104.5fps for the higher RAM amounts. We think this may be a legitimate difference, and while it could be from lack of RAM, it's more likely due to the reduction of memory bandwidth - remember, the 4GB system is running with just one memory stick, and is thus stuck in single-channel mode. Civilization V's "Late Game View Benchmark" requires serious computational power, and may be drawing on significant amounts of memory bandwidth. Whatever the real reason for the delta is, because this benchmark is quite repeatable, we think this is an actual difference in performance.
On to some more benchmarks:
This set of benchmarks is in game engines that tax the video card a bit more than the first set of games. Metro2033's punishing benchmark has the 4GB system 2 percent slower than the other two - that could be related to RAM, but it's certainly small. Hitman oddly has the 4GB system faster than the other two - again showing that benchmarking is part art and part science - even these built-in benchmarks have test-to-test variability. Then we have Tomb Raider, in which our run through the actual game world in Part I of our tests produced what appeared to be a 4 percent advantage for the 8GB system over the 4GB system. Well, there's no advantage here - these three systems are tied! Our guess is that this benchmark, which is heavily-skewed towards showing off graphics features, isn't stressing the memory bandwidth as much as our real-world run.
Bioshock: Infinite puts up results that show a different pattern than any game run previously. There appears to be the first indication that 16GB might, just might, have an advantage over 8GB, at least in the minimums category. The minimums were all over the place in this benchmark, but based on our three runs at each setting, we can definitely say the 16GB system had the most consistent minimums. Alas, the average frames per second shows a lot less variability, with the 4GB rig consistently hitting 69.1fps, and the other two hitting 69.8. Statistically significant? We're not sure, but those minimums definitely look like they might be. [Update: After many more runs of the Bioshock benchmark for subsequent articles, we've learned that the minimums are greatly affected by hard drive access times due to the frequent scene changes in this benchmark, and our 16GB system likely was impacted the least by this. We doubt the effect would be so pronounced in an actual gaming scenario where scene changes are less frequent.]
Phew...that was a lot of benchmarking! Two systems, ten different games, three memory amounts, and lots of runs of each benchmark. And what can we conclude? Well, it seems that the initial findings based on our first test system hold true - RAM amount simply doesn't make a big difference for performance - we witnessed at most a 4 percent difference in average frames per second jumping from 4GB to 8GB, and no boost to average FPS jumping to 16GB. Furthermore, part of the 4GB systems' deficits could have been due to running in single-channel mode. Based on our findings, we're confident saying that if you're on a tight budget, whatever money you spend on more than 4GB would definitely be better spent on the next level of video card, or perhaps a quad-core instead of a dual-core processor.
But there is a caveat, and it's a big one - you really have to "manage" your RAM more with 4GB. First of all, you'll want to limit background tasks. All of our tests were performed with the minimum of applications running in the background - if you tend to have a number of programs running while gaming, having just 4GB may be an issue. Also, keep in mind that you need to allow Windows to run a substantial page file when you have 4GB - our attempts to benchmark with the slim 1GB page file we use with 16GB of RAM led to running out of virtual memory upon launching some of our games with only 4GB of physical RAM.
So what's our advice? 8GB is probably the safest bet, not because it adds much to performance, but because it will allow you to do more with your system. As for 16GB - we'll stick to our guns here. We might run it in our test rig, but in no way, shape, or form do we recommend it to users who are only interested in maximizing gaming performance. The extra $60-70 is much better spent on other components. Maybe one day, if memory drops back down to $4/GB, which it hit in December 2012, we'll say "RAM is cheap, get 16GB," but until then, 8GB is our target for any reasonably-priced gaming system.
UPDATE: How well do games play on 8GB and 16GB of DDR4 memory in 2016? See our feature-length DDR4 Gaming article to find out!