quad rank 533mhz vs dual rank 800mhz

Hi Everyone

I'd be grateful of anyone's input on this. I'm looking to get a Dell
PowerEdge 2970 2 x Quad Core Opteron. When it came to specifying the
memory there was a huge difference in price -

32GB 800mhz dual rank comes at machine cost where as 32GB 667 clocked
to 533 comes at a collosal £1680 on top with 64GB coming in at £4800
on top.

I was after anyone's opinions as to what the better option is in this
scenario.

Thanks
Bryan

wrote:
Hi Everyone

I'd be grateful of anyone's input on this. I'm looking to get a Dell
PowerEdge 2970 2 x Quad Core Opteron. When it came to specifying the
memory there was a huge difference in price -

32GB 800mhz dual rank comes at machine cost where as 32GB 667 clocked
to 533 comes at a collosal £1680 on top with 64GB coming in at £4800
on top.

I was after anyone's opinions as to what the better option is in this
scenario.

Thanks
Bryan

The Dell customize menu for Poweredge 2970

1GB DDR2, 667MHz, 2x512MB Single Ranked DIMMs [subtract $109]
2GB DDR2, 667MHz, 2x1GB Single Ranked DIMMs [subtract $80]
2GB DDR2, 667MHz, 4x512MB Single Ranked DIMMs [subtract $70]
4GB DDR2, 667MHz, 4x1GB Single Ranked DIMMs [Included in Price]
4GB DDR2, 667MHz, 8x512MB Single Ranked DIMMs [add $125]
4GB DDR2, 667MHz, 2x2GB Dual Ranked DIMMs [subtract $28]
8GB DDR2, 667MHz, 8x1GB Single Ranked DIMMs [add $291]
8GB DDR2, 667MHz, 4x2GB Dual Ranked DIMMs [add $70]
8GB DDR2, 667MHz, 2x4GB Dual Ranked DIMMs [add $69]
16GB DDR2, 667MHz, 4x4GB, Dual Ranked DIMMs [add $431]
16GB DDR2, 667MHz, 8x2GB, Dual Ranked DIMMs [add $428]
32GB DDR2, 667MHz, 8x4GB Dual Ranked DIMMs [add $1,023]
4GB DDR2, 667MHz, 2x2GB Dual Ranked DIMMs, Energy Smart [subtract $28]
8GB DDR2, 667MHz, 4x2GB Dual Ranked DIMMs, Energy Smart [add $108]
16GB DDR2, 667MHz, 8x2GB, Dual Ranked DIMMs, Energy Smart [add $509]
32GB DDR2, 533MHz, 4x8GB Quad Ranked DIMMs [add $6,293]
May delay your PowerEdge 2970 ship date
64GB DDR2, 533MHz, 8x8GB Quad Ranked DIMMs [add $13,524]
May delay your PowerEdge 2970 ship date
8GB (2x4GB), 800MHz, Dual Ranked [add $238]
4GB (2x2GB), 800MHz, Dual Ranked [add $108]
2GB (2x1GB), 800MHz, Dual Ranked [add $61]
8GB (8x1GB), 800MHz, Single Ranked [add $311]
16GB (4x4GB), 800MHz, Dual Ranked [add $576]
8GB (4x2GB), 800MHz, Dual Ranked [add $130]
32GB (8x4GB), 800MHz, Dual Ranked [add $1,185] -------------
4GB (4x1GB), 800MHz, Single Ranked [add $111]
16GB (8x2GB), 800MHz, Dual Ranked [add $576]

I don't know how much memory you need, but I'd be selecting the one
I marked with an arrow. If you really need 64GB, then there is only
one option listed for that. And you wouldn't buy the memory from
Dell, you'd price around. Crucial would want $1600 per two DIMM kit,
for a total of $6400 for 64GB of memory.

http://www.crucial.com/store/mpartsp...158EFBA5CA7304

In terms of what the processor supports, certain documents are not
available from the AMD web site. For example, to support quad rank,
presumably there has to be a way of addressing them. I'm surprised
some motherboards support four slots per processor (yours), while others
are providing eight. Looking at the CPU pinout, may hint at whether
such options, come at a price in terms of how the processor can be
used. Maybe ones with eight slots per processor, only support dual
rank.

I think your machine looks like this.

Opteron 2350 Opteron 2350
| | | |
DIMM DIMM DIMM DIMM
| | | |
DIMM DIMM DIMM DIMM

First of all, it appears the memory is "registered ECC". ECC means
the memory is 72bits wide, versus the "normal" 64bits wide memory
of an unprotected memory product. The 8 extra bits store a "checksum"
of sorts.

One little trick you can do with ECC DIMMs, is combine the two
DIMMs in dual channel. This makes a 144 bit wide memory array.
Something called "Chipkill" encoding can be used for that. The
16 extra bits of checksum, allow identifying a burst of 4 bits
in error. If the memory chips are each 4 bits wide, this allows
a memory chip to be ripped off one of the pair of DIMMs, and the
computer continues to run. There may be a slight performance
penalty for doing this, as error correction cycles must be
attempted for accesses to a crippled pair of DIMMs. (I've sat
in front of an older computer with a form of protection like this,
when it failed, and the log tends to fill up with trouble reports,
which slows the machine quite a bit.)

So, knowing we have Chipkill capability, what would be a useful
memory configutation ?

First, we start with a single rank of memory. For the DIMM, we
need to make a 72 bit wide matrix. We can do that with 9 chips which
are 8 bits wide each.

A ------+--+--+--+--+--+--+--+--+ Address bus
| | | | | | | | |
M M M M M M M M M 9x8=72

If I bought that DIMM, Chipkill could not protect me if one of the
memory chips fails. The chip is eight bits wide, so four bit protection
wouldn't repair the fault.

If I use 4 bit wide chips, my single rank example with ECC looks like this.

A ------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ Address bus
| | | | | | | | | | | | | | | | | |
M M M M M M M M M M M M M M M M M M 18x4=72

Now, that DIMM can be protected in a more useful way, by Chipkill.
If I use two DIMMs, run them dual channel, the 144 bit wide array
can protect me from a single dead 4 bit wide chip.

We haven't discussed the "registered" part yet. Typically, 16 to 18
chips, is as many address loads, as we might choose to drive from
the memory controller. By adding a chip which can buffer the electrical
signals, it becomes possible to isolate the additional loading. This
would be "registered ECC". I chose the 8 bit chips for this diagram,
so it wouldn't go off the side of the screen :-)

+----------------+
| register |
A ----| (one cycle |--+--+--+--+--+--+--+--+--+ Address bus
| delay) | | | | | | | | | |
+----------------+ M M M M M M M M M 9x8=72

So, now we're able to handle more chips. Like, maybe even a quad
rank module.

From now on, I'm going to stop drawing the address bus in the diagram.
It drives all the chips on the module, as many as there are. It presents
one electrical load to the memory controller.

Now, let's make a dual rank module. I'll use the x8 wide chips again,
to make the diagram smaller. Here, 8 data wires are in each "|" in
the diagram. There are two electrical loads per wire.

M M M M M M M M M Dual rank (something you'd seen
| | | | | | | | | in your desktop computer - a
M M M M M M M M M double sided DIMM)
| | | | | | | | |
--- 72 bit wide bus ---

So, how much space is available on a DIMM. Traditionally, 36 chips is
what you might find. I could do a dual rank 18 chips each (36 chips total)
that way.

Just for kicks, I checked the Micron site, and they do have a "72 chip"
RDIMM for DDR2. The trick is, two silicon die are placed in a single IC package.
There are physically 36 things soldered to the module, but inside there
are 72 chips total. That is 72 loads on the address bus. Note that, if a chip
dies, it takes out a x4 wide chip in a couple ranks at the same time,
which can still be fixed by Chipkill. Module is 1.18" high, so the FPGA
chip packaging really helps here. Depending on the die orientation, the
module might be thicker.

http://download.micron.com/pdf/datas...TS72C1Gx72.pdf

This would be a memory failure on that example Micron module. Since
a single IC packages two silicon die, two ranks will be needing help
from Chipkill. But the system still works. Only one rank is accessed
at any point in time.

| | | | | | | | | | | | | | | | | |
M M o M M M M M M M M M M M M M M M 18x4=72
| | | | | | | | | | | | | | | | | |
M M o M M M M M M M M M M M M M M M 18x4=72
| | | | | | | | | | | | | | | | | |
M M M M M M M M M M M M M M M M M M 18x4=72
| | | | | | | | | | | | | | | | | |
M M M M M M M M M M M M M M M M M M 18x4=72

This is their module which is currently in production. The chips are
physically bigger. It appears they're using twin die in these chips as well.
18 chips, 36 silicon die.

http://download.micron.com/pdf/datas...C512_1Gx72.pdf

This would be a memory failure in that product. Fixed by Chipkill.

| | | | | | | | | | | | | | | | | |
M M o M M M M M M M M M M M M M M M 18x4=72
| | | | | | | | | | | | | | | | | |
M M o M M M M M M M M M M M M M M M 18x4=72

So, what things do we know

1) Address loading at the processor, is one per DIMM, supported by the register.
The Address register on the DIMM, has its own limits, so maybe the speed
possible drops, as the number of chips rises. But at least at the (wimpy)
processor memory interface, we have one load per slot.

2) Data loading at the processor, is proportional to the number of ranks.
The Opteron has dual channels, splitting the 4 slots into two groups of two.
With quad rank DIMMs on the channel, that is 8 electrical loads. The processor
is slightly less sensitive to data loading, but 8 loads is a lot. (Especially
when we compare to the 4 loads an AMD desktop supports.) Maybe the BIOS
uses a slower speed setting for the quad rank DIMMs.

Sorry I cannot provide any links to web sites that address these issues.
AMD documentation is available for their older products, but they're pretty
secretive these days. AMD has some kind of tech support you can contact with
questions, so perhaps you can force them to draft a response, if they cannot
provide self-help style info for their products. If you can ignore the
bickering in this thread, you can get some idea of who to contact.

http://episteme.arstechnica.com/eve/...r=171008880931

tech DOT support AT amd DOT com
408 749 3060 (US CPU Support) or 44 1276 803299 (EU CPU Support)

The motherboard makers are equally secretive. I downloaded a few S1207 motherboard
manuals, and couldn't find mention of dual or quad rank there and associated rules.
It would help if there was a table of speed versus configuration, for example.

Good luck in your search,
Paul