kkkk wrote:
Hi all,
we have a new machine with 3ware 9650SE controllers and I am testing ...

I found it! I found it!

dd apparently does not buffer writes correctly (good catch, Mark):
apparently disregards bs value and submits very small writes. It needs
oflags=direct to really do that, and even then there's a limit. Also,
elevator merging of small writes does not try hard enough and cannot
achieve good throughput. More details tomorrow.