Constructing a Database Benchmark (average load)

Our first benchmark was custom-written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Loader" for the purposes of discussing what it does.

SQL Loader allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use with which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:

< workload>

< !--- A SAMPLE WORKLOAD QUERY THAT RETURNS ALL THE FIELDS FROM THE PRIVATEMESSAGES TABLE RANDOMLY --->

<query>

<code>select * from privatemessages where imessageid = @pmessageid</code>

<type>read</type>

<randkey>pmessageid</randkey>

</query>

<!--- RANDOM ID GENERATOR FOR SELECTING RANDOM PRIVATE MESSAGES --->

<randomid>

<rcode>select imessageid,newid() as pmsgid from privatemessages order by pmsgid</rcode>

<name>pmessageid</name>

</randomid>

< /workload>


A screenshot of the SQL Loader


Test Information

The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each iteration of the test was run for 10 minutes, with the first being from a cold boot. SQL was restarted in between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.

Hyper Threading and The Tests AnandTech Forums Database Test Results
Comments Locked

44 Comments

View All Comments

  • appu - Monday, September 13, 2004 - link

    #10, I agree with you. I think before doing an AMD vs. Intel shootout a Prestonia vs. Nocona was more in order.
  • Marlin1975 - Monday, September 13, 2004 - link

    Another thing I did not see, maybe missed it?, was the 3.6Ghz P4/Xeon is MORE a paper launch then anything as I can't seem to find it anywhere. The higgest Xeon I see in real quanity is the 3.4Ghz.
    So is it fair to compare a almost impossiable and even higher priced CPU to one that is easy to find and cheaper?
  • vmajor - Monday, September 13, 2004 - link

    Ah, so AMD farted out the specs?

    "When AMD first broke wind with the K8 announcement.."

    That's one way to grab attention...

    Victor
  • Jalf - Monday, September 13, 2004 - link

    #8: Well, why should they? It looks like even Intel are starting to realize that performance doesn't equal MHz. They have much easier and more effective ways to improve their performance than to strive for 4GHz
  • gherald - Monday, September 13, 2004 - link

    Well well well... a couple of inconclusive 32bit benchmarks.

    What this article shows that we didn't already know or didn't expect:

    - Nothing.

    What this article shows that we already knew:

    - The Xeon shared FSB is a significant limiting factor.

    What this article shows that we did expect:

    - The Nocona performs similarly to the Opteron 250 in 32bits.

    What this article did not cover:

    - 64BIT BENCHMARKS !
    - 64BIT vs. 32BIT !
    - How do 2xNocona and 2xPrestonia compare at EQUAL clockspeeds? (i.e. exactly how does the decreased cache and longer pipeline affect numbers)
    - How does a 1xNocona compare to an equally clocked Prescott and Northwood? (i.e is it just about equal to the Prescott, as one might expect?)

    What this article neglected to mention:

    - How did the Nocona's use of DDR2 affect the benchmarks, if at all?
    - More importantly, how does it affect price vs. performance ? DDR2 is still twice as expensive and offers virtually no performance improvement, right? So assuming the 3.6 Nocona sells at a similar price to the Opteron 250, you still have to factor in the increased memory costs and when packing in 4GB, as with these test systems, it is no mean consideration!

    To #9: AMD is in no hurry to do this, what with DDR2 costing twice as much and offering no tangible benefits as of yet.
  • fergiboy - Monday, September 13, 2004 - link

    It is interesting to note that the Intel platform runs on DDR2. AMD is going to have to tweak its processor to release DDR2 support due to the on die memory controller.
  • sprockkets - Monday, September 13, 2004 - link

    There will be versions of Prescott that will have more cache. The question is, will it ever reach 4.0ghz without melt down hahaha.
  • hirschma - Monday, September 13, 2004 - link

    I'm not sure how cache effects database performance - I'd guess not very much, but I honestly don't know. If my guess is right, AMD is on track to scale right by Intel very shortly.
  • Denial - Monday, September 13, 2004 - link

    Nice article Jason, and please keep the server based benchmarks coming. It's refreshing to see numbers from a reputable source, and though the actual results here may not influence many purchases, it at least gets the "Opteron is a viable option" point across.

    The slight difference between the top of the line Opteron and Xeon is so slim that the choice of a platform would obviously come down to other more relevant factors. The scores are also so close that it if another application were used the Xeon could come out ahead.

    The only real way to know if there is any meaningful performance difference between Xeon and Opteron systems would be to benchmark the application you will be using on systems from your vendors of choice. If the difference is less than 10%, more than likely the TCO, support, organizational standards, and more often than not a good sales team will drive the choice of which systems will be purchased. Unfortunately for AnandTech, they'd have to test the systems as they come from the manufacturers, storage and all, for the benchmarks to be useful to a large organization. There can be a very noticeable difference in performance from an HP, IBM and SUN server all using the same processor.

    The interesting data will come when we have 64 bit benchmarks using these CPU's. Win64 with 8-32GB of RAM and 64 bit versions of MS SQL, DB2, Oracle, etc., will be much more informative and useful.

  • DrMrLordX - Monday, September 13, 2004 - link

    Might be nice, knightcrow. I haven't seen those used in many(if any) hardware benchmark reviews, though. At least they made mention of the 2% deviation they got with benchmark results.

    I don't think anybody's really surprised here. I would, however, like to hear from AMDjihad. He's ever-so-insightful.

Log in

Don't have an account? Sign up now