• Beta Client Performance

    I finally managed to sit down and get a little time to benchmark the actual client rather than relying on the official FFXIV benchmark for performance numbers. The numbers for the beta client were obtained by running the same 12 minute course, starting at the Adventurer's Guild in Limsa Lominsa, running down through and past Bearded Rock, and then doubling back. As I obviously can't trace my steps exactly and cannot control other variables such as the number of players along the path at any given time, there's a small margin of error involved. CPU usage was tracked at the logical processor level via the native performance monitor built into Windows 7. GPU usage and framerate were obtained via MSI Afterburner. For all tests, the Sapphire 5870 was set to 850MHz core and 1200MHz memory to approximate a reference 5870 as closely as possible.

    The test system specs are as follows:

    CPU Intel i7 860 (2.8GHz stock)
    Motherboard Gigabyte GA-P55A-UD7
    RAM Gskill DDR3-1600 7-8-7-24
    Video Sapphire 5870 2GB TOXIC Edition (925MHz/1225MHz stock but set to 850/1200 for the benchmark)
    Sound Creative X-Fi Fatality Pro
    Storage SandForce 100GB SSD (OWC Mercury Extreme)
    OS Windows 7 Professional 64-bit at stock performance settings
    Video driver ATI Catalyst 10.7

    The following image quality settings were used for all tests:

    Display Mode Full Screen
    Window Size 1920x1080
    Multisampling 8xQ MSAA
    Buffer Size Resolution
    Shadow Detail Highest
    Ambient Occlusion Off
    Depth of Field Off
    Texture Quality High
    Texture Filtering Highest

    Test 1 - The Baseline

    i7 860 @ 2.8GHz with 2 cores enabled, Hyperthreading disabled, Turbo off
    (2 logical processors consisting of 2 physical cores)
    Official benchmark 1920x1080 score: 3646

    Beta client 1920x1080:

    Average GPU usage 47.4%
    Average Framerate 41.7
    Min Framerate 15.3
    Max Framerate 60.2
    CPU1 Avg usage 83.1%
    CPU2 Avg usage 78.7%
    Total CPU Avg 80.9%

    Observations: Looking at the graph, you see a very nice symmetrical pattern which makes sense since I ran the path in one direction and then doubled back. Note the dips in framerate where neither CPU nor GPU are pegged and framerate and GPU mimic each other for a great deal of the benchmark.

    Test 2

    i7 860 @ 4.0GHz with 2 cores enabled, Hyperthreading disabled, Turbo off
    (2 logical processors consisting of 2 physical cores)
    Official benchmark 1920x1080 score: 4505

    Beta client 1920x1080:

    Average GPU usage 57.2%
    Average Framerate 49.3
    Min Framerate 28.6
    Max Framerate 60.2
    CPU1 Avg usage 86.3%
    CPU2 Avg usage 68.1%
    Total CPU Avg 77.2%

    Observations: Increasing CPU from 2.8GHz to 4.0GHz definitely smoothed the game out and brought the minimum framerate up a lot. Again, we have the nice symmetrical pattern thanks to the path that was followed.

    Test 3

    i7 860 @ 2.8GHz with 2 cores enabled, Hyperthreading enabled, Turbo off
    (4 logical processors consisting of 2 physical cores and 2 virtual cores)
    Official benchmark 1920x1080 score: 3739

    Beta client 1920x1080:

    Average GPU usage 47.4%
    Average Framerate 41.6
    Min Framerate 17.5
    Max Framerate 60.2
    CPU1 Avg usage 62.3%
    CPU2 Avg usage 49.3%
    CPU3 Avg usage 25.7%
    CPU4 Avg usage 47.7%
    Total CPU Avg 46.2%

    Observations: While the system thinks overall system CPU usage dropped, there was little in the way of impact to framerate.

    Test 4

    i7 860 @ 2.8GHz with 4 cores enabled, Hyperthreading disabled, Turbo off
    (4 logical processors consisting of 4 physical cores)
    Official benchmark 1920x1080 score: 4613

    Beta client 1920x1080:

    Average GPU usage 49.6%
    Average Framerate 41.8
    Min Framerate 17.9
    Max Framerate 60.2
    CPU1 Avg usage 47.7%
    CPU2 Avg usage 57.3%
    CPU3 Avg usage 40.9%
    CPU4 Avg usage 43.2%
    Total CPU Avg 47.3%

    Observations: Not much change in the way of framerate off the baseline although the official XIV benchmark would have you believe there's almost a 1000 point difference.

    Test 5

    i7 860 @ 4.0GHz with 4 cores enabled, Hyperthreading disabled, Turbo off
    (4 logical processors consisting of 4 physical cores)
    Official benchmark 1920x1080 score: 4782

    Beta client 1920x1080:

    Average GPU usage 63.6%
    Average Framerate 54.3
    Min Framerate 33.5
    Max Framerate 60.2
    CPU1 Avg usage 20.4%
    CPU2 Avg usage 68.6%
    CPU3 Avg usage 51.8%
    CPU4 Avg usage 30.8%
    Total CPU Avg 42.9%

    Observations: This demonstrates that even at 1920x1080, FFXIV thrives on a faster CPU. FFXIV makes heavy use of two cores and lighter use of a third and fourth core. Framerate spends a great deal more time capped at 60fps when 4 cores are active.

    I'm only making an educated guess here based upon what I see in the performance numbers and in the client but I suspect that the code is not currently able to take full advantage of either CPU or GPU, probably due to how it's been written - think lots of conditional debugging stanzas that will likely remain until sometime after launch when they're satisfied that the client has reached some broad level of stability.

    There are lots of places that you can go in Limsa Lominsa where neither GPU nor CPU are anywhere close to being maxed out and yet the framerate still drops to 20 and it sits at 20 regardless of what you are looking at - you could be nose-to-wall and still have the same low framerate. If I had to guess, I would say this has to do with some combination of thread synchronization, the code waiting on network data, and performance-limiting debugging code. These issues are likely to get addressed in the days to come. There's definitely a lot of room for performance improvements in the client such that it should perform very well even on a moderate build.

    Special thanks to Xariana for setting up the images for this article.

    Comments 11 Comments
    1. Calaelen's Avatar
      (o.o); thank you for this extensive test series!

      There's definitely a lot of room for performance improvements in the client such that it should perform very well even on a moderate build.
      This gives hope to all of us with no i7 highspeed systems!

      Can I call you Mr.Mythbuster now? :D
    1. Calaera's Avatar
      Hey alecrast, this is pretty cool. :) Can't wait to see more tech stuff from you in the future.
    1. alecrast's Avatar
      Quote Originally Posted by Calaelen View Post
      (o.o); thank you for this extensive test series!


      This gives hope to all of us with no i7 highspeed systems!

      Can I call you Mr.Mythbuster now? :D
      Haha, if you like. From the graphs, there just seems to be a huge amount of performance left to squeeze out of the client. I see little reason that it cannot run at 60fps all the time with the right people behind the code.
    1. Alpha_Zeon's Avatar
      This is what I needed to see : ).

      My system specs
      CPU Intel i7 920 2.66GHz
      Motherboard EVGA X58 SLI LE Motherboard
      RAM OCZ DDR3 1333 7-7-7 20 12GB
      Video 1x EVGA GeForce 470 1280MB
      Storage Seagate 425GB HD
      OS Windows Vista Home Premium SP2


      Hmm, does FFXIV only run on Windows 7? I think I read that somewhere.
    1. Calaera's Avatar
      Quote Originally Posted by Alpha_Zeon View Post
      This is what I needed to see : ).

      My system specs
      CPU Intel i7 920 2.66GHz
      Motherboard EVGA X58 SLI LE Motherboard
      RAM OCZ DDR3 1333 7-7-7 20 12GB
      Video 1x EVGA GeForce 470 1280MB
      Storage Seagate 425GB HD
      OS Windows Vista Home Premium SP2


      Hmm, does FFXIV only run on Windows 7? I think I read that somewhere.
      It does not only run on Windows 7. I ran it on XP perfectly fine.
    1. Raspinudo's Avatar
      would love to see an amd based test
    1. alecrast's Avatar
      Quote Originally Posted by Raspinudo View Post
      would love to see an amd based test
      I have a 720BE with a GTX 285 I could test on. If the beta wasn't down, I'd run the numbers before and after I upgrade that system to a 1055T.
    1. Raspinudo's Avatar
      Quote Originally Posted by alecrast View Post
      I have a 720BE with a GTX 285 I could test on. If the beta wasn't down, I'd run the numbers before and after I upgrade that system to a 1055T.
      Yeah I am going to be running a 965BE and GTX 470
    1. Alpha_Zeon's Avatar
      Is it wise, or would it be wise for me, to overclock your cpu. I've never done that, yet.
    1. Ayelwen's Avatar
      Quote Originally Posted by Raspinudo View Post
      would love to see an amd based test
      I'll be running AMD 64x2 Dual 6000+ and AMD ATI graphics. If I manage to get into open I can post those stress results.
    1. alecrast's Avatar
      Just reposting my own guide here so that I don't have to keep digging for it:

      For those people interested in performance, I wrote down the steps to create a very basic performance counter for CPU usage.

      1. Start -> Control Panel -> Administrative Tools -> Performance Monitor

      2. Expand Data Collection Sets

      3. Right click User Defined

      4. New -> Data Collection Set

      5. Call it "All CPU"

      6. Check "Create Manually (Advanced)"

      7. Click "Next"

      8. Make sure "Create data logs" is checked

      9. Then check "Performance Counter" beneath it

      10. Click "Next"

      11. Set "Sample Interval" to 1 (make Units is in Seconds)

      12. Click "Add"

      13. Find "Processor" and click the little down arrow next to it to expand the list

      14. Find and select "% Processor Time"

      15. In the box below, select "<All instances>"

      16. Click "Add" (this will add it to the window on the right)

      17. Click "OK"

      18. Click "Next"

      19. Save the data wherever you want

      20. Click "Next"

      21. Make sure "Run as:" is set to "<Default>" and "Save and close" is checked.

      22. Click "Finish"

      Now you should see "All CPU" under the User Defined list

      23. Click it and something like "DataCollector01" should show up in the right window pane

      24. Right-click DataCollector01 and choose "Properties"

      25. Set "Log Format" to "Comma Separated"

      26. Click "Apply" and then click "OK"

      27. Now you can right-click "All CPU" and choose "Start"

      Remember to click "Stop" when you're done collecting data.

      I recommend the above in tandem with MSI Afterburner which you can set to log data, as well. Then you can take the two datasets and graph them together in Excel or GNUplot for some useful overlays of CPU vs GPU vs Framerate.

    Want to leave a comment? Log in or register with us!