FPS, Frame Time and you: adventures in performance.

cubed2D
April 28th, 2007.

 

Everyone knows what FPS is, so I won’t waste time explaining that it stands for Frames Per Second, and that 60 FPS is the level of performance your game should be targeting, and 30 FPS is acceptable but you shouldn’t drop below that, and all the common knowledge stuff, so don’t worry about that.

Instead I’ll look at what FPS values actually mean, how we count them in game and then why it’s a bad idea to profile your games performance using FPS as your only metric.

So as you no doubt are aware, 60 FPS is where it’s all at. I’m sure you already noticed that the default timing system in XNA locks the game loop at 60 FPS by default. Seeing as we want to make sure our games are running at 60 FPS, wouldn’t it be useful to actually see the frame rate? As XNA has not got a inbuilt way to find the FPS, its time to roll our own!

So how do we measure our FPS? Well, I’m sure if you think about it you can come up with a simple way to find it out. Stop reading now and have a little think!

You were probably thinking something along the lines of this:

public override void Update(GameTime gameTime){               

    frame++; //a frame counter, incremented every time update is called
   (and therefore, once a frame)               

    // calculate current FPS               

    time += gameTime.ElapsedGameTime.TotalMilliseconds; // add the
    amount of time since we last updated a frame to a time counter               

        if (time >= 1000f) //after we have been running for 1 second               

        {               

            time = 0f; // reset the timer               

            currentFPS = frame; //and copy the amount of frames drawn
            to a fps counter, ready to display               

            frame = 0; //reset frames and begin counting all over again!               

        }               

    }

Yep, it seems simple enough. The problem is, this method can’t tell us the immediate frame rate (as in the fps for each frame), or the average frame rate without storing every seconds frame rate value, which would quickly become a massive collection! There must be a more elegant and useful way of doing it. Maths time!

How much time does the computer have to think about each frame if its going to draw 60 frames a second? Well, that would be….

1 second / 60 frames = 16.666..ms or 1 second / 60 FPS =0.0166666667 seconds

wow, not a lot of time! Well, basic maths lets us rearrange this as…

1/ 0.0166666667 = 60

and as simple as reciprocal division, we have a simple formula for calculating our FPS on a frame by frame basis!

currentFPS = 1 / gameTime.ElapsedGameTime.TotalSeconds; // remember, the elapsed time is the time thats passed since we last ran Update.

Using this new found knowledge, we can finally do fun things like know the frame rate for each frame and find out the average over the course of our games life! Stop! It’s coding time.

In your project create a new game component, call it performanceMon. As we’re going to want to draw the FPS to the screen so we can see it easily, change it from using GameComponent as a base to DrawableGameComponent. That will let us override draw and the content methods.

Get your code to look like this.

#region Using Statements               

using System;               

using System.Reflection;               

using System.Collections.Generic;               

using Microsoft.Xna.Framework;               

using Microsoft.Xna.Framework.Content;               

using Microsoft.Xna.Framework.Graphics;               

#endregionnamespace NekoCake               

{               

    public class performanceMon: Microsoft.Xna.Framework.DrawableGameComponent               

    {               

        private ContentManager content;               

        GraphicsDeviceManager graphics;               

        private SpriteBatch batch;               

public performanceMon(Game game, GraphicsDeviceManager graphics)               

    : base(game)               

    {               

        this.graphics = graphics;               

    }               

public override void Initialize()               

    {               

        content = new ContentManager(base.Game.Services);               

        base.Initialize();               

    }               

protected override void LoadGraphicsContent(bool loadAllContent)               

    {               

    if (loadAllContent)               

    {               

        batch = new SpriteBatch(graphics.GraphicsDevice);               

    }               

    }               

protected override void UnloadGraphicsContent(bool unloadAllContent)               

    {               

    if (unloadAllContent == true)               

    {               

        content.Unload();               

    }               

    }               

public override void Update(GameTime gameTime)               

    {               

        base.Update(gameTime);               

    }               

public override void Draw(GameTime gameTime)               

    {               

    }               

}               

}

 

Now that we have a layout to work in, lets get our FPS formula into code! First off, we need some new variables.

//FPS variables               

private int totalFrames;               

private double currentFPS;               

private double averageFPS;               

private double highestFPS;               

private double lowestFPS;Now lets add the FPS logic to the update override.               

public override void Update(GameTime gameTime)               

{               

// increment frame counter.               

totalFrames++;               

//fps calculations               

currentFPS = 1 / gameTime.ElapsedGameTime.TotalSeconds;               

averageFPS = totalFrames / gameTime.TotalGameTime.TotalSeconds;               

// highest fps               

if (currentFPS > highestFPS)) highestFPS = currentFPS;               

// lowest fps               

if (currentFPS < lowestFPS) lowestFPS = currentFPS;               

}

Yay, thats that done and dusted… oh wait a bug! If you run the code and keep an eye on the Highest FPS, you will see that it comes out as Infinity (you can just pause the game sometime and look it up in the immediate window or the local variables window). Turns out that on the first call, the game’s been running for 0 seconds, and therefore our FPS calculation says were running at an impressing speed of an infinite amount of frames a second! So, we have to check if current fps is infinity.

if (currentFPS > highestFPS && !double.IsInfinity(currentFPS)) highestFPS = currentFPS;

Yay, thats that done and dusted! We now know the performance of our game! Unfortunately, FPS isn’t a good metric for game performance, it’s just the most well known. Its problems stem from the fact that its completely non linear.

 

What does that mean for us? Well suppose your game was running using a variable game time loop (remember, XNA defaults to locking your game at 60FPS, you can unlock it and use variable frame time instead). Your game’s running at 500FPS. You change something and now it’s running at 330FPS. Thats a massive loss of 170FPS! Now, in a different situation (say, a much more busy level in the game) its running at 60FPS, you change something and now its running at 55FPS. Which change represents the more computationally expensive change to your game, the 170FPS drop in you test app or the 5FPS drop in your full level? If we look at the Frame Time, the amount of time it takes a frame to draw, we can tell easily! How do we calculate the frame time? We divide 1000ms(or 1second) by the frame rate.

 

1000 / 500 = 2ms per frame.

1000 / 330 = 3ms per frame (ok, I rounded up here a little)

 

1000 / 60 = 16.66 ms per frame

1000 / 55 = 18.18ms per frame

 

Surprisingly, the method that caused a 5FPS drop in the second example is twice as expensive as the one that caused a 170FPS drop in the first! As you can see, the different frame rate drops don’t give an accurate idea of the real impact, computationally speaking, a new drawing method has, on your game loop for example. A increased draw time of 1 ms per frame from 60FPS would give us a frame rate of 58.8something FPS, which is not the same kind of drop you get when you increase the draw time from 2ms to 3ms. Robert Dunlop made a brilliant graph showing this (infact, that’s the article I used when I first learned of Frame Time. LINK)

So, what does that mean for us? FPS is a good way to quickly judge performance. It’s a good number to glance at to get an idea of how well the game’s running, it’s not a good metric to use when trying to optimize your game engine to run as fast as possible, or when comparing the speed of different algorithms. Seeing as they both have merits, we should support both. Heres the new variables we will need.

//Frame Time variables               

private double lastFrameTime;               

private double averageFrameTime;               

private double bestFrameTime = 999; // see lowestFPS               

private double worstFrameTime;               

and add the following to the update method.               

//frame time calculations               

lastFrameTime = 1000 / currentFPS;               

averageFrameTime = gameTime.TotalGameTime.TotalMilliseconds / totalFrames;               

// best frame time               

if (lastFrameTime < bestFrameTime && lastFrameTime != 0) bestFrameTime = lastFrameTime;               

// worst               

if (lastFrameTime > worstFrameTime) worstFrameTime = lastFrameTime;

Notice the && lastFrameTime != 0 bit? This fixes the same issue as the check for infinity last time.

Ok, lets check what we have achieved so far. We can calculate the FPS, average FPS, and store the highest and lowest FPS achieved, and do the same for Frame Time. Good, that’s a start! So, what limits the usefulness of this class? At the moment, we have no way of reading out the FPS or Frame Time, apart from pausing the execution and checking manually. We do not have the ability to reset any data, no way to check the FPS/Frame Time in code (it would be usefully, for example if we had implemented two different path finding systems, and we ran each on the same data one after the other, we would want to know the frame time for each system so we could see which runs quicker).

Also, it might be nice to add in some logging abilities to the class, rolling all our debug systems in to one manager. Also, it might then be a good idea to get a printout of the games version number on screen.

Ok, so lets get these issues sorted one by one. First is output. We will start out with some basic text based output, and the add to that to get a cool console window for it. As a note XNA Game Studio Express 1 Refresh is needed as I’m going to use the new DrawString method, but you can of course use whatever output system you want.

First, we need us a string to write the text in to. Declare a new string at the top of the class called output. Next we need to add a font. Add a new font to your project. If you dont know how to do this yet, it’s easy. In the solution explorer, right click, add new item and chooses sprite font. The XML file it creates is self explaining, and you can look it up on the msdn if you need more help.

As for font, I’m going to use proggy small, my favourite programming font (http://www.proggyfonts.com) . Then we need to add the font to our variable list.

private SpriteFont font;

in the LoadGraphicsContent override, we need to load the font.

font = content.Load<SpriteFont>(“assets\\font\\console”);

now we need to build the output string in the Update override.

//build the output string.
output = "FPS Stats (Higher is better)n CurrentFPS: " + currentFPS.ToString() +
" n Average FPS: " + averageFPS.ToString() +
"n Highest FPS: " + highestFPS.ToString() +
"n Lowest FPS: " + lowestFPS.ToString() +
"nFrame Time Stats (Lower is better) n Last Frame Time: " + lastFrameTime.ToString() +
"n Average Frame Time: " + averageFrameTime.ToString() +
" n Best Frame Time: " + bestFrameTime.ToString() +
" n Worsts Frame Time: " + worstFrameTime.ToString();

last of all, we need to draw it to the screen. Add this to the draw override.

batch.Begin();

batch.DrawString(font,output,Vector2.Zero,Color.White);//draw the text

batch.End();

If you want to have a look at the complete code up to this point, you can download a copy here (http://www.nekocake.com/pad/code-samples/). So far we’re at performanceManager v1.

Next up was the ability to reset the high/low/average scores for FPS and Frame Time. This will work fine for high and low, but seeing as average is calculated using time we are going to need to keep track of how much time had passed when we asked for the reset, otherwise it wont work. Lets give ourselves two more variables

private double resetTimeMS;
private double resetTimeS;              

//and a brand new method called Rest              

public void Reset(GameTime gameTime)
{
totalFrames = 0;
averageFPS = 0;
highestFPS = 0;
lowestFPS = 999;
averageFrameTime = 0;
bestFrameTime = 999;
worstFrameTime = 0;
resetTimeMS += (gameTime.TotalGameTime.TotalMilliseconds - resetTimeMS);
resetTimeS += (gameTime.TotalGameTime.TotalSeconds - resetTimeS);
}

Storing the time elapsed since the reset will allow us to take that time away from the total time when calculating the average, thus giving us the correct average since the last reset.

Change the aveargeFPS calculation to

averageFPS = totalFrames / (gameTime.TotalGameTime.TotalSeconds – resetTimeS);

and the averageFrameTime calculation to

averageFrameTime = (gameTime.TotalGameTime.TotalMilliseconds - resetTimeMS) / totalFrames;

Now thats done, what’s left? Like I mentioned earlier, logging support and time comparisons might be a good addition to this class, but really you can use it as is. It could probably be improved, such as using the string builder system instead of normal strings. Have a look at version on my code page (http://www.nekocake.com/pad/code-samples/). I’ve added a few features to it, such as a performance indication light, loading of the assembly to get the game name and the version number, using a custom render target to render the games name on to the back of the console window texture. Ive also added a few more stats to the print out and added masks to the output to make it easier on the eye.

This is just a small example of how you might modify this component for your engine/game, so get to it, add it to your engine, if you make any cool mods please email me and pass on your experiences!

 

Heres a short video of v2 running in my balls test app.

Last Articles

• TileSets, now in XML
• First screenshot of CrimsonEngine’s new Tile based map support
• A useful code snippit
• Im still alive
• The current state of crimson