See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct 100% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Atlanta Edition, published since February 17, 2009 (33 consecutive business days), is on track to save you $12.37 per year.

See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct over 97% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Chicago Edition, published since March 9, 2009 (19 consecutive business days), is on track to save you $1.59 per year.

See below for an explanation of why that savings is so far out of line with the other cities, and why it will certainly get better over the next few weeks.
See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct 100% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Houston Edition, published since March 9, 2009 (19 consecutive business days), is on track to save you $18.04 per year.

See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct 100% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Nashua, NH Edition, published since January 12, 2009 (57 consecutive business days), is on track to save you $7.96 per year.

See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct 100% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Raleigh Edition, published since February 17, 2009 (33 consecutive business days), is on track to save you $11.45 per year.

See the detailed breakdown of day-to-day savings.
Check our Track Record. As of 4/2/2009, we've been correct 100% of the time.
Subscribe Today for as little as $1.39 for three months of our Gas Predictor Newsletter.

Our Hypothetical Test Case

In the real world, of course, nobody can follow our buy/no-buy recommendations exactly. If you need a tank of gas every three days, and our prediction algorithm calls for declining gas prices five days in a row, you're going to have to buy gas on a day when we recommend waiting for a better price tomorrow. And if we predict rising gas prices tomorrow, and therefore recommend that you buy gas today, you're not likely to do it if you just filled up yesterday.

So how can we measure the money saving power of our predictions?

Of course, we have been following our own recommendations, as far as possible, since long before we began publishing our predictions. Why not just report our own savings? Well, each member of our team has radically different driving and fuel consumption habits. One of us drives a little econo-box and another drives a monster 4x4 pick-up that eats econo-boxes for breakfast. And by coincidence, every one of us has changed jobs in the last few months, so our driving and fuel requirements have all changed in the past 100 business days.

We don't have a good, consistent, real-world result that covers the last 100 days. So, we made one up. Not that we made up the results, just that we made up an artificial consistent gas-purchasing habit and "back tested" against our recommendations and the subsequent results.

For each of our six published editions, we used the same hypothetical scenario. The hypothetical driver has a 14 gallon tank, and he uses exactly 2 gallons of gas each business day. He began following our recommendations on the day we began publishing each edition of the Gas Predictor. In each test case, he begins with a full tank in the morning.

At the end of the day, when he gets his Gas Predictor newsletter, if we recommend buying, he fills his tank, even if he can only buy two gallons.
If we recommend not buying gas, he does not buy gas unless his tank is empty. In that case, he buys only 4 gallons, to get him through a couple of days and allow himself the capacity to take advantage of a buying opportunity in the next couple of days.

For each edition of the Gas Predictor Newsletter, we also ran a control case. The hypothetical control driver has a 14 gallon tank and uses exactly 2 gallons of gas each business day, just as in the test case. However, the control driver buys a full tank of gas whenever his tank gets down to 2 gallons.

The bottom line, the numbers we claim above as the savings for each of our six editions of the Gas Predictor Newsletter, is the difference between how much the control driver would have spent on gas minus how much the test driver would have spent on gas. We also account for the fact that one or the other drivers will have more gas in his tank on the last day of our scenario (April 3). If one driver has more gas than the other, we calculate the value of that gas based on the retail price of gas on April 3, and subtract that from the amount spent on gas.

In each city for which we publish a local Gas Predictor Newsletter, we use the retail price of gas as we recorded it on each day we have been publishing. That is, the second-lowest price of regular unleaded gas at about 4:00 PM local time in each respective city. For the test of the National Gas Predictor Newsletter, we used the average of those second lowest prices in each of the twelve cities that we use in our forecasting algorithm.

Why Can't We Save Money in Chicago?

By unfortunate coincidence, every single day since we began publishing our Chicago Gas Predictor Newsletter, economic circumstances have pointed toward rising prices in Chicago. Our hypothetical test driver, therefore, has bought two gallons of gas on each of the past 19 business days. He has had no opportunity to save money, because gas prices have never declined in Chicago in all that time, not one day. (Note that gas prices have not increased every single day, but they have either increased or remained the same. They never declined.)

Yet even with that, our hypothetical driver did manage to save a little bit of money. There are two reasons for this:

By keeping his tank full while prices rise, our hypothetical subscriber has "stocked up" on cheaper gas before the prices rose. You might say that keeping his tank full of cheaper gas prevents him from buying too much of the more expensive gas after the price rises.
Our hypothetical subscriber ended the test with two more gallons of price-inflated gas in his tank than did the hypothetical control driver. The value of this gas, after the latest price increase, offsets some of the money he spent by buying gas every day.

Those phenomena will work in favor of our subscribers who buy gas frequently in an environment of constantly rising gas prices. Your savings would be more dramatic if there were fluctuations in price that enabled you to skip the most expensive days, but even if prices never decrease, we can save you a little bit of money by telling you in advance that they will not decrease.

By coincidence, our hypothetical control driver had a nearly full tank on the day of the price "pop," when the retail price of gas in Chicago rose 13 cents in one day. If he had to buy a full tank on that day, our test driver would have saved even more. But, we didn't muck with the test scenario. We made up the rules before we knew what the result would be, and we can not, in good conscience, massage the scenario to improve our results. They are what they are.

We are supremely confident that as time goes by, there will be some fluctuations in retail prices in Chicago which will allow our hypothetical driver to save money, just as in the national average case and in the cases of the other four cities.

What We Have Learned in This Analysis

First, We Need Better Predictions of Size of Increase

Preparing for this 100th prediction celebration has given the GasPredictor.com team a chance to look at what we have accomplished from a longer-term perspective than usual, and that has been very valuable to us. Ordinarily, all we do is look at today's economic conditions and crunch the numbers to come up with tomorrow's predictions, with a little bit of a backward look at whether yesterday's predictions were correct (which they virtually always are). As we prepared these pages for our 100th prediction, we took a somewhat longer view for the first time, and we considered what value we have brought to our subscribers.

One thing that we have noticed, especially considering the unfortunate situation with our Chicago forecasts (and the slightly less disappointing, but still disapointing results in Nashua) is that it is not necessarily enough to know whether prices are going to increase or decrease tomorrow. Especially when prices are going up every day, it is especially important to recognize those particular days when prices are going to take a particularly large jump.

We have, in fact done this. The largest single-day price increase we have recorded for cities for which we publish forecasts occurred in Chicago on March 25, 2009 (when the second-lowest price rose from $1.999 to $2.129). The day before that, we included a note in our Chicago forecasts saying that this was going to happen.

However, what we have recognized is a need to call more attention to these phenomena, and to quantify them better. We have tended to ignore the correlation between our predicted price range and the actual price on the next day. We have been measuring our success or failure only on the direction of price changes - up when we say up, down when we say down - rather than on the amount by which the price changed. If we said, for example, that prices were going to increase by four cents per gallon, and they actually increased by six cents per gallon, we called that a successful prediction.

As our Chicago and Nashua analyses have illustrated, there are opportunities to save money on gas if you can know not only that prices are going to increase, but by how much. If our hypothetical subscriber could have known and acted upon information like this, we might have been able to tell him that he doesn't need to buy gas every single day that prices are going up, but only on those days, like March 25, when prices are going up a lot.

We don't quite know how to do this reliably. Even on March 24, we were only able to say that prices in Chicago were going to undergo a "pop" some time that week. And one of the factors that alerted us to the impending "pop" in Chicago was that several other cities had already undergone similar (though smaller) "pops." We did not predict those "pops" in those other cities even though, in retrospect, our data suggested that they could happen.

It seems clear that we are entering another period like last spring, when prices rise steadily, day after day, instead of fluctuating. In such an environment, the only way to save money by predicting tomorrow's gas price is by recognizing the difference betweeen when prices are going to rise a little and when they are going to rise a lot.

We need to figure this out. And we intend to.

This is not an announcement of a new format or new information to be included in our Gas Predictor Newsletters. We haven't figured it out yet. But we clearly see the need, and we have well over a year's worth of data with which to back-test any ideas.

When we do figure it out, we'll let you know.

Second, Our Results Will Be Better Over Longer Time Periods

As we reviewed the highlights of the test scenario in Atlanta, we were struck by many coincidences. Often, when our control driver had to buy gas, when his tank was down to two gallons, it just happened to be on a day when we were predicting gas prices to increase the next day, or it just happened to be a day or two before a very large price increase, or a day or two after a large decrease. Clearly, if the control driver were buying gas on different days, the results of the test would have been dramatically different.

Now, when we set out to document and quantify the results of our predictions, we set up the hypothetical conditions for our control driver and for our hypothetical subscriber, and we simply ran the tests the way the conditions dictated. We were aware that we might be able to rig the tests to get better results, but we didn't do that. However, after seeing those many coincidences of timing in Atlanta, we got curious.

We ran each of the tests over again, modifying only one factor: How much gas was in the control driver's tank on day one. This would force him to buy gas on different days throughout the test period.

By rigging the tests for optimal results, here are the best savings we could come up with:

National: We can rig the test to say we're saving $71.57 per year.
Nashua: We can rig the test to say we're saving $82.28 per year.
Atlanta: We can rig the test to say we're saving $148.53 per year.
Raleigh: We can rig the test to say we're saving $159.68 per year.
Houston: We can rig the test to say we're saving $246.03 per year.
Chicago: We can rig the test to say we're saving $275.74 per year.

The only scientifically defensible conclusion to draw from this is that our data set is too small. If we run the original tests out for a longer period of time, there will be fewer of these coincidences, and we will almost certainly get better results. Tellingly, the places for which we have been publishing our predictions longest are least susceptible to buggering with the results, because the larger data set smooths out all the quirks, both those in our favor and those that work against us. Take another look at that list above. It is not in order of increasingly positive results, it is in order by increasing length of time we have been publishing our forecasts. But that amounts to the same thing.

That all sheds new light on our "real" test results. Notice that both our best (Houston) and our worst (Chicago) savings occur in the two cities we have been forecasting the least amount of time. Notice that the National edition, which has been running the longest and which has the "smoothest" data (because it consists of an average over twelve cities each day) still provides quite positive results.

We will repeat these tests when we have more data. Maybe when we reach 200 days, or when each city reaches 100 days. Certainly, we will recap the results after a whole year of predictions.

But we will not rig the results. We'll stick with the scenarios we created in this 100th prediction celebration. That's just honest.

We're onto something good here. 100 days is not enough time for us to prove exactly how good, but it's good.

Join us for the adventure! Subscribe today!


Stumble It!
Save This Page in del.ico.us

[ Today's Gas Price Prediction ] [ Subscribe ] [ How to Use Gas Predictor ] [ Gas Predictor.com's Track Record ] [ Local Gas Prices ] [ Gas Saving Tips ] [ About Gas Predictor ] [ Privacy Policy ] [ Contact Gas Predictor ]