Join up at Sportsbet - Click here


Forum Home Forum Home > Horse Racing - Public Forums > Tipping and Wagering
  New Posts New Posts RSS Feed - Calibrating Odds with Results
  FAQ FAQ  Forum Search   Events   Register Register  Login Login


Click here for the Thoroughbred Village Home Page - Australia's Premier Online Horse Racing Community. For village news, follow @TBVillage on Twitter. Also follow @Villagebet on Twitter for racing tips. Sign up for the village newsletter by clicking here: Newsletter Signup. To contact the Mayor by email: Click Here.

Calibrating Odds with Results

 Post Reply Post Reply
Author
Message
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (1) Thanks(1)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Topic: Calibrating Odds with Results
    Posted: 05 Apr 2017 at 2:04pm
Does anyone here who frames markets, calibrate their predicted probabilities (odds) with race results?

After listening to a few podcasts relating to Super Forecasting, it dawned upon me that this is the truest measure of testing prediction accuracy.

Simply, it is testing whether over the journey horses you price, actually win as regularly as your odds would suggest.

Ex. Do horses you rate $10 chances win 10% of the time, $2 chances win 50%, horses rated $100 win 1% etc?

I didn’t use this method until just after the spring carnival but was very surprised with the results. In the past I’ve calibrated finishing positions and given a small weighting to financial return but to my surprise across the range of runners that I had priced, they all seemed to win as often as my prices predicted. I thought there would be outliers that would skew certain price ranges but literally across the board the odds aligned almost perfectly with the results.

Anyway, I think it is a very valuable and reassuring test for anyone framing markets. How to undertake the exercise is another question as it took me a couple of days to get going.

Back to Top
Sponsored Links
Click here to grab your bonuses when you join up with Sportsbet.


Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (1) Thanks(1)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 06 Apr 2017 at 10:59am

Here's an example of probability accuracy graph as a guide to what I'm on about. The idea is the closer the results hug the perfect calibration line, the more accurate your markets. 

Perhaps it's a stretch to think anyone here is doing this then??? 

IMO it's a must if you want to frame markets accurately.
Back to Top
kavg View Drop Down
Champion
Champion
Avatar

Joined: 08 Jun 2009
Status: Offline
Points: 2351
Post Options Post Options   Thanks (1) Thanks(1)   Quote kavg Quote  Post ReplyReply Direct Link To This Post Posted: 15 Apr 2017 at 7:12pm
Looks good kincsem. I don't do it because I dont currently use ratings. Shows how accurate your ratings are and only way to see if they are worthwhile.
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 18 Apr 2017 at 10:16am
Thanks Kavg

I think most punters who frame markets looks at their monetary results as a measure of their system/methods but IMO the winner of a race is a very small sample of their predictions.

You can analyse every single runner in each race to help refine your methods. 

My KPI's from a single race are:

- Brier Score - Essentially what is being shown in the above graph, probability calibration.
- Predicted finishing position to actual with higher weighting to 1st, 2nd and 3rd
- and very small weighting to financial return.

Back to Top
Delta Deel View Drop Down
Champion
Champion
Avatar

Joined: 25 Aug 2015
Status: Offline
Points: 8896
Post Options Post Options   Thanks (0) Thanks(0)   Quote Delta Deel Quote  Post ReplyReply Direct Link To This Post Posted: 18 Apr 2017 at 3:24pm
I have software that does this for me. Just put in any rated price you want to check and you can run it for every race you have rated or even back test it against past results with previous data(mine goes back to 2011) and it will give you %SR, POT, ect
https://group1selections.wordpress.com/
Back to Top
Sneck View Drop Down
Champion
Champion
Avatar

Joined: 16 Feb 2013
Status: Offline
Points: 6540
Post Options Post Options   Thanks (0) Thanks(0)   Quote Sneck Quote  Post ReplyReply Direct Link To This Post Posted: 19 Apr 2017 at 11:21am
How concerned are you with unaccounted for factors, do you think your prices will be more accurate at 11:00am than post time.

To phrase it another way, say you price a horse market price 10/1 at 11:00am and it blows to 40/1 comes post time, what do you do?
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 19 Apr 2017 at 2:23pm
Originally posted by Delta Deel Delta Deel wrote:

I have software that does this for me. Just put in any rated price you want to check and you can run it for every race you have rated or even back test it against past results with previous data(mine goes back to 2011) and it will give you %SR, POT, ect

Sounds like I've just wasted the past 6 years of my punting life if you can buy it off the shelf. Can you adjust factors on historical data and see how it effects profit and accuracy?

I've had friends in the industry interested in commercializing my method/system but sounds like it's been done.
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 19 Apr 2017 at 2:29pm
Originally posted by Sneck Sneck wrote:

How concerned are you with unaccounted for factors, do you think your prices will be more accurate at 11:00am than post time.

To phrase it another way, say you price a horse market price 10/1 at 11:00am and it blows to 40/1 comes post time, what do you do?

I have a factor which is essentially inside/collective knowledge, it is probably the biggest factor in my whole system. It takes into account the market at any point in time and adjusts with the market.
Back to Top
Delta Deel View Drop Down
Champion
Champion
Avatar

Joined: 25 Aug 2015
Status: Offline
Points: 8896
Post Options Post Options   Thanks (0) Thanks(0)   Quote Delta Deel Quote  Post ReplyReply Direct Link To This Post Posted: 19 Apr 2017 at 10:59pm
Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Originally posted by Delta Deel Delta Deel wrote:

I have software that does this for me. Just put in any rated price you want to check and you can run it for every race you have rated or even back test it against past results with previous data(mine goes back to 2011) and it will give you %SR, POT, ect


Sounds like I've just wasted the past 6 years of my punting life if you can buy it off the shelf. Can you adjust factors on historical data and see how it effects profit and accuracy?

I've had friends in the industry interested in commercializing my method/system but sounds like it's been done.


Not really sure by what you mean by adjusting factors. With my software I just enter my rated price in the search criteria and you can also choose from multiple other options as in age, sex,class of race, weekday, race no, metro, country, prov, track, date, day of the week, track conditions and about 40-50 others and it will run and draw out all of these. I can also compare my rated price vs the official SP, TAB Fixed and best of totes.

IE I can say I want to look at every horse I have rated $1.25 since 2011 and it will run and produce every horse with that rated price, Finish Position and SP as well as give you SR, units in, units out and Profit%

I prefer to use ranges myself, I often search a range say Rated Price Range 1.01- 1.40 or rated price 21.00-31.00

https://group1selections.wordpress.com/
Back to Top
Delta Deel View Drop Down
Champion
Champion
Avatar

Joined: 25 Aug 2015
Status: Offline
Points: 8896
Post Options Post Options   Thanks (0) Thanks(0)   Quote Delta Deel Quote  Post ReplyReply Direct Link To This Post Posted: 19 Apr 2017 at 11:04pm
Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Originally posted by Sneck Sneck wrote:

How concerned are you with unaccounted for factors, do you think your prices will be more accurate at 11:00am than post time.

To phrase it another way, say you price a horse market price 10/1 at 11:00am and it blows to 40/1 comes post time, what do you do?


I have a factor which is essentially inside/collective knowledge, it is probably the biggest factor in my whole system. It takes into account the market at any point in time and adjusts with the market.


I have no idea how you factor this? How do you identify from mug money for a hyped media horse and informed info. You can almost watch on Dynamic Odds after Dean Lester gives his tips on Friday and Saturday mornings those horses tumble as the TAB listeners jump in blind. Are you not just a sheep following the flock ie getting on as the prices is below its peak?

Each to their own, I feel I am trying to pick up the ones the early price setters get wrong. Before the informed money scrapes any fat of their already skinny markets.
https://group1selections.wordpress.com/
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 10:37am
Originally posted by Delta Deel Delta Deel wrote:

Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Originally posted by Delta Deel Delta Deel wrote:

I have software that does this for me. Just put in any rated price you want to check and you can run it for every race you have rated or even back test it against past results with previous data(mine goes back to 2011) and it will give you %SR, POT, ect


Sounds like I've just wasted the past 6 years of my punting life if you can buy it off the shelf. Can you adjust factors on historical data and see how it effects profit and accuracy?

I've had friends in the industry interested in commercializing my method/system but sounds like it's been done.


Not really sure by what you mean by adjusting factors. With my software I just enter my rated price in the search criteria and you can also choose from multiple other options as in age, sex,class of race, weekday, race no, metro, country, prov, track, date, day of the week, track conditions and about 40-50 others and it will run and draw out all of these. I can also compare my rated price vs the official SP, TAB Fixed and best of totes.

IE I can say I want to look at every horse I have rated $1.25 since 2011 and it will run and produce every horse with that rated price, Finish Position and SP as well as give you SR, units in, units out and Profit%

I prefer to use ranges myself, I often search a range say Rated Price Range 1.01- 1.40 or rated price 21.00-31.00


Okay sounds like it is not the same as what I do. I can't quite do what you can as easily but I have a graph of all rated horses from 0% - 100% chance of winning and it compares the SR vs my rated prices. That's all I really want to know.

As long as my long shots and favorites aren't winning or losing more than I predict, I'm happy. If I started separating results based on certain criteria the sample would be too small anyway. 
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 10:38am
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 10:39am
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 10:45am
Above it the basic information I'm concerned with. The red are my results and the blue would be perfect calibration. The numbers on the left are the percentage of a particular runner winning. I have some outliers in the shorter priced ranges as I haven't rated many runners less than say $1.5 but my system is suggesting that my predictions are pretty well calibrated to the actual race results.
Back to Top
Delta Deel View Drop Down
Champion
Champion
Avatar

Joined: 25 Aug 2015
Status: Offline
Points: 8896
Post Options Post Options   Thanks (0) Thanks(0)   Quote Delta Deel Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 12:07pm
Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Originally posted by Delta Deel Delta Deel wrote:

Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Originally posted by Delta Deel Delta Deel wrote:

I have software that does this for me. Just put in any rated price you want to check and you can run it for every race you have rated or even back test it against past results with previous data(mine goes back to 2011) and it will give you %SR, POT, ect


Sounds like I've just wasted the past 6 years of my punting life if you can buy it off the shelf. Can you adjust factors on historical data and see how it effects profit and accuracy?

I've had friends in the industry interested in commercializing my method/system but sounds like it's been done.


Not really sure by what you mean by adjusting factors. With my software I just enter my rated price in the search criteria and you can also choose from multiple other options as in age, sex,class of race, weekday, race no, metro, country, prov, track, date, day of the week, track conditions and about 40-50 others and it will run and draw out all of these. I can also compare my rated price vs the official SP, TAB Fixed and best of totes.

IE I can say I want to look at every horse I have rated $1.25 since 2011 and it will run and produce every horse with that rated price, Finish Position and SP as well as give you SR, units in, units out and Profit%

I prefer to use ranges myself, I often search a range say Rated Price Range 1.01- 1.40 or rated price 21.00-31.00



Okay sounds like it is not the same as what I do. I can't quite do what you can as easily but I have a graph of all rated horses from 0% - 100% chance of winning and it compares the SR vs my rated prices. That's all I really want to know.

As long as my long shots and favorites aren't winning or losing more than I predict, I'm happy. If I started separating results based on certain criteria the sample would be too small anyway. 


You would be able to do the same with the software, I can show my ratings as Rated Price, Ratings Net Score, and rating % chance so it would be just a matter of dumping the data dump into in excel and using the tables and graphs as you do. I searched and plotted all rated horse that rated less than $1.50 since 2011 and it took about 25min.

Metro runners rated less than $1.40 showed a 7% profit with no further analysis required so am working that angle today for a new bet selector!!
https://group1selections.wordpress.com/
Back to Top
Formerly Kincsem View Drop Down
Yearling
Yearling


Joined: 22 Oct 2016
Location: Melbourne
Status: Offline
Points: 244
Post Options Post Options   Thanks (0) Thanks(0)   Quote Formerly Kincsem Quote  Post ReplyReply Direct Link To This Post Posted: 20 Apr 2017 at 2:04pm
My own sample of horses rated below $1.40 would be very small and unreliable. Also from what I can tell and what I've read, long shot and short price bias doesn't exist anymore. Not sure if that 7% would hold up over a large sample but I could be wrong.
Back to Top
Mayor View Drop Down
Admin Group
Admin Group
Avatar

Joined: 15 Feb 2007
Location: Australia
Status: Offline
Points: 3075
Post Options Post Options   Thanks (0) Thanks(0)   Quote Mayor Quote  Post ReplyReply Direct Link To This Post Posted: 19 May 2017 at 2:02pm
This thread is featured in this week's village newsletter. Thanks to the contributors.


(Please sign up on the page for future newsletters.)

Mayor
Mayor
Back to Top
Dutch View Drop Down
Champion
Champion
Avatar

Joined: 18 Feb 2016
Location: Sunshine Coast
Status: Offline
Points: 708
Post Options Post Options   Thanks (1) Thanks(1)   Quote Dutch Quote  Post ReplyReply Direct Link To This Post Posted: 19 May 2017 at 5:59pm
Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Does anyone here who frames markets, calibrate their predicted probabilities (odds) with race results?

Simply, it is testing whether over the journey horses you price, actually win as regularly as your odds would suggest.

Ex. Do horses you rate $10 chances win 10% of the time, $2 chances win 50%, horses rated $100 win 1% etc?

I thought there would be outliers that would skew certain price ranges but literally across the board the odds aligned almost perfectly with the results.

It would be off topic for me to respond to this question simply because I do not frame my own markets (rate each runner).

But please allow me to insert this post into the discussion, because I do think it has a place in such a discussion.

The problem I see with rating horses, and then collating results to see whether your rated results actually pan out as rated/predicted (although rated and predicted are not the same thing) is that you have no way of knowing whose "turn" it is to win today's race. 

That is not sarcastic, though it might be sardonic, because you can rate the race to death, but still not get the winner. 

Frequently there are two chances in a rate priced (not rated) at <$3. That means that the race percentage is between 66% and 80% (if both were between $3.00 $2.50).

Under the theory of wining the predicted percentages

Originally posted by Formerly Kincsem Formerly Kincsem wrote:

Ex. Do horses you rate $10 chances win 10% of the time, $2 chances win 50%, horses rated $100 win 1% etc?
we are still not guaranteed to have the winner covered - only between 66% and 80% of times will we be correct in wagering on these kinds of situations.

But even being "right" 66% to 80% of the time, does not bring success, so I do not find anything helpful in such an approach. 

Ratings would need to be very good to achieve such percentages, and of course I acknowledge that this IS possible with the right edge in rating certain races.

I also acknowledge that the OP is talking about RATED horses, as opposed to the market's prices, and of course you'd bet only in races where your rated prices are a lot shorter than the available prices (ie overlays). 

I wanted to throw this in to the discussion because even betting on the overlays is still no guarantee of getting a specifically rated race "right."

Yet I agree that over time, this is a better edge than not rating races at all.

Thanks for allowing the intrusion.


Hell, there are no rules here - we're trying to accomplish something ~ Thomas A. Edison
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down

Forum Software by Web Wiz Forums® version 11.10
Copyright ©2001-2017 Web Wiz Ltd.

This page was generated in 0.063 seconds.