| 0 |
Subject: Baseball Prospectus 2008
Posted by: Seattle Zen
- [49112418] Wed, Feb 27, 2008, 16:56
Got my copy yesterday! Amazon.com is great, I bought it back in January and it only cost $13.
Here is my quick review for those with little attention span:
In 2008, everyone will suck.
Here's a little longer version:
In 2008, Baseball will enter into a malaise. Every batter who played a full season in 2007 will hit fewer homers, score fewer runs, knock in few teammates, yet somehow every pitcher's ERA and WHIP will get worse, or regress. Welcome to baseball stagflation.
Hey, I really love reading the Prospectus, it resonates with the inner stat geek inside me that memorized batting averages from the backs of baseball cards when I was 8. I think they do the best job of quantifying what happened in the past. I like VORP (value over replacement player), its the best stat I've seen that is batter and pitcher equivalent.
But their PECOTA projections simply suck. I'm not exaggerating when I say that everyone sucks next year. Here's my biggest beef so far:
Here are the top three pitching VORP projections:
Johan Santana 54.5
Jake Peavy 52.3
Brandon Webb 51.7
Okay, here were a few of the top VORP's from last year:
Jake Peavy: 77
B Webb 66.1
CC Sabathia 65.2
Fausto Carmona 64
Brad Penny 61.7
John Lackey 60.7
Roy Oswalt 59.8
Tim Hudson 59.7
Josh Beckett 58.6
John Smoltz 56.7
Dan Haren 56.4
Eric Bedard 54.9
A Harang 53.8
Johan Santana 51.7
J Vazquez 51.1
So, what we have to look forward to in 2008 is Johan Santana basically pitching like he did last year, but being the best pitcher in baseball. His projected 54.5 would have been the 13th best performance in 2007, nearly 50% behind Jake Peavy.
Of course, everyone else on that list will pitch worse, much worse in most cases.
The pitchers they predict having the biggest increase in VORP from last year to this are Scott Olsen (+31) and F Liriano (going from zip to 29). Just take a look at the best performances of 2007 and fully EIGHT of them had VORP increases of 30 or more.
Batters are nearly as bad. They predict the top VORP decreases will be staggering: M Ordonez at -56.7, Ichiro -48.8, down to number 15 Todd Helton at -28.8 while the top increases are all middling players or fresh faces, not a single player do they predict will go from 40 or more VORP up to 70 or 80. Every year at least a few will and they can't grasp that fact.
Who will be VORP's biggest increase with a bat, you ask? Felix Pie, +27.1
I love reading the individual team essays and there are essays in the back on "Translating baserunning into runs", "Quantifying the Impact of Outfield Arms", and "Why pitchers need to work the count, too." that are great. As for using BP to predict who will break out in 2008 and go from the 30th-40th ranked starter or 60th-80th ranked batter to the top 20, forget it, they simply don't have to balls or ability to make those calls. |
| 1 | blue hen
ID: 16322314
Wed, Feb 27, 2008, 17:06
|
I don't subscribe to their premium website anymore, but I get the book every year. I think mine will be delayed because I packaged it with the new Bill James back in December.
|
| 2 | Flying Polack
Sustainer
ID: 378582811
Wed, Feb 27, 2008, 17:36
|
I got my book earlier this week. I haven't read it yet, saving that for my flight to Colorado tomorrow!! Can't wait to crack it open. I agree with you SZ, I'm mostly excited about the Team Essay's.
I've always had a problem with the same issue you just addressed. I think they explain it away because they are reporting each players weighted "mean" forecast.
Nate Silver is a pretty arrogant dude, I'm growing pretty tired of reading his articles talking about PECOTA as if it's some great PREDICTIVE TOOL. He also hasn't done an analysis recently comparing his projections with Shandler, Rotowire, etc. I wonder why..
My premium subscribtion is up in a month, still haven't decided whether to renew or not. They've annoyed me recently. They ranked the Fantasy OFer's today and didn't have Curtis Granderson as Top 10 in CF.
Here are their Top 10 rankings
1. Sizemore
2. Beltran
3. C. Young
4. B. Upton
5. Pence
6. Bruce
7. Mike freakin' Cameron and his 25 day suspension
8. A. Jones
9. J. Upton
10. Ichiro
|
| 4 | mjd
Leader
ID: 501381415
Wed, Feb 27, 2008, 18:32
|
Yeah, my subscription ends in 3 day and I'm leaning towards letting it lapse. I was also not happy with their CF rankings.
Mike Cameron???
|
| 5 | biliruben
ID: 5610442715
Wed, Feb 27, 2008, 19:15
|
Cameron rocks if you can take the hit to the average.
|
| 6 | mjd
Leader
ID: 501381415
Wed, Feb 27, 2008, 19:33
|
I just don't think he's in the top 10.
Even less thrilled to pay to read someone who thinks that he is.
|
| 7 | biliruben
ID: 5610442715
Wed, Feb 27, 2008, 19:42
|
I guess.
Just not a lot of guys who can contribute in CF in 4 cats, and his career OBP isn't absolutely horrendous, if they look at that instead of average.
Add that to slotting him into the Mil lineup, and he could be pretty good.
No idea how Ichiro's 10th, however. That is a bizarro list.
|
| 8 | mjd
Leader
ID: 501381415
Wed, Feb 27, 2008, 20:40
|
No problem with Cameron, I have multiple drafts coming up, so I may draft him at some point.
My problem was the list.
|
| 9 | Flying Polack
Sustainer
ID: 378582811
Wed, Feb 27, 2008, 20:49
|
Curtis Granderson is essentially, a younger, potentially better version of Mike Cameron. Basically the worst case scenario for Granderson is he becomes Cameron.
Granderson, age 26 - 23/26/122/74/.302/.361/.552
Cameron, age 34 - 21/18/88/78/.242/.328/.431
I'ts pretty tough to argue Cameron will be better this season. Especially when you throw in his suspension.
BP has been bothering me lately, and since Grany is my favorite player this kind of set me over the edge.
|
| 10 | TacoJohn
ID: 590291817
Tue, Mar 04, 2008, 18:26
|
No offense but the OP here is pretty bad.
Picture 50 of us having a contest to guess 10 coinflips. The correct prediction for each of us is that we get 5 right. Now, of course, one of us will get lucky and manage 8 or whatever. But that doesn't mean you arbitrarily pick one of us and say we will get 8 right; that wouldn't make any sense.
If you asked them to project the VORP of the #1 pitcher I'm sure they'd tell you 75 or whatever, but who knows which pitcher that will be? No one! Putting anyone at 75 would be a ridiculous and arbitrary exercise.
There's just way more uncertainty now than at the end of the year. So, yeah, someone will hit 50 HRs or whatever, but no one is a 50/50 shot to do so. I mean name one guy you would bet even money on to hit even 42 HRs. You can't, there's too much uncertainty.
|
| 11 | Seattle Zen
ID: 49112418
Tue, Mar 04, 2008, 18:41
|
Fair enough, Taco. However, BP is selling themselves as having a system (PECOTA) that is "deadly accurate". I'm just pointing out that is very far from the truth.
They are in the business of guessing. And no, this is has nothing to do with flipping a coin, this is baseball. Great baseball players will generally do better than crap players, coin flipping is nothing but chance.
Putting anyone at 75 would be a ridiculous and arbitrary exercise.
Is it? Looking at Johan Santana, you would have been the ridiculous one if you thought he was going to mail in a 50 VORP season in 2006. Predicting 75 when the man had no equal would definitely have been the right call.
There are other websites, books, people who do go out on a limb making predictions and do a much better job than BP.
|
| 12 | biliruben
ID: 5610442715
Tue, Mar 04, 2008, 18:52
|
Is that true? I recall an analysis years back that graded the predictions of half a dozen baseball sites. All I recall was that rotowire did poorly. Did BP do that bad? Pecota's just a silly program, and I rarely rely on it, but the writers and their analyses are usually pretty good.
I agree that I wouldn't just draft off of PECOTA, if that's what you are saying. That would be like voting for a guy for president because you'd like to have a beer with him.
|
| 13 | Razor
ID: 40219216
Tue, Mar 04, 2008, 20:36
|
PECOTA isn't guessing; it's using math to predict. And if it's predicting that the best pitcher in baseball this year will be worse than roughly 12 pitchers from last year, than it sounds like it's doing a crappy job predicting. When has that ever been the case in back-to-back years? There are strong years and weak years, but they are predicting a 50% dropoff from last year's top pitchers to this year's top pitchers. That seems very unreasonable under any circumstances, much less the fact that most of the top pitchers from last year are on the good side of 30 years old.
|
| 14 | Flying Polack
Sustainer
ID: 378582811
Tue, Mar 04, 2008, 21:06
|
Prediction Analysis
|
| 15 | Seattle Zen
ID: 44122722
Tue, Mar 04, 2008, 23:39
|
Post 14
I don't think that article really shows anything as fantasy baseball players are only interested in individual predictions. I promise you that those other predictors listed only care about that as well. Hell, Rotowire and others don't even make predictions for more than the top 400 or so players. But, if I were to ever enter into a League-Wide OPS and ERA prediction competition, I'd lend BP some credence. Boy, let's start one up, dosen't that sound fun! :)
Bili:
BP bragged on the back of their 2008 book that they correctly predicted that Fausto Carmona would "succeed as a starter despite his struggles in the bullpen." Now, this is their book, they can brag about any of their predictions, but they choose Fausto.
What did they predict last year? That he would go 6-6, pitch 103 innings, have a 4.57 ERA, a 1.44 WHIP, 71 strike outs and have a VORP of 12.4. That's what you are most proud of? Not even close. His VORP was 64. Wow.
Now it is true that they wrote the following: "He has the stuff to succeed as a starter if the Indians give him an opportunity to do so." Obivously they are pointing to their blurb prediction, not their numbers. I've noticed on more than one occasion that their blurb on the player and their predictions are incongruous at times. I like the blurbs. If I didn't like the blurbs, I wouldn't buy the book. My complaint is with their numerical predictions and they are bad, quite bad.
|
| 16 | allhair allstars
Sustainer
ID: 50902421
Wed, Mar 05, 2008, 00:26
|
A somewhat related thread...
|
| 17 | Khahan
ID: 486552412
Wed, Mar 05, 2008, 01:04
|
Post 14: Wow, the PECOTA guys published an comparison they did of PECOTA vs other predictors which showed PECOTA finished first based on the critiera they chose.
Sorry, if that is the best they've got, I'm unimpressed. Nothing against PECOTA. I have never used their system and never examined it. But that article is equivalent to somebody entering Miss America and then deciding what swimsuits and talents her opponents will showcase.
|
| 18 | TacoJohn
ID: 81272523
Sat, Mar 15, 2008, 17:35
|
[13] Razor, you're totally missing the point along with others in this thread.
They are NOT saying the best pitcher this year will be worse than 12 pitchers were last year. They're essentially saying that no one knows for sure WHICH pitcher will be the best this year.
Let's say the 5 best hitters in baseball this year can be expected to hit between 30 and 60 home runs. I'm going to project all 5 of them at around 45 then right? Does that mean I predict no one will hit 50 HRs? Of course not. I'm pretty sure that someone WILL hit 50 HRs, but no individual player warrants that projection.
For any group you will have certain members outperform their expectation and others do the opposite.
|
| 19 | blue hen
Leader
ID: 710321114
Sat, Mar 15, 2008, 19:24
|
TacoJohn - Razor is absolutely correct.
If SOMEONE is going to hit 50 home runs, then wouldn't the person predicted to hit the most home runs have 50 or more?
|
| 20 | Seattle Zen
ID: 529121611
Sat, Mar 15, 2008, 19:51
|
TacoJohn:
They're essentially saying that no one knows for sure WHICH pitcher will be the best this year.
Then don't bother making individual player projections, period.
|
| 21 | dpr
ID: 1733917
Sat, Mar 15, 2008, 19:55
|
Taco has it right
The odds of someone hitting 50 HRs may be very high but the odds of a particular player doing it are obviously much lower.
When making there predictions they do it for each player individually. They are forecasting the mean or median of all their possible performances and using that for there estimate.
|
| 22 | TacoJohn
ID: 81272523
Sat, Mar 15, 2008, 20:39
|
[19] What? No. Because any one of 20 guys could be the one to hit 50 this year.
Let's say I ask you to throw a football 10 times. And you throw it 31 yds, 32 yds, ..., 40 yds (in random order) on your 10 attempts.
Now we decide you'll throw it 10 more times, so we write it out:
Throw 1 -
Throw 2 -
.
.
.
Throw 10 -
And I have to guess your distance for each throw. What's a more accurate strategy for me:
1) Guess 35 yds for every throw
2) Guess one throw to be 31 yds, one throw to be 32 yds, one to be 40 yds, etc.
I know that you will probably throw one for 40 yds, but I'm using strategy 1 every time, and I'm an idiot if I don't.
I just don't see how it is taking this many posts to get through that saying "The top HR hitter will hit 54 HRs this year." and saying "A-Rod will hit 54 HRs this year" are completely different. You can say one with about a 50/50 shot of being right, the other is maybe 1 in 10 to be true.
|
| 23 | TacoJohn
ID: 81272523
Sat, Mar 15, 2008, 20:41
|
[20] I can never know for sure how many MPG my car will get on a road trip, but if I'm on a budget you can bet I'm going to make my best guess.
If Baseball Prospectus told me they could predict who will be the top HR hitter this year, I'd probably cancel my subscription. They can't, no one realistically can on any consistent basis, so we make due with what we CAN actually do which is predict an average expectation knowing that some players will beat it and some players will fall short.
|
| 24 | blue hen
Leader
ID: 710321114
Sun, Mar 16, 2008, 03:14
|
[22] If you asked me to write out a distribution of all my throws, you can sure as hell bet one of them will be over 40. I know I can do it, and I will do it, even if I don't know which throw it will be.
If we predict some player will hit 50 homers, and we predict that Ryan Howard will have the most homers, then we should predict that Ryan Howard will hit 50 homers.
|
| 25 | dpr
ID: 1733917
Sun, Mar 16, 2008, 11:18
|
BH that doesnt make sense. Likely what they do is consider all the factors which the consider important and run multiple simulations with the data that they have to get possible HR totals. Say for Howard they run the simulation 10 times and they get totals of 20, 35, 34, 27, 45, 55, 41, 60, 51, 38, 43. They likely do this even more times for each player. In the end they get a mean estimate and a standard deviation. Say as a result these are there results.
Howard: mean = 45 SD = 5
AROD mean = 48 SD = 5
Fielder mean = 43 SD = 10
Dunn mean = 40 SD = 3
Ortiz mean = 40 SD = 10
Pujols mean = 35 SD = 13
And then obviously there are more players but none have a mean over 50. However all of these give significant likelyhoods of hitting over 50 HRs with the exception of Dunn. For example Howard has a 16% chance of 50, AROD has a 38%, Ortiz has 16% chance, Fielder has a 25% chance, Pujols has a 12.5% chance. Now clearly none of these players would be predicted to hit 50 HRs but the odd of one of them hitting fifty is very high. What would you predict as the these players HR totals using this data?
|
| 26 | Razor
ID: 412371519
Sun, Mar 16, 2008, 19:44
|
Their prediction tool, PECOTA, takes into account all the factors you guys are talking about. They just predicted very low, in my opinion, for the top pitchers. Embarrassingly low. If you want to use the 50 HR-analogy, then you use a measure that is a better comparison. Only two players clubbed 50 homers last year. 12 pitchers reached the 55 VORP mark last year. Does it makes sense to predict that no one in the league would reach a level that many players reach every year?
|
| 27 | TacoJohn
ID: 81272523
Mon, Mar 17, 2008, 01:35
|
They didn't actually predict that no one would go over 55. They just didn't predict any individual player to go over 55. I think this is my last post in this thread.
|
| 28 | Razor
ID: 420241513
Mon, Mar 17, 2008, 08:18
|
I don't think you are really understanding what you are saying. Your first two statements in your last post are direct contradictions of each other.
This reminds me of my high school English class when our teacher tried to explain the concept of fate. Some students insisted that by acting differently in a given situation, that you could change your fate while others understood that fate, if it exists, was unchangeable.
|
| 29 | Astade
ID: 5935164
Mon, Mar 17, 2008, 11:36
|
Where is Sludge when you need him?
|
| 30 | dpr
ID: 1733917
Mon, Mar 17, 2008, 11:38
|
They are not saying that no individual will go over....they are saying that no individual is likely to do it. hence they estimate that no one will. Add up the likeliness of all the individuals to break 55 or whatever and it is likely that someone will.
|
| 31 | Razor
ID: 420241513
Mon, Mar 17, 2008, 15:31
|
Oh, I suspect that BP would agree that someone is probably going over 55 VORP. The question to them is why PECOTA did not predict anyone to do so. PECOTA's VORP projections incorporate means and standard deviations.
|
| 32 | Seattle Zen
ID: 551122511
Wed, Feb 25, 2009, 14:55
|
There are other websites, books, people who do go out on a limb making predictions and do a much better job than BP.
Well, I have spent this offseason reading more and more about predictive systems and I will temper my criticism of PECOTA, but just a little bit.
Tango Tiger tests four predictive systems and says PECOTA ...
finished behind CHONE, Zips, and The Hardball Times for batters, but not by a lot. PECOTA did better with pitchers, but CHONE won overall.
I got my BP book on Monday and it is very similar to last year: they cannot get enough of themselves. Yep, they are "deadly accurate"! I'll let you in on a little surprise - they think Matt Wieters is going to be Lou Gehrig wearing the tools of ignorance this year. In fact, they give him the highest WARP in the entire American League!
This should scare O's fans as they LOVED Felix Pie last year :)
Alright, alright, I'll give it a rest. Hey, I still bought the book, didn't I?
|
| 33 | Razor
ID: 371502414
Wed, Feb 25, 2009, 15:14
|
Back to the original point of this thread, 7 pitchers topped 55 VORP last year, 7 more than Baseball Prospectus thought would. 4 players went over 70 VORP. As was pointed out many times, their predictions for pitchers, especially at the top, were way the hell off.
|
| 34 | dpr
ID: 552411820
Sat, Apr 04, 2009, 16:08
|
sorry to bring up an old thread but I looked at their site today and remembered this thread.
You claim that their predictions for pitchers being way off at the top. I don't have access to there data but the initial post but for these three players they predicted 54.5 (santana), 52.3(peavy), 51.7 (webb). The reality for these guys was 73.6(santana), 51.0 (webb), 50.6 (peavy). While santana was obviously off it seemed they did pretty well on these players. Would you ahve rather seen them predict these 3 pitchers all to be in the 70s and then to have got santana but missed on the other 2?
The reality is that pitchers are so variable year to year. Look at the top 10 from last year. Would anyone have predicted Lee, Lester, Dempster, danks or E. santana to even be in the top 30? Due to these large deviations the expectations (which is what their projections list) are very far from the list of possible values.
Also doesnt there PECOTA projections give percentiles? What do the 90th percentile look like for these players? If enough of them have a ten percent change they are saying someone will do it. They just cant say it for any particular player with certainty.
|
| 35 | Razor
ID: 41323216
Sat, Apr 04, 2009, 16:35
|
Well, they aren't making league predictions with these projections. They are making individual predictions. They built PECOTA to be able to predict what pitchers would do from year to year. They missed pretty badly. Anyone can say, "I think 5 pitchers will have under a 3.00 ERA this year, but I don't know which ones." That's not saying anything of value and that's not what PECOTA is attempting to do.
|
| 36 | dpr
ID: 552411820
Sat, Apr 04, 2009, 16:51
|
Right they made individual predictions. You say they did horribly for the top predictions. It looks like that nailed 2 of their top 3 predictions.
You say you want PECOTA to predict people to go over 55 if you think they think that there will be some people to go over 55. Say they know that there would be 7 people over 55 last year. You think this means they should predict 7 individuals over 55? Lets say thy do that.
Here are the top 15 pitchers selected in the RIBC drafts last year (it makes sense that most selections of pitchers most likely to be among that 7 would come from that 15) Next to them is their VORP for 2008.
Santana 73.6
Peavy 51.0
Bedard <10
Webb 50.7
Sabathia 51.9
Beckett 34.6
Hamels 55.5
Verlander 12.3
Haren 53.7
Lackey 35.3
Smoltz <10
Harang 14.8
Kazmir 36.8
hernandez 42.8
Halladay 70.6
So you are saying because they think (know hypothetically) that there will be 7 pitchers over 55 they should take 7 pitchers from this list (most likely as they were the consensus best) and put them there? IF they did that they would have been horribly wrong on the majority of their predictions. Only 3 of the top 15 pitchers going into the year achieved that mark. 4 of the 7 to get the mark were complete nobodies. But you expect them to predict people to do it? Pitchers vary so much year to year.
|
| 37 | Seattle Zen
ID: 1837411
Sat, Apr 04, 2009, 17:30
|
Hey dpr
In post 32 I said I have tempered my criticism of PECTOA regarding pitchers. I think they do a decent job, better than they do for batters.
|
| Rate this thread: | If you wish, you may rate this thread on scale of 1-5. Ratings should indicate how valuable or interesting you believe this thread would be to other users of this forum. A '5' means that this thread is a 'must read'. A '1' means that this is a complete waste of time. If you have previously rated this thread, rating it again will delete your previous rating. If you do not want to rate this thread, but want to see how others have rated it, then click the button without entering a rating, or else click here. |
|
|
|
Post a reply to this message: (But first, how about checking out this sponsor?)
|
|