So I hope you're aware by now of the High Heat Stats blog, which is basically what the Baseball Reference blog used to be, but is no longer hosted at BB-Ref.
They've done a fun couple posts lately, attempting to list the 50 most overrated and then most underrated hitters in baseball history (today, they're starting on the pitchers). Being specifically a stats blog, of course, they were pretty limited in the ways they could come up with to arrive at lists like that. I think the one they devised (with the help of the excellent Sky Kalkman) was pretty brilliant.
Basically: use the player's career rWAR as a proxy for his true worth/value/talent/etc. Then take BB-Ref's EloRater -- a system which gauges where the thousands of BB-Ref users rate each player -- as a measure of how the fans view the player. Then, figure out an expected WAR (eWAR), representing the career rWAR we'd assume a player would have if his production merited the rating he gets on the EloRater, based on the players surrounding him in the EloRater rankings. So, the players with the highest eWAR-rWAR are your most overrated players, and the ones with the highest rWAR-eWAR are your underrateds.
Ingenious...but not necessarily perfectly attuned to tell you what the headlines promise (which they happily recognize, and I'm not sure any stat or collection of stats could do that). Some of the guys who show up on the "overrated" list spent time in the Negro Leagues, or were otherwise prevented from putting up a career rWAR that accurately reflects their worth; most of the underrated list ends up being nineteenth-century guys, whose "replacement level" was likely much, much lower than the current ones. They're interesting lists, but they don't necessarily reflect the most under- and overrated players in baseball history.
I want to focus on two names in particular, though, from the overrated list: Ted Williams and Joe DiMaggio.
Williams shows up as the #2 all-time position player by the EloRater system, which yields an eWAR of 163.4. His real-life rWAR, though, was "only" 125.3. High Heat's system thus concludes that he was "overrated" by 38.1 WAR -- 30% of his actual career WAR, or one full Moises Alou.
DiMaggio ranks #15 (he's actually 16th as I look at it now, but must've been 15 a few days ago), in the neighborhood of Alex Rodriguez, Mike Schmidt and Frank Robinson. His eWAR is thus 108.3, but his actual rWAR just 83.6, so he's "overrated" by 24.7 WAR -- also 30% of his total, or one complete-to-date Dustin Pedroia.
The spanner in the works, as you may already have worked out, is that both these guys served in World War II during what would have been the primes of their careers, and Williams threw in Korean War service, too, just for kicks. If what we're actually doing is giving a guy credit back for time he spent in active military service, I don't think it would be accurate to say that constitutes "overrating" that guy (and I have no doubt the High Heat guys would agree).
So here's what I did. I took the average rWAR of the six years surrounding the years DiMaggio and Williams missed most or all of due to war service -- 1940-42 and 1946-48 for both players, then a second average using 1949-51 and 1954-56 for Ted -- and made that their seasonal rWAR for each year that was missed. So, the average of DiMaggio's 1940-42 and his 1946-48 was 6.9 rWAR per season, so he gets credit for 6.9 in each of 1943, 1944 and 1945. Add that to their career totals (removing the WAR Williams actually earned in his very partial years in 1952 and '53) to get what we'll call xWAR (x just because I feel like all the other letters are pretty well taken at this point), the total we might have expected them to end up with in a peaceful world. Here's what I came up with:
So by this (kind of arbitrary, but reasonable) method, both end up being "rated" within 4% of their actual career value -- close enough, essentially, to be considered exactly right. And considering it was arrived at by a system that involves users comparing against one another endless pairings of (not quite) random players, and that we're talking about players who were just plain absent for a total of eight prime seasons, that's pretty astounding accuracy. Probably something of a fluke, but a pretty impressive one, at least. Go humanity!