On comparing players across positions

by Jason Wojciechowski

As many internet baseball fans know, it's Fan Scouting Report time. Each year, Tom Tango asks fans to weigh in on the fielding abilities of the players that they see a lot. For each player with at least ten games played, he asks for a 1-5 ranking of that player's skill in seven areas:

• Reaction / Instincts
• Acceleration / First Few Steps
• Velocity / Sprint Speed
• Hands / Catching
• Release / Footwork
• Throwing Strength
• Throwing Accuracy

Crowd-scouting is an interesting idea. Most of us have no access to professional scouting reports, so getting us all together to compare notes could conceivably provide useful information. All is well until you get to this part of the directions:

Try to judge 'average' not as an average player at that position, but an average player at any position. If you think that Ben Zobrist has an average arm, then mark him as average, regardless if you've seen him play 2B, SS, 3B, or RF.

DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!
DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!
DO NOT CONSIDER THE POSITION THE PLAYER PLAYS!

Maybe it's just my lack of imagination, but I don't understand this. I get the impulse. It would be nice to have a feel for how a player might fare in another position, for instance, and comparing a catcher's foot-speed only to that of other catchers won't necessarily tell you if a catcher likely has the wheels to handle left field. Furthermore, a set of good data about the defensive skills of players without regard to position can give us a better idea of the current state of the defensive spectrum: how much slower are left-fielders than right-fielders right now?; who has better footwork and first-step quickness, second- or third-basemen?

But can "a set of good data" actually be achieved?

Have you ever seen a catcher's first-step quickness, much less his velocity? Jumping out from behind the plate requires some quickness, but no other position has to leave a crouch at the start of a play, so how comparable are catchers to other positions on this scale? And if you tell me you've watched your team's catcher sprint on defense, then you're a professional scout -- they run down to back up first base, sure, but are you paying attention when they do?

How do you compare a second baseman's throwing accuracy to that of an outfielder? What would it mean to say that Mark Ellis has a more accurate arm than Luke Scott? For that matter, even arm strength isn't as straightforward as it might seem. Do you have a good sense of whether Jemile Weeks or Josh Willingham throws harder? (Substitute your team's second baseman and a mediocre-armed outfielder since you probably don't watch the A's very often. Lucky you.)

How do you judge an outfielder's hands compared to an infielder's? The vast majority of outfield plays require no hands skill at all. The plays that do require hands require them on such vastly different balls than a shortstop's plays that I doubt whether "hands" are a legitimate point of comparison.

My point isn't that these things are difficult to judge -- it's that they're impossible as the question is presented, given the reality of roster and position management. Comparing a shortstop's arm to a third baseman's might be hard because their throws can be different, but there is a reasonable basis for comparison such that appealing to the collective wisdom of a group of fans makes sense. Ranking a left-fielder's arm against the general population of baseball players, by contrast, cannot, I posit, be done by watching the games because the actions one sees a left-fielder taking require him to use his arm in a way that is inherently incomparable to the way that infielders use their arms.

The question of comparing players of different positions isn't, I want to note, entirely incoherent. In a controlled environment, it would be trivial to measure players against each other on these criteria. Laboratory testing could answer whether Mark Ellis or Luke Scott throws harder.

In a league where one's position was not such an immutable characteristic, comparisons would be harder, but not impossible -- if most players were like Omar Infante, moving from position to position on the infield dirt and the outfield grass, we would have a hope of understanding how the footwork of a player on the pivot compares to one setting up to throw home on a potential sacrifice fly. As it is, players who float between the infield and outfield are rare enough that evaluating how other players compare to them suffers from a selection problem -- Eric Sogard is a utility infielder precisely because the A's know he lacks something that would make him a feasible outfielder. Thinking of Infante when ranking Sogard's skills may not result in a comparison to the pool of baseball players in general, which is what the FSR asks.

"Garbage in, garbage out" is the typical way of describing why I don't look at FSR results, but that phrase is too general and a tad confrontational -- input can be "garbage" for any number of reasons. In this case, I think a slightly more on point catchphrase might be "meaningless in, meaningless out."