One of the easily overlooked but essential factors in player evaluation is the consideration of park effects on player stats. Every park in the Majors affects stats differently, and to truly compare players on an even plane, we must account for this, especially for players in the most extreme parks. The problem, and likely one of the main reasons park effects can at times be overlooked, is that these effects are very difficult to get a good grasp on. We know Coors inflates hitting stats, but by how much, and which stats are most effected? These are difficult adjustments to make intuitively, so too often they are just skipped entirely, or thrown in as an afterthought when fans or mainstream analysts discuss numbers.
On the other hand, there are many who fall prey to another, potentially worse trap in evaluating park effects. They use a player's home/away splits to estimate the effects of the home park on a player's numbers and how his numbers would translate to another park.
There are many downfalls to this approach. The most obvious is that inherent in home/away splits is a home-hitter bias independent of park. Since 2000, hitters have put up an OPS of .774 at home and .745 on the road. So you're going to expect to see a .030 point difference on average even in a completely neutral park, and for some reason, this is rarely accounted for properly by fans who use this method. The assumption is all too often that a hitter's road line is what you'll get from him in a neutral park. In light of the clear home-hitter bias, this is obviously not the case.
Furthermore, there exists the possibility that a particular player will naturally exhibit a stronger home-away split for whatever reason. Some hitters may be more comfortable hitting at home than normal or more affected by traveling to road parks. An exaggerated home-away split does not isolate the home park as the primary factor, and it is common to see players on the same team post widely varying degrees of home-away splits, which should not be the case if these splits were heavily dependent on park effects.
There are also some indications that at least some parks, particularly those with the most extreme hitting environments, can actually suppress road production in addition to boosting home production. For example, pitches break differently at Coors because of the thinner air at altitude, so hitters who tune their swings to how pitches break in Coors will find the ball slightly more difficult to center in other parks than they would if they did not play half their games in Colorado (this phenomenon shows in Colorado's team line-drive rates, as well as in their opponents' rates in Coors).
The fact is, there are too many factors in play in home-road splits to truly estimate park effects. These other factors can be accounted for, but the reason home/away splits seem to be so popular is the ease with which they can be found and reported, and some of these adjustments are beyond the scope of what can be reasonably expected from most fans. Even if you do attempt them, they could take as much or more work than just using park factors to make your adjustments.
We have still not gotten to perhaps the biggest issue with using a player's home/away splits. Even assuming you have properly adjusted for the other issues with using a player's splits, you are invariably going to run into sample size issues. A full season's batting data is already pushing the lower limits of the number of at bats you'd like to see. By going to a player's splits, you are more than cutting your sample in half. You will commonly be dealing with AB totals under 250. To get an idea of the uncertainty present in such small samples, a .300 hitter will hit either under .270 or over .330 about 30% of the time over 250 ABs. You simply cannot garner much meaningful information from a player's home/away splits without at least using several years worth of data, even once you've accounted for everything else.
Take, for instance, Brad Hawpe's 2006 season, when he had over 270 PAs in both his home and away samples. His OPS was .144 points higher on the road, away from Coors. No one would have argued that he would hit .144 points better in OPS if he were traded to a neutral park. Or, consider the famously consistent Albert Pujols. In 2007, he hit .211 OPS points higher on the road. In 2008, he hit .119 points better at home. It's obvious that these data points are not reliable indicators of park effects, but fans and analysts routinely take equally unreliable single-season splits and use them to project severely overblown adjustments to players.
Of course, I can cherry pick outlier seasons to make my point just like any other fan does to make his, so if the above paragraph contained a bit too much anecdotal evidence for your liking, I'm with you. Don't worry. I've got more. Namely, I've got every set of paired seasons since 2000 where a player had at least 200 ABs both home and away for the same team two years in a row (there are 883 of them). Using this data, we can compare a player's splits one year to his splits the next and see how reliable they are. The following graph plots each player's splits one year on the X-axis and his splits the following year on the Y-axis. If the splits are reliable, we should see a definite line forming going from the lower left to the upper right.
As you can see, there is no such pattern here. The data is pretty weakly correlated (r=.21), indicating that a player's home/away splits one year are not a very good indicator of what they will be the next year. Home/away splits are simply too volatile to draw any reliable conclusions from for single players.
If we break down our sample to players who hit at least .100 OPS points higher at home, 41% of these players showed splits of .030 points or lower the following season. Of players who hit at least .150 points higher at home, 38% showed a split of .029 points or lower the next year. This, as we saw earlier, is the average home/away split in Major League Baseball over this time period. If we include all players who returned to within a standard deviation of the average, the percentages rise to 45% and 40%. Of the players who showed splits greater than .150 points, 63% cut that split in half or greater the following year. Players who show very high splits one year are still somewhat more likely to be higher than average playing for the same team the following year, but not nearly enough to tell anything significant by it.
When it comes to adjusting for park effects, player home/away splits just don't cut it as substitutes for park factors. They present a certain allure - they're easy to look up and easy to understand, and they sound convincing in their own pseudo-logical way - and for that they've become quite popular. To get a reliable grasp on park effects on a player's numbers, however, you need to be able to look beyond such crude methods. Ideally, find some park factors, or look up park-adjusted stats. In a lot of cases, you would be better off just not adjusting at all than taking the splits at face value. There are times when home/away splits may be appropriate, but make sure you have a good reason for using them over park factors or park-adjusted stats, make sure you are making the proper adjustments, and make sure you are looking at a large enough sample using several years, or even aggregates of several players if need be. Of course, when you do it right, home/away splits start to resemble park factors more and more. They essentially become a method of calculating park factors. But now we're just splitting hairs.