Talking Projections and Data-Based Drafting with Ariel Cohen

The fantasy baseball industry is driven by data and many analysts go so far as to develop their own systems for projecting value. Recently, ATC has become more popular as a baseline for those who are seeking a way to rank players for the upcoming season.

I recently had a conversation with the man behind the numbers in order to gain perspective on his unique system and how to best apply it on draft day.

Ariel Cohen is the creator of ATC projections, reigning FSWA baseball writer of the year, and former presidential candidate (really). 

How to Use Projections

David Emerick:  As someone who publishes projections, can you tell us how most projection systems work and what made you design ATC in response to what was already out there?

Ariel Cohen: I’ll tackle how ATC came about first. In the year 2010, I simply wanted to improve my odds of winning my home leagues. I figured, let’s start by using the most accurate projections that I could possibly find. I didn’t know which projections were better than others, so as any good actuary would do, I initially just used them all. I compiled spreadsheets of all the projections that I could get my hands on and averaged them. Everything that I’ve learned in the actuarial field suggested that averaging different models would produce a better overall result.

ATC isn’t a substitute for other projections... it allows the best components of each system to shine brighter.

That was a great start. At the end of the year, I went back to look and see which individual projections performed better than others. When I started to look into each statistic by projection system - different projections performed better or worse for certain stats. One system might have been awesome for HRs, but was poor for pitcher strikeouts. Another might have been mid-pack in RBI - but was lousy in pitcher walks.

It dawned on me that using a straight average of projections is not the ideal way to aggregate the lot. Instead, weights should be drawn closer to the individual accuracy of its components. So I ran regressions and calculated some prospective weights. The better power systems got a larger share of the total for power, and the better speed systems earned more weight in the speed totals. The following year, the first iteration of ATC was created. ATC stands for Average Total Cost [and yes, ATC are also my initials].

Over the years, I’ve gotten a lot more sophisticated and the data is more robust. With each passing season, I am more accurate. But the general method and principle from the first iteration still holds true today.

Most projection systems use external variables to project stats. Power might be projected based on a player’s average flyball distance, launch angle, barrels, and a bunch of others. Initial HR totals are calculated, and then are ballpark adjusted, regressed for age, and possibly compared to historical levels. ATC isn’t a substitute or competition for other projections. Rather, it is an enhancement; it allows the best components of each projection system to shine brighter.

The key is not to get any one player right - the key is to be directionally right on more players.

DE:  How do you think fantasy managers misuse projection systems?  When we’re looking at ATC projections, what should we be thinking about in terms of variability and reliability? 

AC:  To be honest, I think that most fantasy managers either disregard projections or simply use them as a tool mid-draft to take a sneak peek at a player’s stat line. Others use projections simply to confirm or deny their own general intuitions.

ATC is orchestrated to reduce minimum bias via the “wisdom of crowds.” While other projections could show huge outlier players (either up or down), ATC will be more reliable. Because ATC is an average of other projections, you won’t find an outlandish set of figures; its stability is its key. ATC will help you identify more “profitable” players than other systems, and similarly, it will steer you away from a larger number of “unprofitable” players. The key is not to get any one player right - the key is to be directionally right on more players - which is what ATC facilitates.

ATC is the best “base” set of projections that you can have. Start your exploration into the overvalued and undervalued players with the wisdom of crowds.

DE:  How can we do better? How can we use projections more effectively? 

AC:  To me, projections should be the starting point of anyone’s journey into player evaluation. You could be found guilty of hubris if you think that on the whole, you yourself - unaided - can outperform calibrated projection models. Sure, for any individual player you may have better insight, or you may discover a flaw in the model. But against any fairly accurate projection model, you cannot beat them in the long run.

I believe that the best advice is to always start with a projection set - and then adjust according to your own intuition or knowledge. ATC is the best “base” set of projections that you can have. Start your exploration into the overvalued and undervalued players with the “wisdom of crowds.” ATC represents the best of the projections, so why not start here?

Strengths and Weaknesses of Projection Systems

DE:  Certain stats, like batting average and ERA, are heavily regressed in most ranking systems. How should fantasy managers handle categories where the projections seem so close to one another? 

AC:  If I were to tell you that a player will either exactly hit .290 this season or .270 this season - what would you project him for? If you say .280 - you would be projecting the average, but of course, you would also be wrong. That player can’t hit .280, as we said. But I’d still guess .280 - to obtain the least amount of possible variance in the long run.

Projections are regressed. They won’t encompass the true variance of the final individual batting average distribution, but they will be the best guess for each individual player in its own right. The average of players will all seem closer to one another than what actually happens in a single season alone. That is intended.

There isn’t much to do here. The idea is to get your individual player selections right, rather than model the MLB distribution. You want to find the long-term undervalued players, and the regressed averages still give you the correct relationship between player skills.

That being said, when I do prospective valuation of players (generating player auction prices) - I add in a slight alteration in my process to account for this.

If the projected range of a category is markedly different from that of the past season, I will tilt the range number to align more with the past range. So if the standard deviation of saves in 2019 is 20, but I am projecting only a 10 S.D. for 2020, I’ll adjust that range. I’ll employ a value closer to 15, etc. I do so because I want to make sure that the projections aren’t totally misaligned with the true nature of the statistical distribution. In that way, I implicitly adjust the distribution of projections to a more realistic curve.

Was that too technical? 🙂

DE:  You mentioned that ATC’s stability is its strength. I think that one reason managers tend to undervalue projections is a sense that projections don’t seem to reflect breakouts or collapses. First of all, is it accurate that projection systems don’t capture breakouts and collapses? Secondly, if it is accurate, is that just because projections follow the pattern and breakouts/collapses are exceptional events?

AC: I don’t believe we need projections to predict full breakouts or collapses. Sure, obviously if our projection systems knew that Josh Bell would hit 37 homers and 116 RBI last year, that would have been fantastically helpful.

But the truth is - all that you would have needed to profit from Josh Bell last year from projections was for them to be somewhat above market value. If Bell’s ADP was in the 10th round - all you would have needed was for Bell to be valued as an 8th-round player. That’s called a “BUY” signal. If Bell is undervalued, he will make his way onto several of your fantasy rosters. Once on the roster - you enjoy the entirety of the upside and profit that he generated.

On the flip side, and perhaps even easier - all you need not to realize a collapse on your roster is for your projections to be somewhat below market. That would signal a “PASS” on the player. You don’t need a pitcher to be projected for a 5.50 ERA, if a 4.50 ERA projection will make you pass on a player that the market values as a 4.20 ERA player.

As for outsized results - projections are just the averages or medians of player expectation. Breakouts or collapses should occur for some each season - simply by the process risk alone.

DE:  Do recent changes to the baseball distort our ability to project player performance?

AC:  Yes, it does, in that the composition of the ball greatly affects the major league run environment. Pitchers who are fly-ball oriented would have an outsized difference in their ERA resulting from a change in HR/FB. Hitters who have “warning track power” may show a 5-10 HR difference in their power output. For a number of players, the ball matters greatly.

Max Kepler was recently asked if his career year in 2019 of 36 HRs had anything to do with the ball. The quick recap of his response was, “yeah.”

For most, as for the change in the ball - we can simply just scale home runs up to fit the average change in power. In that scenario, the ball does not matter as much since everyone scales accordingly. For some though, the power increase may be outsized, which distorts our ability for adjusting projections. Likewise with the pitchers.

Don’t rely on the straight projections alone - analyze the players yourself. If you have a reason to believe the projection is off - go ahead and adjust it.

Are Stats Alone Enough?

DE:  How much do you follow your own system? Do you watch a game or see a stat line and think, “I should go add him,” then check the projection and think “Nah”?

AC:  I always start with ATC. I say ‘start’ because part of the fun, and part of being a good analyst is to have your own opinions. If I identify a player as potentially undervalued by ATC - I will then do my own deep dive to see if my own intuition agrees. Is a newfound level a skill - or perhaps it was just dumb luck? Will a higher strikeout rate continue onwards? If Statcast thinks that the player was lucky - do I agree? And so on.

Don’t rely on the straight projections alone - analyze the players yourself. If you have a reason to believe the projection is off - go ahead and adjust it.

The most common case that I will adjust projections is for playing time. If I believe that the projections don’t realize that a particular player hurt his wrist - and won’t have his power stroke back until mid-season - I’ll take down the power totals when I run my own valuations.

As far as seeing a player play live - sure! If I see something I like or don’t like in a player - I’ll change the projection for my own use. That’s part of the fun in this game!

ATC is an amazing tool, and probably the best starting tool that you can use. But you don’t have to, and shouldn’t use it as the lone source of your draft prep.

If what you personally observe can make a strong case to veer from projections - by all means, adjust!

DE:  Do you ever use the eye test and think that the numbers get it wrong?

AC:  Similar to the question above - absolutely! Computers can’t do what the human eye can. Models do what they do - but the human eye is more trained to spot nuances. GIGO. If what you personally observe can make a strong case to veer from projections - by all means, adjust! You are probably right!

Remember - projections are based on algorithms. They look at 2017, 2018 and 2019 data. If an event or injury that occurred in 2018 affected a player’s statistics, AND it is NOT something that is properly accounted for a model - a going forward projection wouldn’t be accurate.

Let’s say that a player’s wife had cancer in 2018. He was completely healthy, but his power was cut in half that year. Perhaps he was more tired and worried about his family. A projection algorithm does not have a “worried about family” variable. The algorithm will simply assume that his skills are diminished. In this case, the human eye and reasoning ability will be the better predictor.

I speak of this example from experience. I believe that in 2011, Adam Dunn’s daughter was sick - and it weighed on him. His 11 home runs were uncharacteristically low for the White Sox slugger. I knew of this and realized that projections took 2011 as a skills decline. But the reality was that his skills were the same. I was able to draft a 40 HR player for just $1 that season en route to winning a few home league championships.

DE:  Based on projections versus ADP or human elements that aren’t captured by the numbers, who are a couple of players you’re targeting this year?

AC:  In general, I try not to call players “targets.” Obviously, there are players that I view as somewhat undervalued and a handful that are very undervalued. There are players I will have more shares of than others - I call them “potential bargains.”

Brian Anderson and Bryan Reynolds are two players that appear to be strong potential bargains. None are that exciting (to others), which helps keep their draft price down. But both provide enough support in at least four of five of the roto categories and are a safe bet to return a profit.

On the pitching side, Miles Mikolas is a player that had a poor 2019 after a wonderful 2018. The pendulum has swung both ways - and the question is … where will 2020 end up? While I don’t believe that the pendulum will sway all the way to his incredible ’18 results - I believe that the shadow of ’19 sets market expectations closer to the dark side. It also doesn’t help that he has a relatively low strikeout rate for a fantasy pitcher.

If you simply set Mikolas’s value close to the middle of the prior two seasons, he will earn a nice profit as a mid to late-round draft pick. I personally believe he will exceed projections, but the point is he should be profitable either way.

I will also be drafting Chris Archer this year. Just kidding.

More 2020 Fantasy Baseball Advice