"What is the evidence, and what does it mean?" Bill JamesThe first version of this article was mailed to the Blue Jays' mailing list Oct 27, 1997. Since then, some excellent feedback from Victor Kwan has led to some changes in the comments on the "significance" of the clutch results, though the conclusions have not changed (it should also be noted that Kwan has not reviewed the changes I made and isn't necessarily endorsing what I wrote).
How did Joe Carter get 102 RBI when he was such a poor hitter on the worst-hitting team in the league? Looking at the situational stats revealed that in fact Carter had a comparable number of RBI- opportunities to the league leaders, including more RBI-chances than Ken Griffey Jr., the league's RBI-leader. Though Carter's stats in the clutch were better than Carter's overall stats, looking at each situation in detail suggests Carter's clutch hitting only netted him 8 extra RBI. Carter appears to have picked up just 6 RBI on ground outs.
I picked 7 players for this comparison: Carter (the player of interest), Ken Griffey Jr., Tino Martinez and Tim Salmon (#1, #2 and #4 in RBIs in the A.L.), Jim Thome and John Olerud (both had 102 RBI, same as Carter), and Rusty Greer (had a surprisingly low RBI total (87) compared to his Estimated Runs Produced (121)). The data was taken from the CNN/SI web site (links below).
Even though Joe Carter played for the lowest-scoring team and Ken Griffey Jr. played for the highest-scoring team, Joe Carter had more RBI-chances than Ken Griffey Jr. in 1997, and had a comparable number to the other league leaders, especially with runners on third-base:
At Bats with Runners in Scoring Position (runner on 2nd or 3rd) TMartinez 186, Salmon 178, Carter 170, Greer 170, Griffey 146, Thome 132, Olerud 130 Plate Appearances (AB+BB+HBP+SF) with Runners In Scoring Position TMartinez 237, Salmon 231, Greer 207, Carter 203, Griffey 188, Thome 185, Olerud 175 At Bats with Runner on Third Base Carter 76, Salmon 69, TMartinez 67, Greer 57, Griffey 54, Thome 50, Olerud 47 Plate Appearances with Runner on Third Base Salmon 98, Carter 96, TMartinez 94, Griffey 83, Greer 78, Thome 72, Olerud 68 RBI Griffey 147, TMartinez 141, Salmon 129, Carter 102, Thome 102, Olerud 102, Greer 87
How did Carter get more chances than Griffey? Carter and Griffey played in the same number of games (157) and Griffey had more plate appearances overall. Otis Nixon had a .347 OBP and stole 48 bases in front of Carter, Orlando Merced had a .349 OBP, and Nixon would seldom have been driven in before Carter came up, so apparently this all worked out to a lot of chances for Carter. I only have a few Mariners' scoresheets, but it looks like Piniella favored his fast runners at the top of the lineup and batted his top-OBP guys *behind* Griffey (in Game 3 of the DivSeries, it was Amaral (.320 OBP, 12 SB) leading off, Kelly (.331 OBP, 9 SB) batting 2nd, Griffey (.646 SLG) 3rd, EMartinez (.451 OBP) 4th, ... , Buhner (.379 OBP) 6th, Blowers (.380 OBP) 7th, ... ), though sometimes Cora (.359 OBP) and ARod (.346 OBP) batted at the top.
Carter hit better with runners in scoring position than not. To find out how many extra RBI he got by distributing his hits this way, I looked at each base situation, projected his season average stats onto the same number of plate appearances, compared to how he actually did in that situation, and from that estimated the extra RBI.
For example, here is the calculation for Carter with the bases loaded:
PA AB S D T HR BB HBP SF
Actual: 15 10 4 0 0 2 4 0 1
Projected: 15 13.9 2.0 0.7 0.1 0.5 0.9 0.2 1.4
Difference: 2.0 -0.7 -0.1 1.5 3.1 -0.2 -0.4
Extra RBI: 3.6 -2.0 -0.3 6.0 3.1 -0.2 -0.4 ==> +9.8 RBI
Note: S=Singles, D=Doubles, T=Triples, BB=Walks, SF=Sacrifice Flies
Doing similar calculations, I got
runners 2nd & 3rd: +2.3 RBI
runners 1st & 3rd: +4.0 RBI
runner on 3rd only: +0.9 RBI
runners 1st & 2nd: +0.8 RBI
runner on 2nd only: -6.6 RBI (BA .140, OBP .246, SLG .211 in 65 PA)
runner on 1st only: -2.0 RBI
none on: -1.0 RBI
Overall, that adds up to 8 extra RBI by doing better in the clutch than not. In other words, Carter's opportunities were so plentiful that a mere .234 hitter with a woeful .281 on-base percentage and mediocre .399 slugging percentage would expect to get 94 RBI (assuming Carter did not get extra RBI on outs, which is looked at next). Ken Griffey Jr. probably would get close to 170 RBI with the same opportunities.
Carter had 96 plate appearances with a runner on third-base and got 64 RBI in those appearances. Based on looking at each situation, I believe only 49 of those RBI came from hits, walks or hit-by-pitch (I could be off by 1 or 2, e.g. it's hard to know for sure how many singles got in a runner from 2nd-base). That leaves 15 RBI which came on outs (or reaching base on error). Carter had 9 sacrifice flies, so I'm estimating Carter only got 6 RBI (15 - 9) on ground outs. Carter did not reach base in 59 of the 96 plate appearances with a runner on third (i.e. he was out or reached base on error), so I believe he got 15 RBI out of those 59 outs.
Is 15 RBI in 59 outs (with a runner on third) a good ratio? A big factor would be how many of those appearances were with 2 outs. I believe an average hitter gets the runner in from third on an out about 40% of the time when there are less than 2 out (this is based on the A.L. average being 57% at driving in a runner from third-base with less than 2 out in 1989, and the A.L. batting average that year being .261, and (.57-.261)/(1-.261)==.4). If you figure two-thirds of Carter's outs were with less than 2 out, you expect 16 RBI from outs, though possibly more of Carter's situations were with 2 outs. Even if Carter got more RBI from outs than an average hitter in his situation, it was probably only a few. So I believe it's fair to stick with the estimate that Carter got only 8 extra RBI overall from his clutch hitting.
Carter had 45 hits in 170 at bats with runners in scoring position (.265 BA) and 98 hits in 442 at bats with no runners in scoring position (.222 BA). What is the chance of a .222 hitter hitting at least .265 in 170 at bats? About 11%. More generally, a .265 average differs from .222 by 43 points. What is the chance of a .222 hitter hitting at least 43 points different in 170 at bats (i.e. of hitting .265 or higher, or .179 or lower)? Almost 20%. Normally statisticians want to see a significance level below 5% before considering the result to be contradictory to the hypothesis (and even then, the hypothesis should be made before the data is collected).
Note that this argument doesn't prove that Carter doesn't have a special ability to hit better in the clutch, nor does it take anything away from Carter's value in 1997. But the hypothesis that Carter's "true" batting average in the clutch is the same as his actual batting average in other situations is not contradicted by his results in 1997, so it seems unwise based on this year's clutch performance to think in future years he will hit better in the clutch than not.
The Estimated Runs Produced formula estimates that Carter produced just 69 runs in 1997. His clutch hitting suggests his value may have been higher, probably by at most 4 runs (remember that a lot of the credit for a run batted in has to go to the runner who got into position to score it). 4 more runs would change Carter's R27 from 3.8 to 4.0, still well below the league average of 5.0, and awful for a 1B/DH (the average 1B/DH is around 6.0).
When I was looking for information on how often runners are driven in from third-base, the closest I could find was the data I cited above from the 1990 Elias Baseball Analyst. By a fluke, I also stumbled upon this passage regarding Joe Carter in 1989:
The 1990 Elias Baseball Analyst, pp. 157-8:
"Last season, Carter drove in 105 runs although his Cleveland Indians ranked last in the American League with an average of 3.73 runs per game. . . . It might seem that Carter overcame a mightly obstacle in driving in 105 runs for a team with such a poor attack. . . . But a closer look reveals that Carter was part of Cleveland's problem. Given the number of runners in scoring position when Carter batted, 105 RBIs hardly represents clutch hitting; only one player in either league had more opportunities last season . . ."
The moral of the story is, don't use how good a team is as a proxy for how many RBI-chances a particular hitter on that team gets.
In the numbers above, notice that the hitter who had the fewest RBI-chances of those listed also managed 102 RBI. John Olerud hit like Ken Griffey Jr. in clutch situations:
BA OBP SLG BA OBP SLG
Olerud: none on .252 .357 .407, scoring position .385 .497 .646
Griffey: none on .281 .354 .609, scoring position .336 .420 .671
Carter: none on .217 .255 .373, scoring position .265 .340 .471
Greer: none on .333 .416 .578, scoring position .259 .382 .435
To run the same significance test as done for Carter, Olerud was 50 for 130 with runners in scoring position (.385), and 104 for 394 with no runners in scoring position (.264), a difference of 121 points. The chance of a .264 hitter hitting at least 121 points better or worse in 130 at bats is less than 1%. So, if you had hypothesized that Olerud was a .264 hitter in clutch situations before the season, this result would contradict the hypothesis, but it's not valid to conclude a contradiction when the hypothesis was made after the data was observed. Also, if you had hypothesized that Olerud's "true" batting average was .309 in all situations, then neither 50/130 nor 104/394 contradicts that, nor does his overall average of .294 in 524 at bats. So I'm not convinced yet that Olerud has a special ability to hit better in the clutch, though he had a great year in the clutch in '97, even compared to Joe Carter.
That's been the story of the year for Joe. Even when we find a stat in which he did well, the man he replaced did better. (Well, they tied on salary.)
The situational stats were taken from the CNN/SI web site, for example:
http://www.cnnsi.com/baseball/mlb/ml/players/Joe.Carter/BaseSituation.html http://www.cnnsi.com/baseball/mlb/ml/players/Ken.Griffey.Jr/BaseSituation.html
(thanks to George for pointing out this site, though CNN/SI should get some credit for getting into position to be pointed out :-) ). Note: these links now contain the '98 numbers instead of '97.
When I've cited OBP above, if it was OBP in a certain situation, it would be from the CNN/SI site and hence be the official OBP ((H+BB+HBP)/(AB+BB+HBP+SF) if I haven't forgotten something). When I cited OBP for a full-season, it was based on just (H+BB)/(AB+BB) from Doug Steele's unofficial site http://shell.rmi.net/~doug/1997.html which is also the source for the data used to calculate the below:
BA OBP SLG R27 ERP RUN RBI BB SB 2B HR PA
Jim Thome CLE .286 .425 .579 8.68 118 104 102 120 1 25 40 616
Ken GriffeyJr SEA .304 .382 .646 8.55 140 125 147 76 15 34 56 684
Rusty Greer TEX .321 .404 .531 7.60 121 112 87 83 9 42 26 684
Tino Martinez NYY .296 .375 .577 7.54 122 96 141 75 3 31 44 669
Tim Salmon ANA .296 .394 .517 7.06 115 95 129 95 10 28 33 677
John Olerud NYM .294 .392 .489 6.86 98 90 102 85 0 34 22 609
Joe Carter TOR .234 .281 .399 3.81 69 76 102 40 8 30 21 652
AMERICAN LEAGUE '97 .271 .338 .428 5.06 89 89 84 63 12 20 684
BA: Batting Average (H/AB)
OBP: On-Base Percentage, estimated as (H+BB)/(AB+BB)
((H+BB+HBP)/(AB+BB+HBP+SF) in situational stats above)
SLG: Slugging Average, (H+D+2T+3HR)/AB (same as (S+2D+3T+4HR)/AB)
ERP: Estimated Runs Produced, 0.16*(3H+2D+4T+6HR+2BB+SB-0.61(AB-H+CS)),
background info at http://www.stephent.com/jays/erp.html
R27: ERP per 27 outs, (26.0*ERP/(AB-H+CS)), estimate of how many runs a
lineup of that player would score in a game, AL avg roughly 5.0
AB: At Bats, BB: Walks, SB: Stolen Bases, CS: Caught Stealing
PA: Plate Appearances (estimated as AB+BB)
((AB+BB+HBP+SF) in situational stats above)
H: Hits, S: Singles, D: Doubles, T: Triples, HR: Home Runs
HBP: Hit-By-Pitch, SF: Sacrifice Flies.
-- Stephen Tomlinson http://www.stephent.com/jays mailto:comments@stephent.com Ottawa, Ontario "What is the evidence, and what does it mean?" (Bill James)
Access count for this page:
Last Updated: 1998 Aug 24
Comments are welcome at comments@stephent.com.