Humans are fallible. Deciding who has won a tennis game or a sprint race can come down to a millimetre-accurate decision. So when an Olympic gold medal is on the line, it’s no wonder we turn to electronics to help us.
But can we really rely on Hawk-Eye, or some other electronic gadget?
Many sports are now using some kind of electronics to assist in the judgement calls of the officials. In essence, these judgement calls are made as to the position and speed of the players. If the players use a moving object, then it’s also the position and speed of that object. Most commonly, that’s a ball.
The position of the player or object is generally measured relative to a line, such as the finishing line of a foot race, or the baseline of a tennis court. These measurements generally need to be done in a split second, requiring expert judges, or the assistance of electronics. In “tight” cases, these judgements are often wrong – and clearly, we need them to be right.
Two outstanding examples are tennis and cricket.
I was at Wimbledon recently, and as I walked into the stadium, I heard the commentators say “Hawk-Eye has got it wrong again, twice, in rapid succession”.
Most of us who have watched tennis will have seen Hawk-Eye in action. It’s an outstanding example of a system that measures the bounce point of the ball relative to the global coordinates of the tennis court, and then tries to infer the position of that bounce point relative to the line.
This is called a “secondary” measurement, and has to be far more accurate than a “primary” measurement, where the bounce point is measured relative to the line itself.
The video below gives a good explanation of how the technology is employed in practice.
Let’s explore Hawk-Eye in the context of tennis and cricket. It gives us a lot of insight on how electronic judging is done in many other sports (thought it’s worth remembering all these sports have very contrasting requirements).
Hawk-Eye uses an area of modern technology called Computer Vision (CV). CV has been enabled by the huge rise in freely-available computing power and our ability to capture images electronically and rapidly.
CV is different to image processing. It uses captured images to infer information about the physical environment around us. It’s central to robotics and to road speed cameras.
The input in CV is an image and the output is information about what is observed. In the case of image processing, the input is an image and the output is another image.
In tennis, we don’t care where the player is, other than when serving, and then only his feet. But we really do care where the ball bounces relative to the lines on the court. In the case of cricket, what we really want to know is where the ball may go after it has bounced.
Hawk-Eye uses exactly the same technology to provide these two contrasting pieces of information about the travel of a ball. It uses very clever algorithms that I will describe later, but it has a very serious failing: it makes a “secondary” measurement.
This secondary measurement measures the flight of the ball relative to its global position in the playing area. In the case of tennis it’s trying to infer millimetre accuracy relative to a playing area that may be 50 metres in length. (In cricket, we are starting to use a technology called “hot-spot”).
This is a primary measurement and is never challenged. It measures where the ball has actually struck by measuring the rise in temperature caused by that collision.
In tennis, Hawk-Eye rapidly captures images of the ball during flight from multiple positions around the court. The cameras have to be very accurately positioned relative to the court lines and the overall court coordinates.
It then uses CV to locate the position of the ball relative to the court multiple times as it travels along its trajectory.
These captured positions are not precise. The ball is a tube-like blur and there’s crowd movement and noise in the background.
These ball location estimates are then combined with the ball ballistics to provide a maximum likelihood estimate of what the ball trajectory may have been. The equation is then used to draw a graphic of the ball’s travel, and infer a bounce point.
It would be a tall order to get it right all the time. Some estimates suggest that Hawk-Eye only gets it right in tennis 60% of the time.
In summary, electronic judging tries to do a good job of reducing the uncertainty of human observation in sport. Mostly it “gets it right” – more often than humans do – but there are no absolutes.
What it definitely does do is add enormously to the entertainment value of sport, and particularly for those of us who watch it on television.
Stephen Prowse
CEO at Wound CRC
I enjoy playing and watching sport; I admire the skill, the dedication, the commitment. Does the outcome really matter that much? Doesn't the referee add to the theatre?
Daryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
Stephen, adjudicators are asked to make judgments that, to be frank, are often beyond the capacity of humans to make with confidence. We expect to much of them. Therefore, if technology can provide assistance in the adjudication process, and there is consensus that it is reliable (even if not perfect), then to me this adds value to the outcome of a contest and, by extension, the experience of fans. I don't go to a sport event to have the "referee add to the theatre". I go with the hope that the referee avoids anything theatrical. Technology can assist that process.
Richard Helmer
REsearch Engineer
Having worked on electronic technologies for other sports, indeed to characterise reliability of an electronic arbitration/judging/scoring system, the only thing that actually matters is whether those that particpate ...athletes, coaches and spectators..collectively accept the technology. It is very difficult to measure anything in the field to the nth degree and so a leap of faith and some humanity is typically required.
like any engineered article, if it continues to get used and adopted then its an indication that it is a perceptual step forward in some way
John Clark
Manager
I am a little intrigued by the "60%" correct. What technology offers the 100% that it aims to achieve?
Chris Barnes
logged in via LinkedIn
This is an example of a ubiquitous problem inherent in the nature of competitions: whether deciding on a merit list of year 12 results, or determining who reaches the finals of the Olympic 100m. In many competitions, there are analogue aspects where winning margins may be arbitrarily small. Practically, we must use discrete yardsticks (such as 0.001 of a second, for instance) for this purpose, and there is always the possibility of a tie for a given choice of yardstick, which should take into account…
Read moreMichael James
Research scientist
I find the basic premise behind the use of these modern tools to be suspect. Quite apart from the dubious accuracy (and not just the basic method but the set-up and calibration for each court then accounting for changes each day, heating and cooling, air humidity, wind etc) calling a ball in when it barely "kisses" the line is fundamentally wrong, and for example does not reflect historically what would have been called in the quite recent past. From the moment Hawkeye was introduced I thought they…
Read moreDaryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
Michael, the net in soccer was introduced well after the goals were the basis for scoring, so there is no logical need for it to actually hit the net. It is there for convenience and, I suspect, because it provides a better appearance in and around the goal. Others will have more details on this than I do.
Turning now to tennis, on what basis do you claim that McEnroe was "correct"? He was certainly a bully, throwing tantrums when decisions didn't go his way. Also, what in your view makes a "competent linesperson"? Presumably someone who is not "near blinded [and] retired". While, as you have argued, Hawkeye may have "dubious accuracy" (it is hardly 100%) it is at least a uniform measure for calls, unlike the various pairs of eyes that squint as best they can to try to ascertain whether a 200km serve is in or out. We expect so much of human eyes!
Michael James
Research scientist
Other than a keen tennis player, there is a long history of other players and commentators essentially agreeing with McEnroe (sometimes hedging their words because of the perceived bully boy thing which I think was a bit overblown). It was certainly agreed amongst top players who were willing to talk about it that Wimbledon habits were not fully professional and needed to change. And they did change, for the better. Quite a few players give thanks to McEnroe.
Incidentally listening to McEnroe…
Read moreMichael James
Research scientist
Actually there is a relatively straightforward technical rationale behind my wish for a significant overlap of ball on line: the "ball" shape is a shadow or profile over the line and would not necessarily mean the ball surface actually touched the line at all (it would have had to have flattened nearly totally to do so). I doubt the studies have been done but for that reason I am guessing that the chalk disturbance test would require considerable overlap.
Daryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
Interesting point, Michael. Funnily enough, both camera technologies and human eyes would need to be "looking" at a flattened ball, rather than a round ball, at impact. So the size of the ball when it strikes the chalk would, I presume, be slightly larger than when at rest. I am not physicist or similar, so I will have to defer to someone with the appropriate expertise to further your point. Thanks for the idea.
Daryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
First, I don't think we can assume that a more "professional" approach to line calling was simply as a consequence of McEnroe's rants. Various sports underwent adjudication reforms in the late 20th century; tennis was not immune to that. But I am happy to concede that McEnroe probably accelerated that process.
Second, I concur about McEnroe the commentator. I have also been pleasantly surprised by the media commentary skills of other controversial athletes - Shane Warne and Lleyton Hewitt…
Read moreMichael James
Research scientist
Shane Warne and Lleyton Hewitt.
Hmm. I suppose I should take the scientific approach and give them the benefit of the doubt until I hear them commentate.....
Your point, or at least partial agreement, is that Hawkeye is making a decision that has no equivalence in the human referee world. It would be one thing if we all agreed it was really, technically that accurate but I wonder if there is anyone who thinks that, including its designers?
And I agree with your last point: ok if the players…
Read moreDaryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
Hi Michael, thanks for referring me to the Hawk-Eye web site and the acknowledgement of a 5% relative error. Goodness knows whether humans do better then this; again I think we need some kind of optical/vision expert to add meaningfully to the discussion. I'm not convinced, for example, that players are particularly good at guessing whether a ball is in or out. Certainly there are many challenges to Hawk-Eye that go unrewarded (unless you want to argue that the machinery is faulty). The other thing to remember is that linespeople and the central umpire still officiate almost every point: Hawk-eye is only used in rare occasions at the discretion of players. So it's not as if technology and machines are "taking over" the game of tennis.
Michael James
Research scientist
You still seem to be missing my point: I do not agree that a mere "kissing" of the (outline) of the ball with the (notional) line edge, is an appropriate measure of being "in". It would be one thing if they employ that 5% error but they don't even do that: I have seen plenty of in calls when there is no effective overlap. And I don't accept the 5% error either: in all likelihood in real world conditions it probably varies a lot (2x, 3x ?) around that as I previously outlined.
And it is not really…
Read moreDaryl Adair
Associate Professor of Sport Management at University of Technology, Sydney
I like the idea of touch-sensitive lines, though I'm not a scientist so unable to comment whether this would work. However, your kissing of the line objection should not be directed solely at Hawk-Eye - that's the standard for the ball being called in by humans. I agree with you that players ought to have more input into adjudication measures, but a sport is not actually managed by players (they're too busy training and playing in order to do that). So a combination of inputs is most feasible. Hawk-eye is not as accurate as you and others might have expected (frankly, it didn't surprise me), but in the absence of alternatives I think it is the best system to use for player challenges (unless we want to rely on TV replays from multiple angles). One alternative, of course, is to abandon Hawk-Eye and thus remove the players' recourse to challenges. I'd love to survey the players on that.