We may not know it, but psychometrics has shaped the way we understand intelligence.
We’re all familiar with the idea of an IQ test, and we might know where we stand on the IQ scale - but what about the rest of the animal world? And how smart are machines becoming? At present, it’s hard to tell.
But understanding how the rest of the world (both biological and electronic) operates can help us understand ourselves.
Intelligence is seen as a property that can be measured in all creatures, one that correlates with their performance in a range of tasks and environments.
What we lack is a proper answer as to exactly what intelligence tests measure. One explanation can be given by the fact that intelligence tests focus on human intelligence.
In fact, intelligence scores are not absolute measurements but a normalised scale relative only to the human population.
So can we devise intelligence tests without taking humans as a reference?
Comparative psychology in the animal kingdom
Comparative psychology and cognition have extended the concept of intelligence tests (and other cognitive tests) to animals. This provides significant sources of information about the nature of intelligence.
But this type of research is “comparative” by definition, so it does not solve the general problem: how can we measure intelligence for a wide range of subjects, from human children to adult dogs, without using our own species as a baseline?
Comparative psychology considers Homo sapiens in the same way Darwin taught us to - as just as another species.
The problem is that typical cognitive tests (and psychometrics tests) have been so anthropomorphic in the past century that it’s difficult to remove all the human (and linguistic) bias, while measuring intelligence both practically and accurately.
We’ve had to develop tests that involved no language whatsoever, not even to give the instructions. This has been addressed very well in comparative psychology using rewards and penalties.
‘The machine kingdom’
If things were not already complex enough for the animal kingdom, there is a diverse new realm that is still unexplored: the machine kingdom.
This uncharted space is much more complex than the animal kingdom in at least one respect: there is no constraint about how a machine can be designed.
Theoretically, we can define a machine to behave in virtually any possible way, including emulating any living or extinct animal.
In practice, we may know how to emulate the behaviour of very simple animals, for example a sponge, but we are still far from capturing the behaviour of a mammal, needless to say a human.
The good news about machines is that we can attempt any possible combination of instructions - the only constraints are computability and resources.
The bad news is that this behaviour will be impossible - or at best intractable - to determine theoretically, even if we know each line of code of the program.
Clearly, in order to assess the behaviour of a plethora of machines, bots, robots, artificial agents, avatars, artificial life beasts, we will require a powerful set of cognitive tests.
Intelligence (tests) and information theory
There has been more than a decade of research including new intelligence definitions and tests using the notions of compression and minimum message length, and applying traditional IQ tests to machines.
It’s time now for a new field of research dealing with universal intelligence tests developed from information theory and computer science.
Considering machines, humans and animals at the same time and in the same way gives a broader view of what intelligence is and how it should be measured. This way, we can avoid the risk of anthropocentrism.
But developing universal intelligence tests will not be easy.
We propose a popular adaptive anytime rewards-based test consisting of rewards (or penalties) in a sequence of environments, depending upon the agent’s scores in past environments.
We initially considered single agents - people, animals or machines - endeavouring to locate and follow rewards that move between locations (or cells).
But we have seen that it is possible to see algorithms scoring better than humans on some instances of this test. This just shows that the tests are falsifiable, which leads the way to designing more general and robust tests, while always having a mathematical setting for constructing them.
For instance, we are now considering agents in environments with other agents, including situations where the agents have co-evolved.
All this is part of our project anYnt, Anytime Universal Intelligence, which explores the possibility of measuring the intelligence of any system, of any intelligence degree, of any kind, of any speed, with tests that can be interrupted at any time.
The time for a new discipline
All this is in its early stages, and we are still far from a universal, practical test. But studying this question is scientifically imperative.
And from here we can get back to where we started. If an intelligence test is able to accurately measure the intelligence of any machine then, as a result of the Church-Turing thesis, it will be able to measure the intelligence of any animal, including humans.
Whether effective tests of this kind can be constructed and, if so, how to build them would be the goal of a new discipline: universal psychometrics.
Universal psychometrics will consider the Homo sapiens as Turing taught us, just another machine.
Soon, our place and level of intelligence in the machine kingdom will finally be more reliably estimated - if not properly put in place.