The Match-to-Sample Task as the First Milestone toward the Realization of Fluid Intelligence

[日本語版]

Introduction

In the first half of this article, I discuss concepts around the term ”fluid intelligence,” which often appears in general intelligence literature. In the second half, I discuss tasks for testing working memory, which is considered to be an essential cognitive function for the realization of fluid intelligence, and propose to use a match-to-sample task to start with.

General Intelligence and Fluid Intelligence

The term "artificial general intelligence (AGI)" refers to the attempt to create general intelligence or the product of such an attempt.  AGI is general (-purpose) in that it is capable of solving problems not anticipated at the time of design, as opposed to "narrow AI" designed for specific purposes.

The term "general intelligence" is originally from psychology and is based on the hypothesis that human intelligence can be attributed to a unique general intelligence factor g. [Spearman 1904].  Various tests (IQ tests in psychometrics) have been devised based on this hypothesis.

Despite the single factor hypothesis, multiple factors have been postulated.  [Cattell 1943] postulated fluid and crystalized intelligences in general intelligence.  In Cattell's original definition, fluid intelligence is the ability to discriminate and perceive relations, and crystalized intelligence is discriminatory habits originally through the operation of fluid ability [Brown 2016].  However, the definition of fluid intelligence has been varied.  According to [Kievit 2016], fluid intelligence is the ability to solve novel, abstract problems that do not depend on task-specific knowledge.  Crystalized intelligence, on the other hand, is considered to be the ability to apply learned knowledge.

The fluid and crystalized aspects can be discerned also in AI.  Learning has the fluid aspect and the application of learned knowledge has the crystalized aspect. [Hernández-Orallo 2017](Chapter 12) addresses fluid intelligence as a policy-generating capability.  In reinforcement learning terms, exploration has the fluid aspect and exploitation the crystalized aspect.  What [Chollet 2019] calls the Intelligent System, which learns, has the fluid aspect, while the Skill Program the crystalized aspect.  Here I will refer to the ability to discover and learn relations and policies as fluid intelligence in a broad sense.

AGI is required to have the fluidity to cope with a new task without learned knowledge of the task, in that it is required to have generality to solve tasks not expected at the time of design.  Fluid intelligence in a broad sense does not specify whether the ability to solve new problems is the result of trials and errors in the environment or internal simulations.

Note that the dichotomy of fluid intelligence (in a broad sense) and crystalized intelligence is not consistent with the dual process theory's dichotomy of System 1 and System 2 (roughly corresponding unconscious and conscious processes)(ref. [Kahneman 2003]).  Reinforcement learning (operant learning), for example, is classified as fluid intelligence in a broad sense, which deals with tasks without acquired knowledge of the tasks, even if it is not performed by conscious processes (System 2).   Also, although humans often perform deductive reasoning in a conscious process (System 2), the ability to deduce with acquired knowledge belongs to crystalized intelligence.  However, regular IQ tests do not assume an unconscious form of intelligence that involves unconscious trials, and System 2 should be involved in the fluid intelligence measured by IQ tests.

Fluid Intelligence and Search

As fluid intelligence measured by IQ tests is not the kind of intelligence that allows for trial and error in the environment, the search for solutions to problems (trial and error) is considered to be internal to the agent.  In the following, the discussion will be limited to fluid intelligence measured by IQ tests, and thus to (narrowly defined) fluid intelligence that solves problems by internal simulation. In an IQ test, the subject is required to find some solution (or some relations if we follow Cattell's definition), but s/he is not required to learn the found solution as a policy.

Search involves action selection and generates a sequence of actions.  The result of search may be arriving at a goal, or it may be a pattern generated by a sequence of actions.  Whether the result of the search satisfies the objective is evaluated by some criteria.  In a case where the result is goal attainment, whether the state is associated with a reward may be evaluated.  Or whether the state or the pattern generated meets certain criteria (or constraints) may be assessed by evaluative actions.

Example: Matrix Reasoning Tasks

Matrix reasoning tasks are often used to measure fluid intelligence in IQ tests (e.g., Fig.1).  ARC, the test for AGI proposed in [Chollet 2019] is also a kind of matrix reasoning task.  Here, the task is used as an example to see the nature of fluid intelligence.

Fig.1 Raven Matrix
CC BY-SA 3.0 Life of Riley @Wikimedia

In a matrix reasoning task, subjects are presented with a matrix, where a cell in the last row is blank.  Subjects discover a rule from the pattern shown in the other rows and apply the rule to the last row to fill in the blank cell.

Subjects are expected to solve the task by the following search.

• Generate a rule that generates (predicts) the content of the last cell of a row other than the last one.
• Check if the above rule is applicable to other rows.
• If not, generate another rule.
• If the rule can be applied to all the filled rows or columns, apply it to the last cell of the last row.

In a matrix inference task, the rule takes the form of operations on the elements of a matrix and the result.  Thus, to solve the task, subjects must have task-specific prior knowledge of matrix element operations and the ability to perform the operations internally.  If the matrix element is a figure, example operations include transformations such as rotation, reduction, and combination of figures.  If the matrix elements are numbers, knowledge of arithmetic operations and 'mental calculation' skill are required.

In matrix inference tasks, search is performed, where the generation of applicable rules is a sub-goal and applicability check is an evaluative action.  In the applicability check, the agreement between the internal representation generated by the rule and the matrix content is checked.

From the above, the cognitive functions required for the matrix reasoning task are as follows.

• The ability to operate internally the objects on the matrix
• The ability to generate and retain elements of internal operations (objects and results) as patterns.
• Pattern application and matching
• The ability to apply patterns to partial patterns and to predict/generate the overall pattern
• The ability to determine if the predicted/generated patterns match reality
• Search ability
• If a pattern fails, try another pattern generation.
• Memorize failed patterns and do not repeat them.

Summary of Terminology

Table 1 summarizes the terms related to fluid intelligence used so far. 

Table 1: Terms related to Fluid Intelligence 

General Intelligence (g)
Fluid Intelligence in a Broad Sense – Exploration – (Intelligent System)				Crystalized Intelligence – Exploitation – (Skill System)
Exploratory				Exploratory	Non- Exploratory
Internal Search Fluid Intelligence in a Narrow Sense Relationship Discovery		External Search		Navigation, Deduction, Parsing, etc.	Simple policy application
No Learning	with Learning	No Learning	with Learning
Fluid Intelligence Tests	Model-based RL	Solution discovery by external search	Model-free RL

Note 1) As it is a simplification, exceptions may be found.

Note 2) The classification is independent of the classification of dual process theory in that any process in the table may be carried out "consciously" and may involve "unconscious" processes (e.g., perception), and "unconscious" exploration may also be possible.

Note 3) In humans, exploratory processes in crystalized intelligence other than parsing are carried out largely "consciously" and may mobilize the same cognitive functions as requiring fluid intelligence measured in IQ tests.

Note 4) Good old fashioned artificial intelligence (GOFAI) has excelled at exploratory crystalized intelligence.

The Realization of Fluid Intelligence

At least for matrix inference tasks, the algorithm seems feasible.  However, exploration is not the forte of artificial neural networks (ANN) currently in mode.  State-of-the-art learning systems such as MuZero [Schrittwieser 2020] combine ANN and search algorithms such as Monte Carlo tree search to cope with search required for playing board games.  It is worth examining if an architecture combining ANN and a search mechanism can solve matrix reasoning tasks (including ARC).

We should be careful about whether the dichotomy between fluid and crystalized intelligences is decisive for AI.  A system that is given or learned knowledge to efficiently solve a matrix reasoning task can be viewed as solving it by crystalized intelligence.  Meanwhile, AI may be designed in a way that learned or prior knowledge are separated from a 'fluid' general task solution mechanism.

Fluid Intelligence and Working Memory

The cognitive functions required for matrix reasoning tasks used for testing fluid intelligence include retention (memory) of patterns to be applied and failed patterns. These are temporary memories that are necessary for task performance and are, by definition, "working memory."  Search generally requires working memory, as paths having failed or been successful should be remembered.

While it is easy to realize working memory in symbolic AI, there is no definitive method to implement it with ANN.  If AGI is to be realized with ANN in a way that mimics the brain, it is important to have an ANN model of working memory, which is a fundamental cognitive capability for fluid intelligence (or exploration for that matter).

Task Proposal

The examination of a cognitive function requires tasks to measure the functioning.  Proposed tests of working memory include the N-back task, in which the subject responds to whether the current stimulus is the same as the one presented N times before, the delayed response task, in which the subject answers the position of the stimulus (usually bait) presented some time beforehand, and the delayed match-to-sample task (to be described below). In the following, the delayed match-to-sample task will be discussed as it has a moderate difficulty as a working memory task.

Fig.2 A Delayed Match-to-Sample Task

In a delayed match-to-sample task, a sample figure is presented first, and after it is hidden, a set of target figures is presented (Fig. 2).  The subject selects a target figure that is regarded as the same as the sample according to a certain criterion (e.g., shape or color).  The task is a test of working memory because the subject has to remember the attributes of the sample to select the correct target figure.  If more than one criterion is used, the subject must remember the criterion as well.  The criterion may be given in natural language or in a sample session presented before the session.  When non-verbal sample sessions are used, the relationship between images in sample sessions, the criteria, and the task to be performed must be learned.  In the sense that they use learned knowledge, nonverbal delayed match-to-sample tasks do not purely measure fluid intelligence.

As a simpler task, non-delayed match-to-sample tasks can be thought of.  Even in a non-delayed match-to-sample task, if the subject cannot recognize more than one image at the same time due to its field of view, images looked at before and after the eye movements should be remembered, and it requires working memory.  In addition, if more than one comparison criterion is used, the subject must remember criteria presented during sample sessions.

Although it is not to test working memory itself, image manipulations such as image reduction and rotation can be added to the task to increase generality (invariant object identification).

As for implementation, there are psychology experiment toolkits such as PEBL or PsyToolkit for match-to-sample tasks. I have also implemented a task environment with PyGame to be a pure Python implementation that meets the above specification.

Conclusion

In this article, concepts around "fluid intelligence" were discussed, and then working memory tasks were proposed, as working memory is considered to be a basic cognitive function for fluid intelligence.

The authors are planning to hold an online hackathon (competition) to solicit participants to implement cognitive architecture for performing the match-to-sample tasks described in this paper.  A challenge is that there is no definitive way to model working memory with ANN.  As the biological mechanisms of mammalian working memory are not fully understood, the hackathon will also evaluate the biological plausibility of submitted models (we have solicited working memory models in a modelathon). The authors hope that readers will participate in the competition and engage in the implementation of working memory and fluid intelligence.