Phase II Plan

I modified the plan from the previous post.

• In the previous post, I thought I'd use the depth map for object recognition, but I'd use regular optical maps in this plan, as the depth map and the optical map are different modalities and it is more difficult to imagine what is going on with the depth perception from the human (phenomenological) point of view.
• In this plan, I dropped machine learning and will use more hard-wired approaches as I'd avoid unpredictable aspects.

---

Again recapitulating the Phase II part of my research plan here:

Phase II: Recognizing Spelke's Objects

• Basic Ideas
• Spelke's Object: coherent, solid and inert bundle of features of a certain dimension that continues over time.
  Features: colors, shapes (jagginess), texture, visual depth, etc.
• While recognition of Spelke's objects may be preprogrammed, recognized objects become objects of categorization by means of non-supervised learning.  In this process, hierarchical (deep) learning would be done from the categorization of primitive features to the re-categorization of categorized patterns.
• Object recognition will be carried out within spontaneous actions of the robot.
• The robot shall gather information preferentially on 'novel' objects (curiosity-driven behavior) ('novelty' to be defined).

The following is the new plan for Phase II.

Robot Basics

• Fish like robot swimming in a 3D space
  Experiments will be done with the SigVerse robot simulator.

Environment

• Keep the robot from wandering away by making it attracted by objects on the floor (see below).
• There are passively movable objects on the ground.

Robot Vision

• Line of Sight and 2D depth sensors (SigVerse)
• Static images
• Optical flow (temporal ⊿)
• Line of sight depth sensor (to avoid collision)

Central visual field (CVF)

• CVF (gaze) moves randomly (saccades) in the fixed visual field.
• CVF is attracted to information-dense areas.
• Information density is measured by the density of extracted features such as line segments and optical flow.
• CVF is 'bored' with each attractor as time passes.
• CVS allows high-resolution feature extraction.

Peripheral visual field

• Information density is measured with low-resolution feature extraction.

Feature extraction

• Line segments (using such as SIFT or Gabor filter) 
• Border ownership/Figure-Ground separation (see below)

Basic activities (hard-wired)

• Randomly change direction
• Direction change is attracted by line of sight
• If reward increases by direction change, then accelerate till next direction change.
• If locomotion decreases reward, then decelerate (with water resistance) (and change direction).
• If depth sensor predicts collision, then decelerate (with water resistance) and change direction.

Rewards

• Increase in information density in CVF gives a positive reward (curiosity; aesthetics)
• Concussion by collision will give a negative reward.

Spelke's object detection

• Uses figure-ground separation algorithms inspired by visual information processing in the brain.  Ref. 
  • Border-ownership coding
  • 視覚皮質の計算論的モデル（mostly English slides on Computational Models of Visual Cortex, Sakai 2014）
• Spelke's objects are recognized as figure-like lumps detected by figure-ground separation algorithms.
• Optical flow may also be used for figure-ground separation.

Phenomenology of Artefacts

In this memorandum, I propose a new scientific endeavor that I call “phenomenology of artefacts.”  The gist of the endeavor is that (human) epistemology would be emulated by constructing artefacts with epistemic functions similar to human beings.  While it is of an epistemology, it is inspired by (Husserlian) phenomenology, as the enterprise of Husserl was to construct a rigorous science (strenge Wissenschaft) where objective knowledge is established.  The basic idea of phenomenology of artefacts is the parallel that while Husserl started with introspection of his own mind to obtain knowledge, the 'mental state' of an artificial intelligence can be inspected (by external observers).  an artificial intelligence (AI) can inspect its own ‘mental states.’ (modified 2015-07-12, see the footnote* below)  Thus, I believe, AI researchers can obtain inspirations from ideas of the Husserlian phenomenology.  I’ll give a few points on this regard in the following.

Transcendence

Husserl regarded his phenomenology as transcendental.  Since one starts from phenomenal (internal) world, s/he would never reach or be sure about external objects.  Yet Husserl’s attempt was to transcend this barrier and to obtain objective knowledge of the external world.  The situation can be similar for an AI.  Though its perceptual states could be causally explained as sensory information processing, it is not clear at all how an AI could construct knowledge of the world from perceptual imagery.  This is especially true when it is not provided with prior knowledge of the world and should learn from scratch.  For example, how can an AI tell that there are 3D physical objects out there from transient perceptual patterns given as part of its internal states?  When it establishes the model of the 3D physical world and becomes ‘sure’ about external objects, then it transcends from internal perceptual imagery.

Perception and kinesthesis

Husserl emphasized the role of perception (especially the visual one) in obtaining objective knowledge.  He also emphasized the relation of perception to our motion.  When we perceive an object while moving, the perceptual images change over time in a certain way.  The way of change can be learned and becomes predictable.  To put it very simply, we learn about the external 3D world as we move and perceive.  Husserl used the term kinesthesis, which in a narrow sense means bodily sensation as we move, but may refer to any motion-related sensation (including visual perception).

Time-consciousness

“According to Husserl, the most fundamental consciousness, presupposed in all other forms and structures of consciousness, is the consciousness of time.” [Bernet et al.] (p.101)   If the knowledge of the world is obtained through kinesthetic interaction with the world, it is apparently true also for phenomenology of artefacts, but this aspect has not been fully explored by AI researchers.

  Husserl distinguished three moments of time-consciousness, namely, 1) premodal sensation corresponding to the now-moment, 2) retention or ‘a comet tail of memory’ and 3) protention or expectation.   Putting philosophical consideration asides, these moments may be understood from a neural perspective.  The cortex (brain) can be regarded as a ‘recurrent neural network’ learning temporal patterns.  At any moment, it retains information of its state of the immediate past (retention) and expects its state of the immediate future (protention) as it learns temporal patterns.

Concluding remarks

I often encounter useful insights when I read books on Husserl and there may be more inspirations from Husserl for AI researchers.  However, even if there is no more inspiration there, I believe phenomenology of artefacts could stand by itself and it could even replace Husserl’s attempt.  Human introspection is notoriously vague/opaque and does not seem to be suited for rigorous pursuit  of science.  Meanwhile, artefacts can be designed definitely and observed without opaqueness.  Thus, I advocate phenomenology of artefacts as a way to rigorous science Husserl envisaged…

Reference

[Bernet et al.] An Introduction to Husserlian Phenomenology, Rudolph Bernet, Iso Kern & Eduard Marbach, Northwestern University Press (1993).
[Gallagher et al.] The Phenomenological Mind (2nd edition), Shaun Gallagher & Dan Zahavi, Routledge (2012).
[J.J.Gibson] The Ecological Approach To Visual Perception, James J. Gibson,, Routledge (1986).

Footnote

* Modified after the comment by Shogo Tanaka that inspection by a machine would lead to the idea of 'consciously' reflecting on the internal representation of the world.  Phenomenology avoids such an idea and stress pre-reflective consciousness.  I fully agree with this comment.  Machine learning-based emergent AI does not normally make symbolic representation of what they are perceiving or doing.

The PDF version of this content (version 2014-02-02) is available here.

Phase I ⇒ Phase II

I spent too much time on attitude control.  It's time to move on.
As for attitude control, I have done:

• 1 dimensional rotation control by simple learning algorithms
  (experimenting in Java)
• porting the algorithms to c++
  (SigVerse controllers must be written in c++.)
• Implementing simple attitude control mechanism (3D but without learning) with SigVerse.

OK, attitude control doesn't require learning, and my learning/constructing c++ environment required a lot of time...

---

Last September, I wrote on my research plan:

Phase II: Recognizing Spelke's Objects

• Basic Ideas
• Spelke's Object: coherent, solid and inert bundle of features of a certain dimension that continues over time.
  Features: colors, shapes (jagginess), texture, visual depth, etc.
• While recognition of Spelke's objects may be preprogrammed, recognized objects become objects of categorization by means of non-supervised learning.  In this process, hierarchical (deep) learning would be done from the categorization of primitive features to the re-categorization of categorized patterns.
• Object recognition will be carried out within spontaneous actions of the robot.
• The robot shall gather information preferentially on 'novel' objects (curiosity-driven behavior) ('novelty' to be defined).

The following is a bit more concrete specification for Phase II.
Experiments will be done with the SigVerse robot simulator.

Robot Basics

• Fish like robot swimming in a 3D space

Environment

• The 'aquarium' is a cube/cuboid enclosed by walls.
  The robot cannot see transparent walls.
  The observer cannot see inside opaque walls.
  ⇒
  Keep the robot from wandering away by making it attracted by objects on the floor.
• There are passively movable objects on the ground.

Robot Vision

• Line of Sight and 2D distance sensors (SigVerse)

Basic activities

• If reward is under a threshold, then change direction.
• Change direction for reward increase.
• Accelerate towards the direction of maximal reward.
• Accelerate for reward increase. Accelerate inversely for reward decrease.

Rewards

• Complexity in the (variance of) 2D distance (depth) patterns will give a positive reward (motivation of reaching for objects; curiosity; aesthetics)
• Concussion by collision will give a negative reward.
• Getting bait will give a positive reward. 

Learning behaviors

• Learning by rewards (reinforcement learning)
• For example, the robot may learn bumping objects away to get bait.

Learning sensory patterns

• Sensory input
  • 2D distance depth (including optical flow)
  • Acceleration and rotation (kinesthetic input)
  • Concussion (large acceleration)
• Clustering sensory input
• Of course, feature selection (manual or automatic) is the key for successful learning.

Spelke's Objects

Check if the sensory pattern learning above yields the recognition of Spelke's objects.
If it does not, then add built-in mechanisms.

Learning attitude control

In the previous post, a supposedly swimming robot was set up in the Sigverse simulator, where I wrote;

The motion is out of control now.
So the current issue is attitude control perhaps by reinforcement learning.

It took a while for experimenting attitude control learning.  Though I've got ideas from actor-critic learning, I used simpler algorithms as the task is simple. Here's the idea (＆ result).

• Body rotation ＆ the stabilization goal
  The body to be  stabilized rotates and the goal of stabilization is to align the bottom center to the vertical direction.  The action of the body is acceleration to either direction.
• States
  The learner learns policies mapping from states to actions.
  The states are represented with an analog vector whose dimensions represent the rotation angle of the body.  In fact, the angle of the body is mapped to the vector by gaussian filters corresponding to the dimensions (and the angles allocated to the dimensions).  For example, if there are 36 dimensions, each of them is allocated to a 10 degree notch.  The gaussian closest to the current rotation angle output the largest value.
• Linear architecture
  The learner learns the mean acceleration for the current angle.  The mean acceleration is modeled as a linear sum of the state vector so that the learner learns the sum weights.
      average(output) = Σ wi * vi
  The actual output (acceleration) is determined by a gaussian distribution with the mean acceleration given by the linear architecture and the variance, which is also learned.
• Weights are learned with one of the following algorithms:
• Average success
  The weight is set to the average output for the state.
• Estimation difference
  Move the weight toward the actual output in proportion to (actual output - estimated output), where the estimated output is the mean acceleration given by the linear architecture.
• Reward
  Action success is measured by the following rewards:
• When the bottom sways away from the vertical position, then speed reduction is the reward.
• When the bottom is approaching to the vertical position, then the approach is the reward.
Perhaps simpler rewards may work as well.
• Variance adjusting
  When the action is successful, then adjust the variance σ in the following way (to adjust σ according to the difference to the discrepancy between the actual output and the estimate):
  
  σ = σ + α × reward × (|output - estimate| - σ)
  where α is the learning rate.
• Result
  Either algorithm worked (somehow the body stabilizes very quickly).  Learning seems to converge faster with the average success.  I opt for the average success as it is an accumulative method (the other method has no accumulative memory of the past).  The estimation difference method also requires a mechanism to reduce its learning rate to fix the learning, which I didn't implement this time.
• Current plan
  This trial was not implemented as a Sigverse simulation but a stand-alone Java program.  A Sigverse implementation is due where rotation will be three dimensional. 

Phase I Progress Memo 1

In my previous post, I delineated my research plan (as of 2013-09).
In the overall plan for the language acquisition experiment, Phase I was described as :

Phase I: Robot & World Setting

• Basic Ideas
• Locomotion: Rover［with wheels］or Swimmer［like fish］
• Vision: chromatic, binocular & without saccade
• No manipulator
  Still the robot can interact with objects by physical contact.
  The robot may have a 'mouth-like' manipulator afterwards.
• The robot acts autonomously/spontaneously.Spontaneity may be motivated by the motion itself, attitude (posture) control, learning new motions, learning new objects, etc.  Learning by spontaneous acts can be said to be learning by 'playing' or curiosity.
• Designing and implementing on drives related to learning will not be done in Phase I.
• Simulator design & implementation
  Spontaneous actions should be designed elaboratively.

---

Below is the current status of progress for Phase I (as of 2013-10):

• Swimmer (Fish) has been chosen for the robot
  setting.  The robot will have vernier-thruster-like
  kinematic controllers (2nd photo) instead of tilt
  rotors in the original sketch (for the sake of
  simplicity):
  
  
  
  
• SigVerse has been chosen for the robot simulator, which can obtain visual data from the robot.
• Currently unable to give a fish shape to the robot in SigVerse, so that it has the shape of a humanoid head flying around (see the photo ↓).
  
  
  Kinda spooky but so be it for the moment.
• The motion is out of control now.
  So the current issue is attitude control perhaps by reinforcement learning.
• Spontaneous motion may be implemented after having made its attitude stable.

My New Research Plan (2013-09)

I changed the plan.
There are two major changes.

• I would use simulator software rather than a real robot.
• I would focus on language acquisition rather than using conventional NLP.

---

Purpose

Research on language acquisition (symbol grounding) with a robot simulator.

Language Acquisition

Research will follow the human (infant) language acquisition process.  More concretely speaking, the simulated robot shall learn labels on the properties, movements and relations of objects, based on the cognition of bundles of coherent and continuous properties (Spelke's objects). It shall utter with learned labels. Moreover, it shall create adequate internal semantic representations and relate them with corresponding syntactic structures.
(See below for more details.)

Core cognitive architecture

The core cognitive architecture shall have the following functions.
Upon designing and implementing the cognitive architecture, generic mechanisms shall be (re)used whenever possible.

• Time-series pattern learner/recognizer
  having motor commands and sensory input as their input
• Attention and situation assessment
  on what will be learned and which action will be taken.
  It is part of the executive control function.
• Cognitive model based on association
  to memorize and recollect (temporal) generative patterns as associative sequences.
  Linguistic competence will be realized with this model.
  It contains backtracking mechanism based on the function of attention and situation assessment mentioned above.
• Episodic memory
  Patterns (the representation of situations -- combinations of abstract patterns created by (non-supervised) learning) positively assessed by the attention and situation assessment will be memorized.

 cf. A Figure of Human Cognitive Function

Platform

• Robot simulator
  Sigverse, etc.
• Visual data processing
  OpenCV, etc.
• Speech recognition/synthesis: option
  (Research can be carried out on the text basis.)
• Learning module
  SOINN, k-means, SOM, SVN, DeSTIN, HMM, etc.
  (To be used as plug-ins depending on the purpose)

A Tentative Research Steps

Phase I: Robot & World Setting

• Basic Ideas
• Locomotion: Rover［with wheels］ or Swimmer［like fish］
• Vision: chromatic, binocular & without saccade
• No manipulator
  Still the robot can interact with objects by physical contact.
  The robot may have a 'mouth-like' manipulator afterwards.
• The robot acts autonomously/spontaneously.Spontaneity may be motivated by the motion itself, attitude (posture) control, learning new motions, learning new objects, etc.  Learning by spontaneous acts can be said to be learning by 'playing' or curiosity.
• Designing and implementing on drives related to learning will not be done in Phase I.
• Simulator design & implementation
  Spontaneous actions should be designed elaboratively.

an example fish robot

Phase II: Recognizing Spelke's Objects

• Basic Ideas
• Spelke's Object: coherent, solid & inert bundle of features of a certain dimension that continues over time.
  Features: colors, shapes (jagginess), texture, visual depth, etc.
• While recognition of Spelke's objects may be preprogrammed, recognized objects become objects of categorization by means of non-supervised learning.  In this process, hierarchical (deep) learning would be done from the categorization of primitive features to the re-categorization of categorized patterns.
• Object recognition will be carried out within spontaneous actions of the robot.
• The robot shall gather information preferentially on 'novel' objects (curiosity-driven behavior) ('novelty' to be defined).
• Determining the recognition method & implementation
• Recognition experiments
• Determining (non-supervised) learning methods  & implementation
• Experiments on object categorization
• Designing novelty-driven recognition & behavior & implementation

Phase III: Labeling

• Basic Ideas
• The robot shall relate specific types of Spelke's objects it looks at with linguistic labels.
• Labels may be nominals representing shapes and adjectives representing features such as colors.
  Types of objects may be learned in a supervised manner with labels or have been categorized by non-supervised learning.
• The robot shall utter labels on recognizing types after learning association between the labels and types.
• The robot shall recognize/utter the syntactic pattern 'adjective + noun'.
• Determining the recognition method & implementation
• Designing and implementing mechanism for handling syntactic structure.
• Labeling experiment

Phase IV: Relation Learning

• Basic Ideas
• The robot shall learn labels for
• object locomotion such as moving up/down, right/left and closer/away
• orientational relations between objects such as above/below, right/left and short/thither
• Objects should be get the robot's attention by force (programming) or by certain preprogrammed mechanism of attention (such as attention to moving objects). 
• Designing & implementing labeling mechanism
• Experiments

Phase V: Linguistic Interaction

• Basic Ideas
• The robot shall answer to questions using labels learned in Phase III & Phase IV.
• The robot shall respond to requests on its behavior.
• The robot shall utter clarification questions.
• Designing & implementing mechanism for linguistic interaction
• Experiments

Phase VI: Episodic memory

• Basic Ideas
• Episodes (situations) to be memorized are the appearance of objects and changes in relations among them.
• The memory of novel objects and situations is prioritized.
• Designing & implementing episodic memory and attentional mechanism
• Designing & implementing episodic recollection & linguistic report.
• Experiments

Phase VII: More complicated syntax

• Basic Idea
  The robot shall understand/utter linguistic expressions having nested phrase structure.
• Designing & implementing nested phrase structure.
• Experiments

My New Research Plan (2013-08)

Purpose

To create a rover that explores the environment, learns from it and communicates with human-beings in a human language.
The cognitive model of the rover shall be based on association. The real purpose of the research is to verify the feasibility of cognitive models based on association.

The cognitive model based on association

In the model, generative patterns are represented as series of association.
Patterns can be visual, auditory, tactile, and linguistic.

Recognition of situations

The pattern representing a situation could be obtained by integrating pattern recognition results from various modalities within a time series.
An associative series of the situation is a "semantic network", which gives the semantics for the linguistic function.

Learning

The learning mechanism will be neural networks (in a broad sense).
The learning includes the recognition of exterior things, the episodic memory and the categorization of situations.  Categorization may be done by supervised and/or unsupervised  learning.
The rover will learn from its voluntary actions.

Language

The interface between syntax (parsing/generation) and semantics (association of situations) shall be learned (acquired), by means of association-based cognitive models.
While morpheme dictionaries may be given initially, vocabulary acquisition will be an issue in the future.
The reason for the linguistic interface is that language will be the key for realizing the human-level intelligence, besides it is an effective means for communicating with human-beings.
The rover will have basic language features such as the following .

• Description of the scene (verbalization of situation recognition)
• Response to (human) questions
• Response to (human) instructions

Sociality

Human (animal) infants tend to have the innate ability to recognize other individuals and communicate with them. The rover will be given certain recognitive abilities such as face recognition, motion capture, speech recognition and gaze recognition.

Core cognitive architecture

• Time-series pattern recognizers having motor commands and sensory input as their input
• Attention and situation assessment
• Episodic memory (← pattern recognizers, situation assessment & attention)
• Backtracking (parsing and planning with certain evaluation functions)

 cf. A Figure of Human Cognitive Function

System configuration

Hardware

• Locomotive function
  Roomba / Kobuki, etc.
• Visual function
  Early stage: Kinect
  Later adding saccade
• Acceleration sensor
  (works also as collision detector)
• Audio function
• On board PC (notebook)
• Wireless connection (WiFi / BlueTooth) for monitoring

Software

• OS
  ROS, etc.
• Visual data processing
  Peripheral vision, central visual field, optical flow, depth recognition
  Object detection, tracking, motion capture, facial recognition, gaze recognition
  Kinect software, OpenCV, etc.
• Speech recognition
  HARK, etc.
• Speech synthesis
  Vocaloid, etc.
• Learning module
  SOINN, k-means, SOM, SVN, DeSTIN, HMM, etc.
  ※To be used as plug-ins depending on the purpose.

A Tentative Research Steps

Phase I: Kinect + OpenCV + HARK + Vocaloid (preliminary exercises)

• Checking for visual functions (facial recognition, motion capture)
• Checking for speech recognition function
• Checking for speech synthesis function
• Implementating a conventional linguistic (speech) interface
• Experimenting on visual experience reports with a conventional linguistic interface

Phase II: Pattern Recognition

• Selection and implementation of time-series pattern recognizers
• Visual pattern recognition experiment
• Experiments on pattern recognition reports with a conventional linguistic interface

Phase III: Episodic memory

• Defining the situations to be remembered
• Implementing episodic memory and attentional mechanism
• Experimenting on episode reports with a conventional linguistic interface

Phase IV: Eye movement

• Kinect may be put on a movable (controllable) stage (w. an acceleration sensor)
• Human tracking
• Behavior control (extending a conventional language generation mechanism) 
• Gaze induction by instruction with a conventional linguistic interface
• Q & A with a conventional linguistic interface

Phase V: Roaming (Roomba / Kobuki)

• Coupling vision and roaming (reflexive) 
• Defining the relation between attention and roaming ("curiosity")
• 3D object learning/recognition via roaming
• Instruction of motion with a conventional linguistic interface

Phase VI: Design and Implementation of a non-conventional (associative) linguistic information processing