Research Plan Updated (2014-06)

I streamlined the plan: I dropped three dimensional concept learning and (therefore) the use of robotics simulator, to focus more on language acquisition.
While the robotics/embodiment is essential for phenomenology of artifacts, as it incurs too much technical complication, it is put off until relevant tools are readily available.  The new plan also drops the emphasis on the recognition of Spelke's objects, which would require action during perception.

---

Purpose

Creating a system that performs language acquisition (symbol grounding) with a robot simulator pattern recognizers and association networks, to verify associationist hypotheses of cognition.

Language acquisition

The system will follow the human (infant) language acquisition process: it shall associate linguistic expressions with the shapes, colors, movements and relations of perceived objects, while creating adequate internal syntactic/semantic representations.

Core cognitive architecture

The core cognitive architecture shall have the following functions.
Upon designing and implementing the cognitive architecture, generic mechanisms shall be (re)used whenever possible.

• Time-series pattern learner/recognizer
  having motor commands and sensory input as their input
• Attention and situation assessment
  on what will be learned and which action will be taken.
• Cognitive model based on association
  to memorize and recollect (temporal) generative patterns as associative sequences.
  Linguistic competence will be realized with this model.
  It contains backtracking mechanism based on the function of attention and situation assessment mentioned above.
• Episodic memory
  Patterns (the representation of situations -- combinations of abstract patterns created by (non-supervised) learning) positively assessed by the attention and situation assessment will be memorized.

 cf. A Figure of Human Cognitive Function

Platform

• Visual data processing
  OpenCV, SimpleCV, etc.
• Learning module
  BESOM, DeSTIN, k-means, SOM, SVN, HMM, SOINN, etc.
  (To be used as plug-ins depending on the purpose)
• Publicly available cognitive architectures may be surveyed
  e.g., OpenCog and LIDA

A Tentative Research Steps

Phase I: Survey on robot simulators (done)

Research on robot simulators such as SigVerse and V-rep and trial on attitude control.

Phase II: Survey on Spelke's Object recognition (done)

Proper recognition of Spelke's objects, having coherent, solid & inert bundle of features of a certain dimension that continues over time, would require optical flow processing associated with the action of the perceiver.

The following part doesn't have much change from the previous plan.

Phase III: Labeling

• Basic Ideas
• The system shall relate specific types of figures it looks at with linguistic labels.
• Figures get the system's attention by force (exterior instruction)
• Labels may be nominals representing shapes and adjectives representing features such as colors.
  Types of objects may be learned in a supervised manner with labels or have been categorized by non-supervised learning.
• The system shall utter labels on recognizing types after learning association between the labels and types.
• The system shall recognize/utter the syntactic pattern 'adjective + noun'.
• Determining the recognition method & implementation
• Designing and implementing mechanism for handling syntactic structure.
• Incorporating episodic memory at this stage is to be considered.
• Labeling experiment

Phase IV: Relation Learning

• Basic Ideas
• The system shall learn labels for
• object locomotion such as moving up/down, right/left and closer/away
• orientational relations between objects such as above/below, right/left and short/thither
• Objects should be get the system's attention by force (programming) or by certain preprogrammed mechanism of attention (such as attention to moving objects). 
• Designing & implementing the mechanism
• Experiments

Phase V: Linguistic Interaction

• Basic Ideas
• The system shall answer to questions using labels learned in Phase III & Phase IV.
• The system shall respond to requests on its behavior.
• The system shall utter clarification questions.
• Designing & implementing mechanism for linguistic interaction
• Experiments

Phase VI: Episodic memory

• Basic Ideas
• Episodes (situations) to be memorized are the appearance of objects and changes in relations among them.
• The memory of novel objects and situations is prioritized.
• Designing & implementing episodic memory and attentional mechanism
• Designing & implementing episodic recollection & linguistic report.
• Experiments

Phase VII: More complicated syntax

• Basic Idea
  The system shall understand/utter linguistic expressions having nested phrase structure.
• Designing & implementing nested phrase structure.
• Experiments