Toward Simple Language (Verb) Learning

I kind-of finished with the simulation environment (the previous post).
The LU (caretaker) now 'instructs' the LL (language learner) with commands such as 'stay,' 'turn,' 'come,' and 'go to the blue item' (in Interlingua).

"Luca, turn right."

When LL 'follows' the instruction, LU smiles and says "Good" to give a reward.

"Good!"

LU also 'declares' (thinks aloud) its own actions before carrying them out.

"I go to the blue one."

---

Now I guess I should go for language learning.
Here is a tentative plan:
(For the time being, I'll forget about LU's declaration mode, which requires the recognition of LU's behavior on LL's part.)

1. The association of LU's words with LL's own actions
   As this should be done at a conceptual level, it requires:
• Generation of action concepts
• Generation of word concepts
LL may carry out three kinds of learning:
• Reinforcement learning of action under instruction
• Association between word concepts and action concepts
• Creating a language (syntactic) model linked with the concepts
2. Bubbling
   LL will utter based on the language model.
   • If the utterance matches its action, LU repeats it and gives a reward.
     This reinforces the bubbling.
   • If not, LU describes LL's action after saying "No."
     This alters the conceptual association and language model.

New architecture

I altered the agent architecture.  The previous one used competing production-like rules for control.  The current one uses simpler perceive-reward-action cycles.  In case conflicts arise, they will be resolved by the action selector.  This would be more compatible with regular reinforcement learning architecture.  Besides the abolishment of competing rules, the rest is the same as the old design.
BTW, two agents are now split to the Language Learner and the Language User (caregiver).  LL chooses its action randomly and LU calls LL when LL is not looking back to LU.  LU's call is displayed as a small rectangular balloon on it, containing LL's current name "Luca."  Agents also 'smile' when they are 'looking into each other.'  These are part of the plan in my verb learning experiments.

The simulator was made with V-Rep and python.
The code is found on GitHub (Version 2 snapshot: 2016-08-01).

Carrying items

I made the agents to carry items to a given direction by pushing them.
This is a trick to be carried out in verb learning experiments.
I thought I'd use reinforcement learning to make them do the trick and tried to use PyBrain, but failed to use it without learning the cause.  I got back to the old-fashioned programming and fixed the trick while keenly feeling my lack of sense in geometry.  The agent first goes to the opposite side around the item against the carrying direction, start pushing, and adjusts the direction when the item veers.


The simulator was made with V-Rep and python.
The code is found on GitHub (Version 1 snapshot: 2016-06-13).

Thought on Architecture

The recent posts showed the agents under development in action.
Perhaps, it is the time to stop and think before continuing the development...

For the moment, the agents have very simple architecture, with a blackboard kind of internal communication system.  Namely, an agent has an input buffer and a state buffer, where 'codelets' can read from and write to, respectively.  As both buffers are python dictionaries (hash tables), they can contain any data, which are accessed by 'names/keys.'

Codelets are registered as 'rules,' which have a condition part and an action part.  In each execution cycle, the action of the codelets/rules whose conditions meet 'fires' and it may try to modify the content of the state buffer.  Each codelet has a score and if more than one codelet tries to modify a same part of the state buffer (with the same name/key), the value written by the codelet with the highest score (the winner) is chosen.

In the recent posts, I mentioned affect terms such as 'reward' and 'urge.'  In fact, they correspond to variables with the same names on the state buffer, and the agents 'smile' when they read a positive 'reward' in the buffer.

While the mechanism is quite simple, generic and usable, there are a few points to consider for future modifications:

• Is it appropriate as a cognitive architecture?
  To make it look more like a respectable cognitive architecture, perhaps, the affect mechanism should have its own modules.
• Is it biologically plausible?
  (The answer is, of course, No.)  To make it biologically plausible, the architecture should mimic the brain architecture.  Or, more simply, real brains are not supposed to use blackboard architecture.
• Symbolic representation?
  The current system uses symbolic representation (python dictionaries) in internal communication.  Besides that the brain does not use such symbolic representation, using vector representation would be good when the system is to be controlled machine learning algorithms.

Spontaneous walk

After 'looking into each other' a while, the agents get bored (develop the urge to move) and start moving.  (The previous post showed the footage in which they follow each other, stop and smile when they meet.)

 
The simulator was made with V-Rep and python.
The code is found on GitHub.

Adding Emotional Expressions to Agents

Facial expressions were added to the agents in my simulation environment. They follow each other and stop and smile when they meet (kind of cute :-D).  (The smile is driven by 'reward' given when the agent meets another.)

 
The simulator was made with V-Rep and python.
The code is found on GitHub.

AGI in Japan, 2015

More than a year passed since I wrote the article 'AGI in Japan, 2014' and some of AGI-related activities in Japan, then grass-roots, have been institutionalized. The following is an update mostly in the chronological order.

November 2014: Dwango Co. LTD., a media company, launched its AI laboratory. Its current function is 'promotive' by being a communicative hub for AGI researchers.

April 2015: Chiba Institute of Technology inaugurated STAIR (Software Technology and Artificial Intelligence Research) Laboratory (HP in Japanese), headed by Akinori Yonezawa.

May 2015: The National Institute of Advanced Industrial Science and Technology (AIST) inaugurated its Artificial Intelligence Research Center (AIRC), headed by Jun'ichi Tsujii.

The annual convention of the Japanese Society for Artificial Intelligence (JSAI) hosted two-part sessions on AGI. Part I mainly discussed models of the hippocampus and Part II discussed mainly the social and ethical aspects of AI. The entire time table (in Japanese) can be accessed from here, where you may find other topics related to AGI.

July 2015: The New Energy and Industrial Technology Development Organization of Japan (NEDO) announced a plan to create a center of excellence in AI (news release in Japanese). NEDO has a large funding base and at least one of AIRC's intelligent robot projects is funded by NEDO. (Ref. an English article from NEDO on robots)

August 2015: The Whole Brain Architecture Initiative (WBAI), an NPO explicitly aiming for the realization of AGI, was inaugurated. Its main objective is to foster a research community based on the approach where researchers build cognitive architecture with machine learning modules while mimicking the brain. It held the first hackathon in September and a workshop at BICA 2015 (Lyon) in November. The winners of the hackathon (students) were present at the BICA conference.

September 2015:

A project on the Symbol Emergence in Robotics was accepted by CREST, a funding program of the Japan Science and Technology Agency (JST) under the Ministry of Education, Culture, Sports, Science and Technology (MEXT) (ref. Job Info of the project in English). The project will hold a kickoff event in November. The community has been quite active aiming for human-level AI in robotics. Tadahiro Taniguchi, one of the major proponents of the approach, is currently visiting the Imperial College.

On September 29, WIRED Japan held its first Singularity Summit (Japanese site). Lav Varshney and Ben Goertzel were among the speakers. Another speaker, Takuya Matsuda, the author of ‘The Year 2045 Problem (2045年問題)’, is regularly hosting the Singularity Salon, by the way.

The Special Interest Group for AGI (Japanese site) was officially recognized by the Japanese Society for Artificial Intelligence (JSAI). The SIG had been an unofficial group, mainly surveying articles on AGI. The SIG held a mini-inaugurating-symposium on September 30th, where Ben Goertzel gave a talk. After the symposium, there were discussions about potential collaborations towards open AGI research among groups (such as OpenCog and WBAI) world-wide. The SIG will hold the first workshop in December.

---

While they may not be directly related to AGI, companies in Japan are now coming into the AI field as AI has getting more popularity. For example, Recruit Holdings and Toyota are having their AI research sections, inviting researchers from abroad. In the venture domain, companies like Preferred Infrastructure, which released Chainer (the neural network framework), and Nextremer are engaged in researches in AI and machine learning.