Featured Papers
Featured Paper
A Reinforcement Learning Architecture That Transfers Knowledge Between Skills When Solving Multiple Tasks
Authors: Paolo Tommasino, Daniele Caligiore, Marco Mirolli, Gianluca Baldassarre
Publication: IEEE Transactions on Cognitive and Developmental Systems (TCDS)
Issue: Volume 11, Issue 2 – June 2019
Pages: 292-317
Abstract: When humans learn several skills to solve multiple tasks, they exhibit an extraordinary capacity to transfer knowledge between them. We present here the last enhanced version of a bio-inspired reinforcement-learning (RL) modular architecture able to perform skill-to-skill knowledge transfer and called transfer expert RL (TERL) model. TERL architecture is based on a RL actor-critic model where both actor and critic have a hierarchical structure, inspired by the mixture-of-experts model, formed by a gating network that selects experts specializing in learning the policies or value functions of different tasks. A key feature of TERL is the capacity of its gating networks to accumulate, in parallel, evidence on the capacity of experts to solve the new tasks so as to increase the responsibility for action of the best ones. A second key feature is the use of two different responsibility signals for the experts' functioning and learning: this allows the training of multiple experts for each task so that some of them can be later recruited to solve new tasks and avoid catastrophic interference. The utility of TERL mechanisms is shown with tests involving two simulated dynamic robot arms engaged in solving reaching tasks, in particular a planar 2-DoF arm, and a 3-D 4-DoF arm.
Index Terms: Autonomous robotics, bio-inspired modular neural architecture, catastrophic interference, cumulative learning, functioning and learning responsibility signals, mixture-of-expert networks, reaching tasks, transfer reinforcement learning (TRL)
IEEE Xplore Link: https://ieeexplore.ieee.org/document/7592409
Special Issues
IEEE Transactions on Cognitive and Developmental Systems
Current Special Issues:
- IEEE TCDS Special Issue on Neuro-Inspired Learning for Robotics. Guest Editors: Qinyuan Ren, Nelson Sepulveda Alancastro, Wenyu Liang, Hareesh Godaba, and Fei Cao. Submission Deadline: May 1, 2024. [Call for papers]
- IEEE TCDS Special Issue on Embodied AI in Indoor Robotics: Bridging Perception, Interaction, and Autonomy. Guest Editors: Yaran Chen, Chenguang Yang, Chaomin Luo, and Dongbin Zhao. Submission Deadline: May 31, 2024. [Call for papers]
- IEEE TCDS Special Issue on Bridging the Gap between Machine and Brain in Speech Processing. Guest Editors: Jibin Wu, Haizhou Li, Shih-Chii Liu, Jonathan Timcheck, and Sumit Bam Shrestha. Submission Deadline: September 30, 2024 [Call for papers]
Instructions for Special Issues
IEEE Transactions on Cognitive and Developmental Systems publishes special issues on emerging topics guest edited by distinguished researchers in all areas related to cognitive and developmental systems, emphasizing on emerging and interdisciplinary topics. Below are instructions about how a special issue is organized.
Phase 1 – Proposal: A special issue proposal on an emerging topic in cognitive and developmental systems, or a closely related area should be sent to the Editor-in-Chief (EiC). The proposal should include at least the following components: the theme of the special issue, its relevance/importance and need in the present context, a list of specific topics, a list of potential authors, and a suggested time line. Note that, the list of potential authors is needed just to get an idea that there are sufficient researchers to support the proposed special issue. The special issue would be organized based on an open call for papers and we do not consider special issues based on papers from a single conference.
Phase 2 – Evaluation: Once the EiC receives a proposal, it will be circulated to all Associate Editors for comments and feedback. They take into account various issues including the technical merit, relevance to the scope of the journal, timeliness, and feasibility of the proposed special issue. Based on the input from the associate editors, the EiC makes a decision on the special issue proposal. The decision can be acceptance, rejection or a revision of the proposal. A revised proposal will again be reviewed by the associate editors and there is no guarantee that a revised proposal will be accepted.
Phase 3 – Call for Papers: Once a proposal is accepted, the Guest Editor(s) will be asked to prepare a call for papers (CFP) formatted to one transactions page so that it can be published in the Transactions. The CFP should include all relevant information such as the theme, topics, deadlines, and submission guidelines. The CFP must include an instruction to authors suggesting them to mention something like "This paper is for the special issue on XXXX" as a note to the Editor at the time of submission through the manuscript central. The CFP will be announced in the journal’s web site, circulated through CIS Newsletter and published in the CIS Transactions and Magazine.
Phase 4 – Review Process: Papers submitted to the special issue will be assigned to one Guest Editor for handling the review process. All papers submitted to a special issue papers will undergo exactly the same procedure as that of regular submissions. Papers submitted to the special issue co-authored by a Guest Editor will be handled a different Associate Editor assigned by the EiC.
Phase 5 – Publication: Once the special issue review process is complete, the EiC requests the Guest Editor to write a preface to the special issue (usually not more than 2 formatted transactions pages) for inclusion in the special issue. The special issue is then published as soon as possible.
We do not encourage guest editors to submit papers to the special issue and under no circumstances the Guest Editor should submit more than one manuscript to the special issue. Please contact the EiC for further information.