The IEEE Transactions on Neural Networks and Learning Systems publishes technical articles that deal with the theory, design, and applications of neural networks and related learning systems.
Journal Citation Metrics Journal Citation Metrics such as Impact Factor, Eigenfactor Score™ and Article Influence Score™ are available where applicable. Each year, Journal Citation Reports© (JCR) from Thomson Reuters examines the influence and impact of scholarly research journals. JCR reveals the relationship between citing and cited journals, offering a systematic, objective means to evaluate the world's leading journals. Find out more about IEEE Journal Rankings.
Call for Special Issues
IEEE TNNLS Special Issue on "Deep Integration of Artificial Intelligence and Data Science for Process Manufacturing," Guest Editors: Feng Qian, East China University of Science and Technology, China, Yaochu Jin, University of Surrey, United Kingdom, S. Joe Qin, University of Southern California, United States, Kai Sundmacher, Max Planck Institute for Dynamics of Complex Technical Systems, Germany. Submission Deadline: October 30, 2019. [Call for Papers]
IEEE TNNLS Special Issue on "Robust Learning of Spatio-Temporal Point Processes: Modeling, Algorithm, and Applications," Guest Editors: Junchi Yan, Shanghai Jiao Tong University, China, Hongteng Xu, Duke University, USA, Liangda Li, Yahoo Research, USA, Mehrdad Farajtabar, DeepMind, USA, Xiaokang Yang, Shanghai Jiao Tong University, China. Submission Deadline: October 20, 2019. [Call for Papers]
IEEE TNNLS Special Issue on "Deep Representation and Transfer Learning for Smart and Connected Health," Guest Editors: Vasile Palade, Coventry University, UK,Stefan Wermter, University of Hamburg, Germany,Ariel Ruiz-Garcia, Coventry University, UK,Antonio de Padua Braga, University of Minas Gerais, Brazil,Clive Cheong Took, Royal Holloway (University of London), UK.Submission Deadline: September 30, 2019. [Call for Papers]
The Boundedness Conditions for Model-Free HDP( λ )
Authors: Seaar Al-Dabooni, Donald Wunsch
Publication: IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
Issue: Volume 30, Issue 7 – July 2019
Abstract: This paper provides the stability analysis for a model-free action-dependent heuristic dynamic programing (HDP) approach with an eligibility trace long-term prediction parameter (λ). HDP(λ) learns from more than one future reward. Eligibility traces have long been popular in Q-learning. This paper proves and demonstrates that they are worthwhile to use with HDP. In this paper, we prove its uniformly ultimately bounded (UUB) property under certain conditions. Previous works present a UUB proof for traditional HDP [HDP(λ = 0)], but we extend the proof with the λ parameter. By using Lyapunov stability, we demonstrate the boundedness of the estimated error for the critic and actor neural networks as well as learning rate parameters. Three case studies demonstrate the effectiveness of HDP(λ). The trajectories of the internal reinforcement signal nonlinear system are considered as the first case. We compare the results with the performance of HDP and traditional temporal difference [TD(λ)] with different λ values. The second case study is a single-link inverted pendulum. We investigate the performance of the inverted pendulum by comparing HDP(λ) with regular HDP, with different levels of noise. The third case study is a 3-D maze navigation benchmark, which is compared with state action reward state action, Q(λ), HDP, and HDP(λ). All these simulation results illustrate that HDP(λ) has a competitive performance; thus this contribution is not only UUB but also useful in comparison with traditional HDP.
Index Terms: λ-return, action dependent (AD), approximate dynamic programing (ADP), heuristic dynamic programing (HDP), Lyapunov stability, model free, uniformly ultimately bounded (UUB)
IEEE Xplore Link: https://ieeexplore.ieee.org/document/8528554