This paper describes how the performance of motion primitive-based planning algorithms can be improved using reinforcement learning. Specifically, we describe and evaluate a framework that autonomously improves the performance of a primitive-based motion planner. The improvement process consists of three phases: exploration, extraction, and reward updates. This process can be iterated continuously to provide successive improvement. The exploration step generates new trajectories, and the extraction step identifies new primitives from these trajectories. These primitives are then used to update rewards for continued exploration. This framework required novel shaping rewards, development of a primitive extraction algorithm, and modification of the Hybrid A* algorithm. The framework is tested on a navigation task using a nonlinear F-16 model. The framework autonomously added 91 motion primitives to the primitive library and reduced average path cost by 21.6 s, or 35.75% of the original cost. The learned primitives are applied to an obstacle field navigation task, which was not used in training, and reduced path cost by 16.3 s, or 24.1%. Additionally, two heuristics for the modified Hybrid A* algorithm are designed to improve effective branching factor.
Multi-phase, pseudospectral optimization is employed in a variety of applications, but many of the world-class optimization libraries are closed-source. In this paper we formulate an open-source, object-oriented framework for dynamic optimization using the Pyomo modeling language. This strategy supports the reuse of common code for rapid, error-free model development. Flexibility of our framework is demonstrated on a series of dynamic optimization problems, including multi-phase trajectory optimization using highly accurate pseudospectral methods and controller gain optimization in the presence of stability margin constraints. We employ numerical procedures to improve convergence rates and solution accuracy. We validate our framework using GPOPS-II, a commercial, MATLAB-based optimization program, for a vehicle ascent problem. The trajectory results show close alignment with this state-of-the-art optimization suite.
Pyomo and Dakota are openly available software packages developed by Sandia National Labs. In this tutorial, methods for automating the optimization of controller parameters for a nonlinear cart-pole system are presented. Two approaches are described and demonstrated on the cart-pole example problem for tuning a linear quadratic regulator and also a partial feedback linearization controller. First the problem is formulated as a pseudospectral optimization problem under an open box methodology utilizing Pyomo, where the plant model is fully known to the optimizer. In the next approach, a black-box approach utilizing Dakota in concert with a MATLAB or Simulink plant model is discussed, where the plant model is unknown to the optimizer. A comparison of the two approaches provides the end user the advantages and shortcomings of each method in order to pick the right tool for their problem. We find that complex system models and objectives are easily incorporated in the Dakota-based approach with minimal setup time, while the Pyomo-based approach provides rapid solutions once the system model has been developed.
Select one or more publication years and click "Update search results".
This list has already been filtered by author.
SELECTED PUBLICATION YEARS
MATCHING PUBLICATION YEARS
ALL PUBLICATION YEARS
No matches found.
Select a document type
Select one or more document types and click "Update search results".
This list has already been filtered by author.
SELECTED DOCUMENT TYPES
MATCHING DOCUMENT TYPES
ALL DOCUMENT TYPES
No matches found.
Search for an author
Search for a Sandian author by first name, last name, or initials. Click on the author's name to add them as an option, and then click "Update search results".
This list has already been filtered by author.
SELECTED AUTHORS
MATCHING AUTHORS
ALL AUTHORS
No matches found.
Search for a funding sponsor
Search for one or more funding sponsors and click "Update search results".
This list has already been filtered by author.
SELECTED FUNDING SPONSORS
MATCHING FUNDING SPONSORS
ALL FUNDING SPONSORS
No matches found.
Search for a research partner
Search for one or more research partners and click "Update search results".
This list has already been filtered by author.
SELECTED RESEARCH PARTNERS
MATCHING RESEARCH PARTNERS
ALL RESEARCH PARTNERS
No matches found.
Search for a subject
Search for one or more subjects and click "Update search results".