Compositional Reinforcement Learning from Logical Specifications 

Kishor Jothimurugan University of Pennsylvania 

Osbert Bastani University of Pennsylvania 

Suguman Bansal University of Pennsylvania 

Rajeev Alur University of Pennsylvania 

Abstract 

We study the problem of learning control policies for complex tasks given by logical specifications. Recent approaches automatically generate a reward function from a given specification and use a suitable reinforcement learning algorithm to learn a policy that maximizes the expected reward. These approaches, however, scale poorly to complex tasks that require high-level planning. In this work, we develop a compositional learning approach, called DIRL, that interleaves highlevel planning and reinforcement learning. First, DIRL encodes the specification as an abstract graph; intuitively, vertices and edges of the graph correspond to regions of the state space and simpler sub-tasks, respectively. Our approach then incorporates reinforcement learning to learn neural network policies for each edge (sub-task) within a Dijkstra-style planning algorithm to compute a high-level plan in the graph. An evaluation of the proposed approach on a set of challenging control benchmarks with continuous state and action spaces demonstrates that it outperforms state-of-the-art baselines

Keywords: Theory of the Online Firm · Markov Decision Process · Bi-level Optimization · Complexity Theory · Approximation Algorithms · Stackelberg Equilibrium

A deep machine learning algorithm for construction of the Kolmogorov-Arnold representation

A. Polar[^1] and M. Poluektov[^2]

[1] Independent Software Consultant, Duluth, GA, USA

[2] International Institute for Nanocomposites Manufacturing, WMG, University of Warwick, Coventry CV4 7AL, UK 

Abstract

The Kolmogorov-Arnold representation is a proven adequate replacement of a continuous multivariate function by an hierarchical structure of multiple functions of one variable. The proven existence of such representation inspired many researchers to search for a practical way of its construction, since such model answers the needs of machine learning. This article shows that the Kolmogorov-Arnold representation is not only a composition of functions but also a particular case of a tree of the discrete Urysohn operators. The article introduces new, quick and computationally stable algorithm for constructing of such Urysohn trees. Besides continuous multivariate functions, the suggested algorithm covers the cases with quantised inputs and combination of quantised and continuous inputs. The article also contains multiple results of testing of the suggested algorithm on publicly available datasets, used also by other researchers for benchmarking. 

Keywords: deep machine learning, Kolmogorov-Arnold representation, discrete Urysohn operator, classification trees.

Abstract: 

Using basic category theory (category, functor, natural transformation, etc.), this Element describes all the central concepts and proves the main theorems of theoretical computer science. Category theory, which works with functions, processes, and structures, is uniquely qualified to present the fundamental results of theoretical computer science. In this text, we meet some of the deepest ideas and theorems of modern computers and mathematics, e.g., 

• Turing machines, 
• unsolvable problems, 
• the P=NP question, 
• Kurt Gödel’s incompleteness theorem, 
• intractable problems, 
• Turing’s Halting problem, 
• ... 

The concepts come alive with many examples and exercises. This short text covers the usual material taught in a year-long course.

This is the book written by Noson S. Yanofsky. 

There is another paper with the same name as this book from the same author. 

Abstract. Theoretical computer science discusses foundational issues about computations. It asks and answers questions such as:

• “What is a computation?”, 
• “What is computable?”, 
• “What is efficiently computable?”,
• “What is information?”, 
• “What is random?”, 
• “What is an algorithm?”, 
• ... 

We will present many of the major themes and theorems with the basic language of category theory. Surprisingly, many interesting theorems and concepts of theoretical computer science are easy consequences of functoriality and composition when you look at the right categories and functors connecting them.

The more legs the merrier: A newcomposition for symmetric (multi-)lenses 

Michael Johnson CoACT, Departments of Mathematics and Computing Macquarie University michael.johnson@mq.edu.au Robert Rosebrugh Department of Mathematics and Computer Science Mount Allison University rrosebrugh@mta.ca 

This paper develops a new composition of symmetric lenses that preserves information which is important for implementing system interoperation. It includes a cut-down but realistic example of a multi-system business supply chain and illustrates the new mathematical content with analysis of the systems, showing how the new composition facilitates the engineering required to implement the interoperations. All of the concepts presented here are based on either pure category theory or on experience in solving business problems using applied category theory.

CATEGORICALONTOLOGY I - EXISTENCE

DARIO DENTAMARO AND FOSCO LOREGIAN

Abstract.

The present paper is the first piece of a series whose aim is to develop an approach to ontology and metaontology through category theory. We exploit the theory of elementary toposes to claim that a satisfying “theory of existence”, and more at large ontology itself, can  both be obtained through categorytheory. In this perspective, an ontology is a mathematical  object: it is a category, the universe of discourse in which our mathematics (intended at large,  as a theory of knowledge) can be deployed. The internal language that all categories possess prescribes the modes of existence for the objects of a fixed ontology/category.  

This approach resembles, but is more general than,fuzzy logics, as most choices of $\mathcal{E}$ and thus of $\Omega_\mathcal{E}$ yield non classical, many-valued logics.

Framed this way, ontology suddenly becomes more mathematical: a solid corpus of techniques can be used to back up philosophical intuition with a useful, modular language, suitable  for a practical foundation. As both a test-bench for our theory, and a literary divertissement,  we propose a possible category-theoretic solution of Borges’ famous paradoxes of Tlön’s “nine copper coins”, and of other seemingly paradoxical construction in his literary work. We then  delve into the topic with some vistas on our future works.

A KIF Formalization for the IFF Category Theory Ontology 

Robert E. Kent 

550 Staley Dr. 

Pullman, WA 99163 

rekent@ontologos.org 

Abstract 

This paper begins the discussion of how the Information Flow Framework can be used to provide a principled foundation for the metalevel (or structural level) of the Standard Upper Ontology (SUO). This SUO structural level can be used as a logical framework for manipulating collections of ontologies in the object level of the SUO or other middle level or domain ontologies. From the Information Flow perspective, the SUO structural level resolves into several metalevel ontologies. This paper discusses a KIF formalization for one of those metalevel categories, the Category Theory Ontology. In particular, it discusses its category and colimit sub-namespaces.

Keywords: Hypergraph categories, compact closed categories, Frobenius algebras, cospan, wiring diagram.

Keywords: regular logic, category theory, primitive positive formula

Lenses are a well-established structure for modelling bidirectional trans- formations, such as the interactions between a database and a view of it. Lenses may be symmetric or asymmetric, and may be composed, forming the morphisms of a monoidal category. More recently, the notion of a learner has been proposed: these provide a compositional way of modelling supervised learning algorithms, and again form the morphisms of a monoidal category. In this paper, we show that the two concepts are tightly linked. We show both that there is a faithful, identity-on-objects symmetric monoidal functor embedding a category of asymmetric lenses into the category of learners, and furthermore there is such a functor embedding the category of learners into a category of symmetric lenses.