Modelling dynamic programming problems by generalized d-graphs

Dynamic programming (DP) as optimization method was proposed by Richard Bellman in 1957 [1]. Since the first book in applied dynamic programming was published in 1962 [2] DP has become a current problem solving method in several fields of science: Applied mathematics [2], Computer science [3], Artificial Intelligence [6], Bioinformatics [4], Macroeconomics [13], etc. Even in the early book on DP [2] the authors drew attention to the fact that some dynamic programming strategies can be formulated as graph search problems. Later this subject was largely researched. As recent examples: Georgescu and Ionescu introduced the concept of DP-tree [7]; Kátai [8] proposed d-graphs as special hierarchic dependency-graphs for modelling DP problems; Lew and Mauch [14,15,16] used specialized Petri Net models to represent DP problems (Lew called his model Bellman-Net).

All the above mentioned modelling tools are based on cycle free graphs. As Mauch [16] states, circularity is undesirable if Petri Nets represent DP problem instances. On the other hand, however, there are DP problems with “cyclic functional equation” (the chain of recursive dependences of the functional equation is cyclic). Felzenszwalb and Zabih [5] in their survey entitled Dynamic programming and graph algorithms in computer vision recall that many dynamic programming algorithms can be viewed as solving a shortest path problem in a graph (see also [9,11,12]. But, interestingly, some shortest path algorithms work in cyclic graphs too. Kátai, after he has been analyzing the three most common single-source shortest path algorithms (The algorithm based on the topological order of the vertices, Dijkstra algorithm and Bellman-Ford algorithm), concludes that all these algorithms apply cousin DP strategies [10,17]. Exploiting this observation Kátai and Csiki [12] developed general DP algorithms for discrete optimization problems that can be modelled by simple digraphs (see also [11]). In this paper, modelling finite discrete optimization problems by generalized d-graphs (admitting cycles), we extend the previously mentioned method for a more general class of DP problems. The presented new modelling method also makes possible to classify DP problems and the corresponding DP strategies in term of graph theory.

Then again the most common approach taken today for solving real-world DP problems is to start a specialized software development project for every problem in particular. There are several reasons why is benefiting to use the most specific DP algorithm possible to solve a certain optimization problem. For instance this approach commonly results in more efficient algorithms. But a number of researchers in the above mentioned various fields of applications are not experts in programming. Dynamic programming problem solving process can be divided into two steps: (1) the functional equation of the problem is established (a recursive formula that implements the principle of the optimality); (2) a computer program is elaborated that processes the recursive formula in a bottom-up way [12]. The first step is reachable for most researchers, but the second one not necessary. Attaching graph-based models to DP problems results in the following benefits:

• it moves DP problems to a well research area: graph theory, • it makes possible to class DP strategies in terms of graph theory,

• as an intermediate representation of the problem (that hides, to some degree, the variety of DP problems) it enables to automate the program-ming part of the problem-solving process by an adequately developed software-tools [12], • a general software-tool that automatically solves DP problems (getting as input the functional equation) should be able to save considerable software development costs [16].

DP can be used to solve optimization problems (discrete, finite space) that satisfy the principle of the optimality: The optimal solution of the original problem is built on optimal sub-solutions respect to the corresponding subproblems. The principle of the optimality implicitly suggests that the problem can be decomposed into (or reduced to) similar sub-problems. Usually this operation can be performed in several ways. The goal is to build up the optimal solution of the original problem from the optimal solutions of its smaller subproblems. Optimization problems can often be viewed as special version of more general problems that ask for all solutions, not only for the optimal one (A so-called objective function is defined on the set of sub-problems, which has to be optimized). We will call this general version of the problem, allsolutions-version.

The set of the sub-problems resulted from the decomposing process can adequately be modelled by dependency graphs (We have proposed to model the problem on the basis of the previously established functional equation that can be considered the output of the mathematical part and the input of the programming part of the problem solving process

This content is AI-processed based on open access ArXiv data.