5.2 Hierarchical control of single or parallel machine systems without deterioration and cancelation rates

Consider a manufacturing system whose system dynamics is given by (5.1) with a_k=0, k=1,...,n, that is,

$$\dot {\mbox{\boldmath$x$ }}^{\varepsilon}(t)={\mbox{\boldmath$u$ }}^{\varepsilon}(t)-{\mbox{\boldmath$z$ }}.$$ (5.13)

We assume that the machine capacity process $m(\varepsilon,t)$ is a finite-state Markov process given in Section 3.1. Now we define the set of admissible controls as follows:

Definition 5.3   We say that a control ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot)=(u_1^{\varepsilon}(\cdot),...,u_n^{\varepsilon}(\cdot))=\{{\mbox{\boldmath$u$ }}^{\varepsilon}(t): \ t\geq 0\}$ is admissible, if

(i)

${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot )$ is an ${\cal F}^{\varepsilon}_t =\sigma\{m(\varepsilon, s), \ 0 \leq s\leq t\}$ adapted measurable process;

(ii)

$u^{\varepsilon}_i(t)\geq 0$ and $\sum_{i=1}^nu^{\varepsilon}_i(t) \leqm(\varepsilon,t)$.

We use ${\cal A}^{\varepsilon}(i)$ to denote the set of all admissible controls with the initial condition$m(\varepsilon,0)=i$.

Definition 5.4   A function ${\mbox{\boldmath$u$ }}({\mbox{\boldmath$x$ }},j)$ defined on$R^n\times {\cal M}$ is called an admissible feedback control, or simply a feedback control, if

(i)

for any given initial surplus ${\mbox{\boldmath$x$ }}$ and production capacity i, the equation

$$\dot {{\mbox{\boldmath$x$ }}}^{\varepsilon}(t)={\mbox{\bold......mbox{\boldmath$x$ }}^{\varepsilon}(0)={\mbox{\boldmath$x$ }},$$

has a unique solution;

(ii)

the control defined by${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot)=\{{\mbox{\boldmath$u$ }}^{\varepsil......}^{\varepsilon}(t),m(\varepsilon,t)), \ t\geq0\}\in {\cal A}^{\varepsilon}(i)$.

With a slight abuse of notation, we simply call ${\mbox{\boldmath$u$ }}({\mbox{\boldmath$x$ }},i)$ a feedback control when no ambiguity arises. For any ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot) \in{\cal A}^{\varepsilon} (i)$, define $$\begin{eqnarray*}\bar J^{\varepsilon}({\mbox{\boldmath$x$ }}, i, {\mbox{\boldm......lon}(t))+c({\mbox{\boldmath$u$ }}^{\varepsilon}(t)) \right )dt,\end{eqnarray*}$$ where ${\mbox{\boldmath$x$ }}^{\varepsilon}(\cdot)$ is the surplus process corresponding to the production rate ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot )$ with ${\mbox{\boldmath$x$ }}^{\varepsilon}(0)={\mbox{\boldmath$x$ }}$. Our goal is to choose ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot) \in{\cal A}^{\varepsilon} (i)$ so as to minimize the cost $\bar J^{\varepsilon}({\mbox{\boldmath$x$ }},i,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$. We formally summarize the control problem as follows:

$$\bar {\cal P}^{\varepsilon} \ : \left \{\begin{array}{lll}&\min......dmath$x$ }},i, {\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot)).\end{array}\right.$$ (5.14)

We make the same assumptions on the surplus cost function $h(\cdot)$ and the production cost function $c(\cdot)$, and the Markov chain $m(\varepsilon,t)$ as in Section 5.1. Without a positive inventory deterioration/cancelation rate for each product type, the system state ${\mbox{\boldmath$x$ }}(\cdot)$ may no longer remain bounded. Thus the approach presented in Section 5.1 cannot be used for the system given by (5.13). Here we will use the vanishing discount approach to treat the problem. In order to derive the dynamic programming equation for the average cost control problem formulated above and carry out an asymptotic analysis of the minimum long-run average expected cost for (5.14), we introduce a related control problem with the cost discounted at a rate $\rho>0$. For ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot) \in{\cal A}^{\varepsilon} (i)$, let $J^{\varepsilon,\rho}({\mbox{\boldmath$x$ }},i,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$ denote the cost function, i.e., $$J^{\varepsilon,\rho}({\mbox{\boldmath$x$ }},i,{\mbox{\boldm......lon}(t))+c({\mbox{\boldmath$u$ }}^{\varepsilon}(t))\right)dt,$$

where ${\mbox{\boldmath$x$ }}^{\varepsilon}(\cdot)$ satisfies (5.13), i.e., the surplus process corresponding to the production rate ${\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot )$. Then, our related control problem can be written as (3.3) in Section 3.1.

In order to study the long-run average cost control problem by using the vanishing discount approach, we must first obtain some estimates for the value function $V^{\varepsilon,\rho}({\mbox{\boldmath$x$ }},i)$($=\inf_{{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot) \in {\calA}^{\varepsilon}(......n,\rho}({\mbox{\boldmath$x$ }},i,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$). Sethi and Zhang (1998) prove the following result.

Theorem 5.5   There exist constants$\rho_0$and$\varepsilon_0$such that

(i)

$\{ \rho V^{\varepsilon, \rho}(0,0): \ 0 <\rho \leq \rho_0,\0 < \varepsilon \leq \varepsilon_0\}$is bounded;

(ii)

For $\varepsilon \in (0,\varepsilon_0]$and$\rho \in (0, \rho_0]$, the function

$$W^{\varepsilon, \rho}({\mbox{\boldmath$x$ }}, j)\equiv V^{\...... \rho}({\mbox{\boldmath$x$ }},j)-V^{\varepsilon, \rho}(0,0)$$

is convex in${\mbox{\boldmath$x$ }}$;

(iii)

For $\varepsilon \in (0,\varepsilon_0]$and$\rho \in (0, \rho_0]$,$W^{\varepsilon, \rho}({\mbox{\boldmath$x$ }},j)$is locally Lipschitz continuous in x, i.e., there exists a constant C independent of $\rho$and$\varepsilon$, such that

$$\vert W^{\varepsilon, \rho}({\mbox{\boldmath$x$ }},j)-W^{\v......31}})\vert{\mbox{\boldmath$x$ }}-{\mbox{\boldmath$x$ }}'\vert$$

for $j \in {\cal M}$and all${\mbox{\boldmath$x$ }},{\mbox{\boldmath$x$ }}' \in R^n$, where$\kappa_{31}$is given in Assumption 3.2.

Corresponding to the dynamic programming equation (3.4) associated with the discounted cost case, we can formally write the dynamic programming equation associated with the long-run average cost case as

$$\hat \lambda^{\varepsilon}= \inf_{{\mbox{\boldmath$u$ }}\in \Omeg......\sum_{j=0}^mq^{\varepsilon}_{ij}\barW^{\varepsilon}({\mbox{\boldmath$x$ }},j),$$ (5.15)

for any $i\in {\cal M}$ and ${\mbox{\boldmath$x$ }}\in R^n$, where $\hat\lambda^{\varepsilon}$ is a constant and $\bar W^{\varepsilon}({\mbox{\boldmath$x$ }},i)$ is a real-valued function on $R^n\times {\cal M}$.

Lemma 5.1   The dynamic programming equation (5.15) has a unique solution in the following sense: If$(\hat \lambda^{\varepsilon}_1, \bar W^{\varepsilon}_1({\mbox{\boldmath$x$ }},i))$and$(\hat \lambda_2 ^{\varepsilon}, \barW_2^{\varepsilon}({\mbox{\boldmath$x$ }},i))$are viscosity solutions to (5.15), then$\hat \lambda^{\varepsilon}_1=\hat\lambda^{\varepsilon}_2$.

For the proof, see Theorem G.1 in Sethi and Zhang (1994a).

Remark 5.3   While the proof of Theorem G.1 in Sethi and Zhang (1994a) works for the uniqueness of $\hat\lambda^{\varepsilon}$, it cannot be adapted to prove whether$\bar W^{\varepsilon}_1({\mbox{\boldmath$x$ }},i)$ and $\barW_2^{\varepsilon}({\mbox{\boldmath$x$ }},i)$ are equal. Clearly, if $(\hat\lambda^{\varepsilon}, \bar W^{\varepsilon}({\mbox{\boldmath$x$ }},i))$ is a viscosity solution to (5.15), then for any constant C, $(\hat \lambda^{\varepsilon}, C_{55}+\barW^{\varepsilon}({\mbox{\boldmath$x$ }},i))$ is also a viscosity solution to (5.15). However, we do not need $\bar W^{\varepsilon}({\mbox{\boldmath$x$ }},i)$ to be unique for the purpose of this paper.According to (i) and (iii) of Theorem 5.5, for a fixed$\varepsilon \in (0, \epsilon_0]$ and any subsequence of$\{\rho\}$, there is a further subsequence denoted by $\{\rho'\}$, such that

$$\lambda^{\varepsilon,*}=\lim_{\rho' \rightarrow 0} \rho'V^{\varepsilon, \rho'}(0,0)$$ (5.16)

and

$$W^{\varepsilon}({\mbox{\boldmath$x$ }},j) \lim_{\rho'\rightarrow 0}W^{\varepsilon, \rho'}({\mbox{\boldmath$x$ }},j)$$ = (5.17)

for all $({\mbox{\boldmath$x$ }},j)\in R^n \times {\cal M}$. Furthermore, we have the following lemma proved in Sethi and Zhang (1998).

Lemma 5.2   There exists a constant$\varepsilon_0$such that for$\varepsilon \in (0,\varepsilon_0]$

(i)

$W^{\varepsilon}({\mbox{\boldmath$x$ }},j)$given in (5.17) is convex and is locally Lipschitz continuous in ${\mbox{\boldmath$x$ }}$, i.e., there is constant C such that

$$\vert W^{\varepsilon}({\mbox{\boldmath$x$ }},j)-W^{\varepsi......31}})\vert{\mbox{\boldmath$x$ }}-{\mbox{\boldmath$x$ }}'\vert$$

for all${\mbox{\boldmath$x$ }},{\mbox{\boldmath$x$ }}' \in R^n$and$j \in {\cal M}$;

(ii)

$(\lambda^{\varepsilon, *}, W^{\varepsilon}({\mbox{\boldmath$x$ }},j))$is a viscosity solution to the dynamic programming equation (5.15);

(iii)

there exists a constant C such that for all${\mbox{\boldmath$x$ }}\in R^n$and $i,j\in{\cal M}$,

$$\begin{eqnarray*}\frac {1}{\varepsilon}\vert W^{\varepsilon}({\mbox{\boldmat......qC_{57} (1+\vert{\mbox{\boldmath$x$ }}\vert^{\kappa_{31}+1}).\end{eqnarray*}$$

From (ii) of Lemma 5.2, for each subsequence of $\{\rho\}$, denoted by $\{\rho'\}$, the limit of $\{\rho'V^{\varepsilon,\rho'}(0,0)\}$ with some function $W^{\varepsilon}({\mbox{\boldmath$x$ }},i)$ is a viscosity solution to the dynamic programming equation (5.15). With the help of Lemma 5.1, we get that$\lambda^{\varepsilon, *}$, the limit of $\{\rho'V^{\varepsilon,\rho'}(0,0)\}$ does not depend on the choice of the subsequence$\{\rho'\}$. Thus, we have the following lemma.

Lemma 5.3$\lambda^{\varepsilon,\rho}$converges to$\lambda^{\varepsilon}$, as $\rho \rightarrow 0$.

Multiplying both sides of (5.15) at i by $\nu_i$, i=0,1,...,p defined in Assumption 2.3 and summing over i, we have

$$\lambda^{\varepsilon} =\sum_{i=0}^p \nu_i \inf_{{\mbox{\bol......({\mbox{\boldmath$u$ }})\right\}+h({\mbox{\boldmath$x$ }}),$$

keeping in mind that $\nu Q=0$. Then we let $\varepsilon\rightarrow0$ to obtain

$$\hat \lambda = \sum_{i=0}^p \nu_i \inf_{{\mbox{\boldmath$u$ }}\in......boldmath$x$ }}} + c({\mbox{\boldmath$u$ }})\right\} +h({\mbox{\boldmath$x$ }}).$$ (5.18)

This procedure will be justified in the next theorem. Note that $$\begin{eqnarray*}&&\sum_{j=0}^p \nu_j \inf_{{\mbox{\boldmath$u$ }}\in \omega(j......+ \sum_{j=0}^p \nu_j c({\mbox{\boldmath$u$ }}^{(j)})\right\},\end{eqnarray*}$$ where $\inf$ is taken by $( {\mbox{\boldmath$u$ }}^{(0)},{\mbox{\boldmath$u$ }}^{(1)},...,{\mbox{\boldmath$u$ }}^{(p)})$ in $\Omega (0)\times \Omega (1) \times \cdots \times\Omega(m)$. Thus, (5.18) is also equivalent to

$$\hat \lambda = \inf \left \{ \left ( \sum_{j=0}^p \nu_j{\mbox{\b......}^p \nu_j c({\mbox{\boldmath$u$ }}^{(j)})\right \} +h({\mbox{\boldmath$x$ }}),$$ (5.19)

which is the dynamic programming equation for the following deterministic problem:

$$\bar{\cal P}: \left\{\begin{array}{lll}&\displaystyle \mbox{min......({\mbox{\boldmath$x$ }},\bar{\mbox{\boldmath$u$ }}(\cdot)),\end{array}\right.$$ (5.20)

where $$\begin{eqnarray*}% latex2html id marker 5723\bar {\cal A}&& =\{\bar {\mbox{\......(i)}(\cdot) \ \mbox{ is a deterministic measurableprocess}\}.\end{eqnarray*}$$ With the help of (i) of Theorem 5.5, (i) of Lemma 5.2 and Lemma 5.3, Sethi and Zhang (1998) derive the convergence of the minimum average expected cost $\lambda^{\varepsilon}$ as $\varepsilon$ goes to zero.

Theorem 5.6$\lambda^{\varepsilon}$, the minimum average cost of the problem$\bar {\cal P}^{\varepsilon}$defined in (5.14), converges to$\lambda$, the minimum average cost of the limiting control problem$\bar {\calP}$defined in (5.20), i.e.,

$$\lim_{\varepsilon \rightarrow 0} \lambda^{\varepsilon}= \lambda.$$