3.4 Hierarchical controls for production-investment models

Sethi, Taksar, and Zhang (1992b) incorporate an additional capacity expansion decision in the model discussed in Section 3.1. They consider a stochastic manufacturing system with the inventory/backlog or surplus ${\mbox{\boldmath$x$ }}^{\varepsilon}(t)\in R^n$ and production rate ${\mbox{\boldmath$u$ }}^{\varepsilon}(t)\in R^n$ that satisfy

$$\dot {\mbox{\boldmath$x$ }}^{\varepsilon}(t)={\mbox{\boldmath$u$ ......math$z$ }},\; \{\mbox{\boldmath$x$ }}^{\varepsilon}(0)={\mbox{\boldmath$x$ }},$$ (3.25)

where ${\mbox{\boldmath$z$ }}\in R^n$ denotes the constant demand rate and${\mbox{\boldmath$x$ }}$ is the initial surplus level. They assume${\mbox{\boldmath$u$ }}^{\varepsilon}(t)\geq 0$ and ${\mbox{\boldmath$r$ }}\cdot{\mbox{\boldmath$u$ }}^{\varepsilon}(t)\leq {\mbox{\boldmath$m$ }}(\varepsilon,t)$ for some${\mbox{\boldmath$r$ }} \geq 0$, where ${\mbox{\boldmath$m$ }}(\varepsilon, t)$ is the machine capacity process described by (3.27). The specification of ${\mbox{\boldmath$m$ }}(\varepsilon, t)$ involves the instantaneous purchase of some given additional capacity at some time $\tau$,$0\leq \tau\leq\infty$, at a cost of K, where $\tau=\infty$ means not to purchase it at all; see Sethi, Taksar, and Zhang (1994a) for an alternate model in which the investment in the additional capacity is continuous. Therefore, their control variable is a pair $(\tau,{\mbox{\boldmath$u$ }}(\cdot))$ of a Markov time $\tau \geq 0$ and a production process${\mbox{\boldmath$u$ }}(\cdot)$ over time.

They consider the cost function $J^{\varepsilon}$ defined by

$$J^{\varepsilon}({\mbox{\boldmath$x$ }},{\mbox{\boldmath$m$ }},\t......}^{\varepsilon}(t),{\mbox{\boldmath$u$ }}^{\varepsilon}(t))dt+Ke^{-\rho\tau}],$$ (3.26)

where ${\mbox{\boldmath$m$ }}(\varepsilon,0)={\mbox{\boldmath$m$ }}$ is the initial capacity and$\rho>0$ is the discount rate. The problem is to find an admissible control $(\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$ that minimizes$J^{\varepsilon}({\mbox{\boldmath$x$ }},{\mbox{\boldmath$m$ }},\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$.

Define $m_1(\varepsilon,t)$ and $m_2(\varepsilon,t)$ as two Markov processes with state spaces ${\cal M}_1=\{0,1,\cdots,m_1\}$ and${\cal M}_2=\{0,1,\cdots,m_1+m_2\}$, respectively. Here,$m_1(\varepsilon,t)\geq 0$ denotes the existing production capacity process and $m_2(\varepsilon,t)\geq 0$ denotes the capacity process of the system if it were to be supplemented by the additional new capacity at time t=0.

Define further a new process ${\mbox{\boldmath$m$ }}(\varepsilon, t)$ as follows: For each ${\cal F}_1(t)$-Markov time $\tau \geq 0$,

$${\mbox{\boldmath$m$ }}(\varepsilon,t)=\left\{\begin{array}{ll}m......boldmath$m$ }}(\varepsilon,\tau)=m_2(\varepsilon,0):=m_1(\varepsilon,\tau)+m_2.$$ (3.27)

Here m₂ denotes the maximum additional capacity resulting from the investment in the new capacity. We make the following assumptions on the cost function $G(\cdot,\cdot)$ and the process ${\mbox{\boldmath$m$ }}(\varepsilon, t)$.

Assumption 3.7$G({\mbox{\boldmath$x$ }},{\mbox{\boldmath$u$ }})$ is a nonnegative jointly convex function that is strictly convex in either ${\mbox{\boldmath$x$ }}$ or ${\mbox{\boldmath$u$ }}$ or both. For all ${\mbox{\boldmath$x$ }},{\mbox{\boldmath$x$ }}' \in R^n$ and ${\mbox{\boldmath$u$ }}, {\mbox{\boldmath$u$ }}' \in R^n_+$, there exist constant C₃₅ and $\kappa_{33}$ such that

$$\left\vert G({\mbox{\boldmath$x$ }},{\mbox{\boldmath$u$ }})......\vert{\mbox{\boldmath$u$ }}-{\mbox{\boldmath$u$ }}'\vert\right]$$

Assumption 3.8$m_1(\varepsilon,t)\in{\cal M}_1$ and $m_2(\varepsilon,t)\in{\calM}_2$ are Markov processes with generators $\varepsilon^{-1}Q_1$ and $\varepsilon^{-1}Q_2$, respectively, whereQ₁=(q⁽¹⁾_ij) and Q₂=(q⁽²⁾_ij) are matrices such that $q^{(k)}_{ij}\geq0$ if $i\neq j$ and$q^{(k)}_{ii}=-\sum_{i\neq j}q^{(k)}_{ij}$ for k=1,2. Moreover, Q₁ and Q₂ are both irreducible.Let ${\cal F}_1(t)=\sigma\{m_1(\varepsilon,s):0 \leq s\leq t\}$,${\cal F}_2(t)=\sigma\{m_2(\varepsilon,t):0 \leq s\leq t\}$, and ${\cal F}(t)=\sigma\{{\mbox{\boldmath$m$ }}(\varepsilon,t):0 \leq s\leq t\}$.

Definition 3.5   We say that a control$(\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$ is admissible if

(i)

$\tau$ is an ${\cal F}_1(t)$-Markov time;

(ii)

${\mbox{\boldmath$u$ }}^{\varepsilon}(t)$ is ${\cal F}(t)$-adapted and${\mbox{\boldmath$r$ }}\cdot {\mbox{\boldmath$u$ }}^{\varepsilon}(t)\leq m(\varepsilon,t)$ for$t\geq 0$.

We use ${\cal A}^{\varepsilon} ({\mbox{\boldmath$x$ }},{\mbox{\boldmath$m$ }})$ to denote the set of all admissible controls $(\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$. Then the problem is: $$\begin{eqnarray*}{\cal P}^{\varepsilon}:\left\{\begin{array}{cl}\min_{(\......}}^{\varepsilon}(0)={\mbox{\boldmath$x$ }}.\end{array}\right.\end{eqnarray*}$$ We use $v^{\varepsilon}({\mbox{\boldmath$x$ }},m)$ to denote the value function of the problem and define an auxiliary value function$v^{\varepsilon}_a({\mbox{\boldmath$x$ }},m')$ to be K plus the optimal cost with the capacity process $m_2(\varepsilon,t)$ with the initial capacity $m'\in{\cal M}_2$ and no future capital expansion possibilities. Then the dynamic programming equations are as follows:

$$\min\big\{ \min_{{\mbox{\boldmath$u$ }}\geq 0,{\mbox{\boldmath$r$......u$ }})\big ]+\varepsilon^{-1}Q_1v^\varepsilon({\mbox{\boldmath$x$ }},\cdot)(m)$$

$$\ \ \ \ \ \ \ \ \\ \ \ \ -\rho v^\varepsilon({\mbox{\boldmath$x$......h$x$ }},m+m_2)-v^\varepsilon({\mbox{\boldmath$x$ }},m)\big \}=0,m\in{\cal M}_1$$ (3.28)

$$\min_{{\mbox{\boldmath$u$ }}\geq 0,{\mbox{\boldmath$r$ }}\cdot{\m......$u$ }})]+\varepsilon^{-1} Q_2 v^\varepsilon_a({\mbox{\boldmath$x$ }},\cdot)(m)$$

$$\ \ \ \ \ \ \ \ \ \ -\rho(v^\varepsilon_a({\mbox{\boldmath$x$ }},m)-K)=0, m\in{\cal M}_2.$$ (3.29)

Let $\nu^{(1)}=(\nu^{(1)}_0,\nu^{(1)}_1,\cdots,\nu^{(1)}_{m_1})$ and$\nu^{(2)}=(\nu^{(2)}_0,\nu^{(2)}_1,\cdots,\nu^{(2)}_{m_1+m_2})$ denote the equilibrium distributions of Q₁ and Q₂, respectively. We now proceed to develop a limiting problem. We first define the control sets for the limiting problem. Let $$U_1=\{({\mbox{\boldmath$u$ }}^0,\cdots,{\mbox{\boldmath$u$ ......0,{\mbox{\boldmath$r$ }}\cdot{\mbox{\boldmath$u$ }}^i\leq i\}$$

and

$$U_2=\{({\mbox{\boldmath$u$ }}^0,\cdots,{\mbox{\boldmath$u$ ......{\mbox{\boldmath$r$ }}\cdot {\mbox{\boldmath$u$ }}^i\leq i\}.$$

Then $U_1\subset R^{n\times(m_1+1)}$ and $U_2\subset R^{n\times(m_1+m_2+1)}.$
Definition 3.6   We use ${\cal A}^0({\mbox{\boldmath$x$ }})$ to denote the set of the following controls (admissible controls for the limiting problem):

(i)

a deterministic time $\sigma$;

(ii)

a deterministic U(t) such that for $t<\sigma$,$U(t)=({\mbox{\boldmath$u$ }}^0(t),\cdots,{\mbox{\boldmath$u$ }}^{m_1}(t))\in U_1$ and for$t\geq\sigma$,$U(t)=({\mbox{\boldmath$u$ }}^0(t),\cdots,{\mbox{\boldmath$u$ }}^{m_1+m_2}(t))\inU_2$.

Let $$\begin{eqnarray*}J({\mbox{\boldmath$x$ }},\sigma,U(\cdot)) &=&\int_0^\sigma e^......h$x$ }}(t),{\mbox{\boldmath$u$ }}^i(t))dt+e^{-\rho \sigma} K,\end{eqnarray*}$$ and let $$\begin{eqnarray*}\bar {\mbox{\boldmath$u$ }}(t)= \left\{\begin{array}{ll}\......th$u$ }}^i(t)&\mbox{if} \ \t\geq\sigma.\end{array}\right.\end{eqnarray*}$$ We can now define the following limiting optimal control problem:

$${\cal P}^0:\left\{\begin{array}{cl}\displaystyle \min_{(\sigma......h$z$ }},\; {\mbox{\boldmath$x$ }}(0)={\mbox{\boldmath$x$ }}.\end{array}\right.$$ (3.30)

Let $(v({\mbox{\boldmath$x$ }}),v_a({\mbox{\boldmath$x$ }}))$ denote the value functions for${\cal P}^0$. Let $(\tau,U(\cdot))\in {\cal A}^0$ denote any admissible control for the limiting problem ${\cal P}^0,$ where

$$\begin{eqnarray*}U(t)=\left\{\begin{array}{ll}({\mbox{\boldmath$u$ }}^0(t)......}(t))\inU_2&\mbox{if} \ \ t\geq \sigma.\end{array}\right.\end{eqnarray*}$$ We take $$\begin{eqnarray*}{\mbox{\boldmath$u$ }}^\varepsilon(t)=\left\{\begin{array}{......psilon,t)=i\}}&\mbox{if} \ \ t\geq\tau.\end{array}\right.\end{eqnarray*}$$ Then the control $(\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$ is admissible for ${\cal P}^{\varepsilon}$.
The following result is proved in Sethi, Taksar, and Zhang (1992b).

Theorem 3.3 (i)   There exists a constant C₃₆ such that

$$\begin{eqnarray*}\big \vert v^\varepsilon({\mbox{\boldmath$x$ }},m)- v({\mbox{......t{\mbox{\boldmath$x$ }}\vert^{\kappa_{33}}) \sqrt{\varepsilon}.\end{eqnarray*}$$ (ii) Let$(\tau,U(\cdot))\in {\cal A}^0$be an$\varepsilon$-optimal control for the limiting problem ${\cal P}^0$ and let $(\tau,{\mbox{\boldmath$u$ }}^\varepsilon(\cdot))\in {\calA}^{\varepsilon}$ be the control constructed above. Then,$(\tau,{\mbox{\boldmath$u$ }}^{\varepsilon}(\cdot))$ is asymptotically optimal with error bound $\sqrt{\varepsilon}$, i.e., $$\begin{eqnarray*}\big\vert J^\varepsilon({\mbox{\boldmath$x$ }},m,\tau,{\mbox{......\mbox{\boldmath$x$ }}\vert^{\kappa_{33}}) \sqrt{\varepsilon}.\end{eqnarray*}$$