\documentclass[12pt,american,11pt]{extarticle}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=1in,bmargin=1in,lmargin=1in,rmargin=1in}
\usepackage{babel}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{setspace}
\usepackage[authoryear]{natbib}
\doublespacing
\usepackage[unicode=true,pdfusetitle,
 bookmarks=true,bookmarksnumbered=false,bookmarksopen=false,
 breaklinks=true,pdfborder={0 0 0},pdfborderstyle={},backref=false,colorlinks=false]
 {hyperref}

\makeatletter
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.
\usepackage{lineno}

\newcommand{\myhttp}[1]{\href{http://#1}{\nolinkurl{#1}}}
\newcommand{\foothttp}[1]{\footnote{{\href{http://#1}{\nolinkurl{#1}}}.}}

\makeatother

\begin{document}

\appendix

\section{Appendix: A comparison of classical methods}

\numberwithin{equation}{section}

In recent years we have witnessed an interesting debate: HCC vs.\ the
life-table method (LT). Even though the controversy seems to be closed
after the insightful analyses of Elbasha and Chhatwal \citep{elbasha2016b,elbasha2016a},
we hope that this appendix can still shed some light on the issue.
Here ``classical methods'' are those not based on numerical integration.

\subsection{Absence of within-cycle transitions\label{subsec:Classical-approach}}

The usual way of presenting discrete-time Markov models states that
transition can only occur at the boundaries between cycles, i.e.,
at $t=0,\tau,2\tau,3\tau\ldots$ It implies that the state of the
system does not change within a cycle. When transitions are not allowed
at $t=0$, most authors say that ``transitions occur at the beginning
of each cycle''. In this case the probability in the interval $[i\tau,(i+1)\tau)$
is $P_{i}(s)$. With these assumptions and approximations, the total
cost is 
\begin{equation}
C_{E}=\tau\sum_{i=0}^{h-1}\sum_{s}P_{i}(s)\cdot c_{i}(s)\cdot\gamma(i\tau)\;,\label{eq:c-end}
\end{equation}
where the subindex $E$ stands for ``end''. When the instantaneous
cost function (cf.\ Eq.~1) decreases monotonically, this assumption
leads to an overestimation of the total cost, i.e.,$C_{E}$ is higher
than the true cost, $C$, given by Equation~2.

When transitions at $t=0$ are allowed, it is said that ``transitions
occur at the beginning of the cycle''. Therefore the probability
in the interval $(i\tau,(i+1)\tau]$ is $P_{i+1}(s)$ and the total
cost is 
\begin{equation}
C_{B}=\tau\sum_{i=0}^{h-1}\sum_{s}P_{i+1}(s)\cdot c_{i}(s)\cdot\gamma(i\tau)\;,\label{eq:c-beginning}
\end{equation}
where the subindex $B$ stands for ``beginning''. When the cost
function decreases monotonically, this assumption leads to an underestimation
of the total cost, so that we have $C_{B}<C<C_{E}$.

\subsection{The half-cycle correction\label{subsec:HCC}}

In an attempt to obtain a better approximation, \citep{sonnenberg1993}
introduced the half-cycle correction, which consists in including
a cycle of length $\tau/2$ at the beginning of the process, in which
no transition has yet occurred, and another cycle of the same length
at the end, so that the total duration of the process is $h\tau$.
This is equivalent to assuming that the transitions occur at the time
points $\{0.5\tau,1.5\tau,2.5\tau,\ldots\},$ i.e., halfway through
each cycle \citep{sonnenberg1993,naimark2008}. The approach is justified
in the literature as a method for approximating the\emph{ state-occupancy
probabilities}—the vertical axis in Figure~10 in \citep{sonnenberg1993},
reproduced as Figure~3 in \citep{naimark2008}, clearly shows it.
Those figures also show that the HCC still assumes that transitions
only occur at certain points in time, which makes the occupancy probability
constant between consecutive points.

The approximate state-occupancy probabilities are multiplied by the
cost of each state to compute the cost accrued in each interval. This
calculation makes sense when the cost is constant, but encounters
a problem when there is an abrupt discontinuity at the boundary between
two intervals. In the HIV example there is a discontinuity at $t=2\tau$,
so it is not clear which costs this method should use for the second
interval, which extends from $1.5\tau$ to $2.5\tau$ .

In practice, HCC is implemented using (the equivalent of) Equation~5,
whose first term on the right-hand side can be interpreted as the
cost accrued in a cycle of length $\tau/2$ in which no transition
has yet occurred. However, that equation stems from applying the trapezoidal
rule the \emph{instantaneous costs}, not to the occupancy probabilities.
This inconsistency was severely criticized by \citep{barendregt2009,barendregt2014},
who proposed the life-table method as a better alternative.

\subsection{The life-table method\label{subsec:LT}}

This method, being consistent with the arguments put forward by the
advocates of HCC, first estimates the average state-occupancy probabilities
inside each cycle by averaging the probabilities at its boundaries
and then calculates the costs:
\begin{equation}
\int_{i\tau}^{(i+1)\tau}c(t)\cdot dt\approx\sum_{s}\frac{P_{i}(s)+P_{i+1}(s)}{2}\cdot c_{i}(s)\cdot\gamma(i\tau)\cdot\tau\;,\label{eq:c-ith-LT}
\end{equation}
where $c_{i}(s)$ is the cost inside the $i$-th interval, again assumed
to be constant. This implies that
\begin{equation}
C_{LT}=\sum_{i=0}^{h-1}\sum_{s}\frac{P_{i}(s)+P_{i+1}(s)}{2}\cdot c_{i}(s)\cdot\gamma(i\tau)\cdot\tau\;,\label{eq:C-LT}
\end{equation}
This is called the \emph{life-table method} because it is based on
the procedure that demographers use it to estimate life expectancy
\citep{barendregt2009}.

It is easy to check that 
\[
C_{LT}=\frac{C_{E}+C_{B}}{2}\;.
\]
We have already seen that when the total-cost function decreases monotonically,
$C_{E}$ is an overestimation of the true cost and $C_{B}$ is an
underestimation, so their arithmetic mean, $C_{LT}$, is expected
to be closer to the true value than if we assumed that all transitions
occur either at the beginning or at the end of each cycle.

Please note that this method is insensitive to discontinuities that
occur at the boundary between intervals, which explains the small
error it returned for the HIV model—see Table~1.

\subsection{A comparison of HCC and LT}

The justification of HCC implicitly assumes that the instantaneous
cost for each state is time independent; this allows us to write $c(s)$
instead of $c(s,t)$. Under this assumption, HCC computes exactly
the same value as the trapezoidal rule, $C_{TR}$. Equation~5 stems
from Equation~4, which can be rewritten as

\begin{equation}
\int_{i\tau}^{(i+1)\tau}c(t)\cdot dt\approx\sum_{s}\frac{P_{i}(s)\cdot\gamma(i\tau)+P_{i+1}(s)\cdot\gamma((i+1)\tau)}{2\hspace{2.5em}}\cdot c(s)\cdot\tau\;.\label{eq:c-without-discontinuities-1}
\end{equation}
Comparing this expression with Equation~\ref{eq:c-ith-LT}, we can
see that they only differ in the way of applying the discounts.\footnote{Therefore, the assertion of \citep{elbasha2016a} that the trapezoidal
rule gives the same result as LT is true only when there is no discount.
In fact, Section~9.2.1 in \citep{gray2011} shows an example in which
the two methods yield different numerical results.} When the cycle length $\tau$ is short or the discount function decreases
slowly, then $\gamma(i\tau)\approx\gamma((i+1)\tau)$ and $C_{\text{\emph{LT}}}\approx C_{TR}$.
The main difference is that Equation~\ref{eq:c-without-discontinuities-1},
implicitly used by the trapezoidal rule and HCC, applies the correct
discount at each boundary of the cycle, while LT applies the same
discount, $\gamma(i\tau)$, from the beginning to the end of the cycle.

Therefore we disagree with \emph{some} of the arguments claiming the
superiority of LT over HCC\@.\footnote{In these remarks we agree with \citep{elbasha2016b,elbasha2016a},
except for minor details that do not deserve a discussion here.} For example, Barendregt~\citep{barendregt2009} argued that the
standard HCC method is incompatible with discounting. Naimark et al.~\citep{naimark2013},
who were initially strong advocates of the HCC, finally agreed with
him. However, the computation of $C_{\text{\emph{TR}}}$ clearly uses
the discounted costs (cf.\ Eqs.~1 and~5).

Barendregt \citep{barendregt2009} also said: ``I know of very few
relevant Markov models in medical decision making where QALY weights
and unit costs are constant across all cycles''. However, HCC does
not require that the cost function $c(t)$, given by Equation~1,
be constant. In our analysis, the derivation of Equation~5 only required
that $c(s,t)$ be constant for every state~$s$, as it is in many
models—the HIV examined in this paper is rather an exception.

In turn, \citep{naimark2013} said that ``the standard approach to
the HCC assumes that the state membership curve is declining and monotonic.''
However, when Equation~5 is justified as the application of the trapezoidal
rule, it does not require that assumption.

In summary, we claim that there was nothing wrong in the application
of the HCC in the absence of discontinuities. On the contrary, $C_{\text{\emph{TR}}}$,
given by Equation~5, is generally more accurate than $C_{\text{\emph{LT }}}$,
given by \ref{eq:C-LT}. The problem was in the way of explaining
and justifying the method.

\bibliographystyle{unsrt}
\newcommand{\earlyuaititle}[2]{Uncertainty in Artificial Intelligence #1
  (UAI'#2)} \newcommand{\ijcaititle}[2]{Proceedings of the #1 International
  Joint Conference on Artificial Intelligence (IJCAI'#2)}
  \newcommand{\uaititle}[2]{Proceedings of the #1 Conference on Uncertainty in
  Artificial Intelligence (UAI'#2)} \newcommand{\pgmtitle}[2]{Proceedings of
  the #1 European Workshop on Probabilistic Graphical Models (PGM'#2)}
  \newcommand{\ecsqarutitle}[2]{Proceedings of the #1 European Conference on
  Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU
  #2)} \newcommand{\aaititle}[3]{Proceedings of the #2 #1 Conference on
  Artificial Intelligence (AAAI'#3)}
  \newcommand{\naaititle}[2]{\aaititle{National}{#1}{#2}}
  \newcommand{\aaaititle}[2]{\aaititle{AAAI}{#1}{#2}}
  \newcommand{\bmawtitle}[2]{Proceedings of the #1 UAI Bayesian Modelling
  Applications Workshop (BMAW'#2)}
\begin{thebibliography}{1}

\bibitem{elbasha2016b}
E.~H. Elbasha and J.~Chhatwal.
\newblock Myths and misconceptions of within-cycle correction: a guide for
  modelers and decision makers.
\newblock {\em Pharmacoeconomics}, 34:13--22, 2016.

\bibitem{elbasha2016a}
E.~H. Elbasha and J.~Chhatwal.
\newblock Theoretical foundations and practical applications of within-cycle
  correction methods.
\newblock {\em Medical Decision Making}, 36:115--131, 2016.

\bibitem{sonnenberg1993}
F.~A. Sonnenberg and J.~R. Beck.
\newblock {M}arkov models in medical decision making: A practical guide.
\newblock {\em Medical Decision Making}, 13:322--339, 1993.

\bibitem{naimark2008}
D.~M.~J. Naimark, M.~Bott, and M.~Krahn.
\newblock The half-cycle correction explained: two alternative pedagogical
  approaches.
\newblock {\em Medical Decision Making}, 28:706--712, 2008.

\bibitem{barendregt2009}
J.~J. Barendregt.
\newblock The half-cycle correction: banish rather than explain it.
\newblock {\em Medical Decision Making}, 33:961--970, 2009.

\bibitem{barendregt2014}
J.~J. Barendregt.
\newblock The life table method of half-cycle correction: getting it right.
\newblock {\em Medical Decision Making}, 29:500--502, 2014.

\bibitem{gray2011}
A.~M. Gray, P.~M. Clarke, J.~Wolstenholme, and S.~Wordsworth.
\newblock {\em Applied Methods of Cost-effectiveness Analysis in Healthcare}.
\newblock Oxford University Press, New York, 2011.

\bibitem{naimark2013}
D.~M.~J. Naimark, N.~N. Kabboul, and M.~D. Krahn.
\newblock The half-cycle correction revisited: redemption of a kludge.
\newblock {\em Medical Decision Making}, 33:961--970, 2013.

\end{thebibliography}


\end{document}
