Changes between Version 2 and Version 3 of WorkingGroups/TAM/ReferenceManual/NemoTamBasics

Timestamp:

2010-02-22T16:40:05+01:00 (14 years ago)

Author:

avidard

Comment:

--

WorkingGroups/TAM/ReferenceManual/NemoTamBasics

@@ +1 @@
+= TAM what for ? =
+The development of tangent and adjoint models is
+an important step in addressing sensitivity analysis and variational
+data assimilation problems in Oceanography. Sensitivity analysis is the study of how model output varies with changes in model inputs. The sensitivity information
+given by the adjoint model is used directly to gain an understanding
+of the physical processes. In data assimilation, one considers a cost
+function which is a measure of the model-data misfit. The adjoint
+sensitivities are used to build the gradient for descent algorithms.
+Similarly the tangent model is used in the context
+of the incremental algorithms to linearize the
+cost function around a background control.
 
-Describe here the main principles of TAM : 
+= Tangent and Adjoint coding techniques = 
+the original program {{{P}}},
+whatever its size and run time, computes a function
+[[Image(ad_1)]]
+which is the composition of the elementary functions
+computed by each run-time instruction. In other words if
+{{{P}}} executes a sequence of elementary statements
+[[Image(ad_2.png)]], then {{{P}}} actually evaluates
+[[Image(ad_3.png)]]
+where each ''f,,k,,'' is the function implemented by ''I,,k,,''.
+Therefore one can apply the chain rule of derivative
+calculus
+to get the Jacobian matrix ''F!''', i.e. the partial
+derivatives of each component of ''Y'' with respect to
+each component of ''X''. Calling ''X,,0,,=X'' and
+''X,,k,,=f,,k,,(X,,{k-1},,)'' the successive values of all
+intermediate variables, i.e. the successive '''states''' of
+the memory throughout execution of {{{P}}}, we get
 
- * what tangent and adjoint models are
+[[Image(ad_4.png)]]
+
+The derivatives ''f!',,k,,''
+of each elementary instruction are easily built, and must
+be inserted in the differentiated program so
+that each of them has the values ''X,,k-1,,'' directly available
+for use.
+This process yields analytic derivatives,
+that are exact up to numerical accuracy.
+
+In practice, two sorts of derivatives are of
+particular importance in scientific computing: the
+tangent (or directional) derivatives, and the
+adjoint (or reverse) derivatives.
+
+The tangent derivative is the product
+$\dot{Y} = F'(X) \times \dot{X}$ of the full Jacobian times
+a direction $\dot{X}$ in the input space.
+>From equation above, we find
+
+[[Image(ad_5)]]
+
+which is most cheaply executed from right to left
+because matrix {{{x}}} vector products are much cheaper
+than matrix {{{x}}} matrix products.
+This is also the most convenient execution order because
+it uses the intermediate values ''X,,k,,'' in the same order
+as the program {{{P}}} builds them.
+On the other hand the adjoint derivative is the product
+[[Image(ad_6)]] of
+the ''transposed'' Jacobian times a weight vector
+$\overline{Y}$ in the output space. The
+resulting $\overline{X}$ is the gradient of the
+dot product $(Y \cdot \overline{Y})$.
+>From equation (\ref{eqchainrule}), we find
+\begin{equation}\label{eqadjmode}
+\overline{X} = F'^{*}(X) \times \overline{Y} =
+f'^{*}_1(X_0) \times \dots \times f'^{*}_{p-1}(X_{p-2})
+\times f'^{*}_p(X_{p-1}) \times \overline{Y}
+\end{equation}
+which is also most cheaply executed from right to left.
+However, this uses the intermediate values $X_k$ in the
+inverse of their building order in {\tt P}.
+
+= Potential issues = 
+
  * the major approximations : non differentiability issues, simplification of the direct model before linearization,  validity of the tangent linear hypothesis over some time window, etc.
  * the validation interface : classical tests for checking the correctness of tangent and adjoint models