Index: branches/2011/dev_NEMO_MERGE_2011/DOC/TexFiles/Chapters/Chap_OBS.tex
===================================================================
--- branches/2011/dev_NEMO_MERGE_2011/DOC/TexFiles/Chapters/Chap_OBS.tex	(revision 3115)
+++ branches/2011/dev_NEMO_MERGE_2011/DOC/TexFiles/Chapters/Chap_OBS.tex	(revision 3116)
@@ -13,39 +13,37 @@
 $\ $\newline    % force a new line
 
-The observation and model comparison code (OBS) reads in observation files
-(profile temperature and salinity, sea surface temperature, sea level anomaly,
-sea ice concentration, and velocity) and calculates  an interpolated model equivalent
-value at the observation location and nearest model timestep. The OBS code is
-called from \np{opa.F90} in order to initialise the model and to calculate the
-model equivalent values for observations on the 0th timestep. The code is then
-called again after each timestep from \np{step.F90}. The code was originally
-developed for use with NEMOVAR. 
-
-For all data types a 2D horizontal  interpolator is needed
-to interpolate the model fields to the observation location.
-For {\em in situ} profiles, a 1D vertical interpolator is needed in addition to
-provide model fields at the observation depths. Currently this only works in
-z-level model configurations, but is being developed to work with a
-generalised vertical coordinate system.
-Temperature data from moored buoys (TAO, TRITON, PIRATA) in the
-ENACT/ENSEMBLES data-base are available as daily averaged quantities. For this
-type of observation the
-observation operator will compare such observations to the model temperature
+The observation and model comparison code (OBS) reads in observation files (profile
+temperature and salinity, sea surface temperature, sea level anomaly, sea ice concentration,
+and velocity) and calculates  an interpolated model equivalent value at the observation
+location and nearest model timestep. The resulting data are saved in a ``feedback'' file (or
+files). The code was originally developed for use with the NEMOVAR data assimilation code, but
+can be used for validation or verification of model or  any other data assimilation system.
+
+The OBS code is called from \np{opa.F90} for model initialisation and to calculate the model
+equivalent values for observations on the 0th timestep. The code is then called again after
+each timestep from \np{step.F90}. To build with the OBS code active \key{diaobs} must be
+set.
+
+For all data types a 2D horizontal  interpolator is needed to interpolate the model fields to
+the observation location. For {\em in situ} profiles, a 1D vertical interpolator is needed in
+addition to provide model fields at the observation depths. Currently this only works in
+z-level model configurations, but is being developed to work with a generalised vertical
+coordinate system. Temperature data from moored buoys (TAO, TRITON, PIRATA) in the
+ENACT/ENSEMBLES data-base are available as daily averaged quantities. For this type of
+observation the observation operator will compare such observations to the model temperature
 fields averaged over one day. The relevant observation type may be specified in the namelist
-using \np{endailyavtypes}. Otherwise the model value from the nearest
-timestep to the observation time is used.
-
-The resulting data are saved in a ``feedback'' file (or files) which can be used
-for model validation and verification and also to provide information for data
-assimilation. This code is controlled by the namelist \textit{nam\_obs}. To
-build with the OBS code active \key{diaobs} must be set.
-
-Section~\ref{OBS_example} introduces a test example of the observation operator
-code including where to obtain data and how to setup the namelist.
-Section~\ref{OBS_details} introduces some more technical details of the
-different observation types used and also shows a more complete namelist.
-Finally section~\ref{OBS_theory} introduces some of the theoretical aspects of
-the observation operator including interpolation methods and running on multiple
-processors. 
+using \np{endailyavtypes}. Otherwise the model value from the nearest timestep to the
+observation time is used.
+
+The code is controlled by the namelist \textit{nam\_obs}. See the following sections for more
+details on setting up the namelist. 
+
+Section~\ref{OBS_example} introduces a test example of the observation operator code including
+where to obtain data and how to setup the namelist. Section~\ref{OBS_details} introduces some
+more technical details of the different observation types used and also shows a more complete
+namelist. Section~\ref{OBS_theory} introduces some of the theoretical aspects of the
+observation operator including interpolation methods and running on multiple processors.
+Section~\ref{OBS_obsutils} introduces some utilities to help working with the files produced
+by the OBS code.
 
 % ================================================================
@@ -69,5 +67,5 @@
 \item Add the following to the NEMO namelist to run the observation
 operator on this data. Set the \np{enactfiles} namelist parameter to the
-observation  file name (or link in to \np{profiles\_01\.nc}):
+observation  file name:
 \end{enumerate}
 
@@ -76,5 +74,5 @@
 %-------------------------------------------------------------------------------------------------------------
 
-The option \np{ln\_t3d} and \np{ln\_s3d} switch on the temperature and salinity
+The options \np{ln\_t3d} and \np{ln\_s3d} switch on the temperature and salinity
 profile observation operator code. The \np{ln\_ena} switch turns on the reading
 of ENACT/ENSEMBLES type profile data. The filename or array of filenames are
@@ -88,8 +86,8 @@
 Setting \np{ln\_grid\_global} means that the code distributes the observations
 evenly between processors. Alternatively each processor will work with
-observations located within the model subdomain.
-
-The NEMOVAR system contains utilities to plot the feedback files, convert and
-recombine the files. These are available on request from the NEMOVAR team.
+observations located within the model subdomain (see section~\ref{OBS_parallel}).
+
+A number of utilities are now provided to plot the feedback files, convert and
+recombine the files. These are explained in more detail in section~\ref{OBS_obsutils}.
 
 \section{Technical details}
@@ -713,4 +711,5 @@
 
 \subsection{Parallel aspects of horizontal interpolation}
+\label{OBS_parallel}
 
 For horizontal interpolation, there is the basic problem that the
@@ -779,7 +778,245 @@
 \subsection{Vertical interpolation operator}
 
-The vertical interpolation is achieved using either a cubic spline or
+Vertical interpolation is achieved using either a cubic spline or
 linear interpolation. For the cubic spline, the top and
 bottom boundary conditions for the second derivative of the 
 interpolating polynomial in the spline are set to zero.
 At the bottom boundary, this is done using the land-ocean mask.
+
+\newpage
+
+\section{Observation Utilities}
+\label{OBS_obsutils}
+
+Some tools for viewing and processing of observation and feedback files are provided in the
+NEMO repository for convenience. These include OBSTOOLS which are a collection of Fortran
+programs which are helpful to deal with feedback files. They do such tasks as observation file
+conversion, printing of file contents, some basic statistical analysis of feedback files. The
+other tool is an IDL program called dataplot which uses a graphical interface to visualise
+observations and feedback files. OBSTOOLS and dataplot are described in more detail below.  
+
+\subsection{Obstools}
+
+A series of Fortran utilities is provided with NEMO called OBSTOOLS. This are helpful in
+handling observation files and the feedback file output from the NEMO observation operator.
+The utilities are as follows
+
+\subsubsection{corio2fb}
+
+The program corio2fb converts profile observation files from the Coriolis format to the
+standard feedback format. The program is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+corio2fb.exe outputfile inputfile1 inputfile2 ...
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{enact2fb}
+
+The program enact2fb converts profile observation files from the ENACT format to the standard
+feedback format. The program is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+enact2fb.exe outputfile inputfile1 inputfile2 ...
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{fbcomb}
+
+The program fbcomb combines multiple feedback files produced by individual processors in an
+MPI run of NEMO into a single feedback file. The program is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbcomb.exe outputfile inputfile1 inputfile2 ...
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{fbmatchup}
+
+The program fbmatchup will match observations from two feedback files. The program is called
+in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbmatchup.exe outputfile inputfile1 varname1 inputfile2 varname2 ...
+\end{verbatim}
+\end{alltt}
+
+
+\subsubsection{fbprint}
+
+The program fbprint will print the contents of a feedback file or files to standard output.
+Selected information can be output using optional arguments. The program is called in the
+following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbprint.exe [options] inputfile
+
+options:
+     -b            shorter output
+     -q            Select observations based on QC flags
+     -Q            Select observations based on QC flags
+     -B            Select observations based on QC flags
+     -u            unsorted
+     -s ID         select station ID  
+     -t TYPE       select observation type
+     -v NUM1-NUM2  select variable range to print by number 
+                      (default all)
+     -a NUM1-NUM2  select additional variable range to print by number 
+                      (default all)
+     -e NUM1-NUM2  select extra variable range to print by number 
+                      (default all)
+     -d            output date range
+     -D            print depths
+     -z            use zipped files
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{fbsel}
+
+The program fbsel will select or subsample observations. The program is called in the
+following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbsel.exe <input filename> <output filename>
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{fbstat}
+
+The program fbstat will output summary statistics in different global areas into a number of
+files. The program is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbstat.exe [-nmlev] <filenames>
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{fbthin}
+
+The program fbthin will thin the data to 1 degree resolution. The code could easily be
+modified to thin to a different resolution. The program is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+fbthin.exe inputfile outputfile
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{sla2fb}
+
+The program sla2fb will convert an AVISO SLA format file to feedback format. The program is
+called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+sla2fb.exe [-s type] outputfile inputfile1 inputfile2 ...
+
+Option:
+     -s            Select altimeter data_source
+\end{verbatim}
+\end{alltt}
+
+\subsubsection{vel2fb}
+
+The program vel2fb will convert TAO/PIRATA/RAMA currents files to feedback format. The program
+is called in the following way:
+
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+vel2fb.exe outputfile inputfile1 inputfile2 ...
+\end{verbatim}
+\end{alltt}
+
+\subsection{building the obstools}
+
+To build the obstools use in the tools directory use ./maketools -n OBSTOOLS -m [ARCH].
+
+\subsection{Dataplot}
+
+An IDL program called dataplot is included which uses a graphical interface to visualise
+observations and feedback files. It is possible to zoom in, plot individual profiles and
+calculate some basic statistics. To plot some data run IDL and then:
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+IDL> dataplot, "filename"
+\end{verbatim}
+\end{alltt}
+
+To read multiple files into dataplot, for example multiple feedback files from different
+processors or from different days, the easiest method is to use the spawn command to generate
+a list of files which can then be passed to dataplot.
+\begin{alltt}
+\footnotesize
+\begin{verbatim}
+IDL> spawn, 'ls profb*.nc', files
+IDL> dataplot, files
+\end{verbatim}
+\end{alltt}
+
+Fig~\ref{fig:obsdataplotmain} shows the main window which is launched when dataplot starts.
+This is split into three parts. At the top there is a menu bar which contains a variety of
+drop down menus. Areas - zooms into prespecified regions; plot - plots the data as a
+timeseries or a T-S diagram if appropriate; Find - allows data to be searched; Config - sets
+various configuration options.
+
+The middle part is a plot of the geographical location of the observations. This will plot the
+observation value, the model background value or observation minus background value depending
+on the option selected in the radio button at the bottom of the window. The plotting colour
+range can be changed by clicking on the colour bar. The title of the plot gives some basic
+information about the date range and depth range shown, the extreme values, and the mean and
+rms values. It is possible to zoom in using a drag-box. You may also zoom in or out using the
+mouse wheel.
+
+The bottom part of the window controls what is visible in the plot above. There are two bars
+which select the level range plotted (for profile data). The other bars below select the date
+range shown. The bottom of the figure allows the option to plot the mean, root mean square,
+standard deviation or mean square values. As mentioned above you can choose to plot the
+observation value, the model background value or observation minus background value. The next
+group of radio buttons selects the map projection. This can either be regular latitude
+longitude grid, or north or south polar stereographic. The next group of radio buttons will
+plot bad observations, switch to salinity and plot density for profile observations. The
+rightmost group of buttons will print the plot window as a postscript, save it as png, or exit
+from dataplot.
+
+%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+\begin{figure}     \begin{center}
+%\includegraphics[width=10cm,height=12cm,angle=-90.]{./TexFiles/Figures/Fig_OBS_dataplot_main}
+\includegraphics[width=9cm,angle=-90.]{./TexFiles/Figures/Fig_OBS_dataplot_main}
+\caption{      \label{fig:obsdataplotmain}
+Main window of dataplot.}
+\end{center}     \end{figure}
+%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+
+If a profile point is clicked with the mouse button a plot of the observation and background
+values as a function of depth (Fig~\ref{fig:obsdataplotprofile}).
+
+%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+\begin{figure}     \begin{center}
+%\includegraphics[width=10cm,height=12cm,angle=-90.]{./TexFiles/Figures/Fig_OBS_dataplot_prof}
+\includegraphics[width=7cm,angle=-90.]{./TexFiles/Figures/Fig_OBS_dataplot_prof}
+\caption{      \label{fig:obsdataplotprofile}
+Profile plot from dataplot produced by right clicking on a point in the main window.}
+\end{center}     \end{figure}
+%>>>>>>>>>>>>>>>>>>>>>>>>>>>>
+
+
+
+
