This chapter provides an administration guide for managers of projects or systems which are using FCM.
Note that, where this section refers to the FCM team this applies only to Met Office users. External users will either need to refer to the equivalent team within their organisation or will need to perfom these tasks themselves.
The FCM system assumes that each project directory has sub-directories called trunk, branches and tags (Subversion recommends it but doesn't insist on it). We recommend that each project within a Subversion repository is in a sub-directory of the repository root.
<root>
|
|-- <project 1>
| |
| |-- trunk
| |-- branches
| |-- tags
|
|-- <project 2>
| |
| |-- trunk
| |-- branches
| |-- tags
|
|-- ...
In theory you could also have the project as the root directory or several
levels below the root directory. However, this is not tested and could cause
problems with some fcm commands so is best avoided.
You will need to decide whether to use a single project tree for your system or whether to use multiple projects.
Advantages of a single project tree:
Disadvantages of a single project tree:
checkout and
status can become frustratingly slow if your working copy is
too large.One common approach is to split the admin type files (things that only the system manager should need to touch) into a separate project from the core system files (which all the developers need access to). If you include any large data files under version control you may also want to use a separate project for them to avoid making your working copies very large when editing code.
Note that there is often no obvious right or wrong answer so you just have to make a decision and see how it works out. You can always re-arrange your repository in the future (although be aware that this will break any changes being prepared on branches at the time).
You also need to decide whether your system requires its own repository (or multiple repositories) or whether it can share with another system.
For simplicity, in most cases you will probably want your own repository for your system.
You will not normally want to have multiple repositories for a system. One exception may be if you are storing large data files where you might not want to keep all the old versions for ever. Removing old versions can't be done without changing all the revision numbers which would mess up all your code history and Trac tickets. Storing the large data files in a separate repository reduces the impact if you do decide to remove old versions in the future. One disadvantage of this approach is that, for the moment at least, Trac only handles one repository so you will need a separate Trac system for the data files.
For further details please see the section Planning Your Repository Organization from the Subversion book.
Normally the FCM team will help you to set up your initial repository.
However, it is quite simple if you need to do it yourself. First you need to
issue the command svnadmin create /path/to/repos. This creates
an empty repository which is now ready to accept an initial import. To do so,
you should create a directory tree in a suitable location, and issue the
svn import command. At the top level of your directory tree
should be the project directories. Each project should then contain three
directories trunk, branches and tags.
The directories branches and tags should be empty.
The directory trunk should contain your source files in a
directory structure you have chosen. For example, if your directory tree is
located at $HOME/foo, you will do the following to import it to
a new repository:
(SHELL PROMPT)$ svnadmin create FOO_svn (SHELL PROMPT)$ svn import $HOME/foo file://$PWD/FOO_svn -m 'Initial import' Adding FOO Adding FOO/trunk Adding FOO/trunk/doc Adding FOO/trunk/doc/hello.html Adding FOO/trunk/doc/world.html Adding FOO/trunk/src Adding FOO/trunk/src/bar Adding FOO/trunk/src/bar/egg.f90 Adding FOO/trunk/src/bar/ham.f90 Adding FOO/branches Adding FOO/tags Committed revision 1.
Note that the svnadmin command takes a PATH as an
argument, as opposed to a URL for the svn command.
For further details please see the section Planning Your Repository Organization from the Subversion book.
You will not normally want to allow anonymous write access to your repository since this means that changes do not get identified with a userid. Therefore system managers need to provide the FCM team with a list of users who should have write access to their repositories. You may also want to arrange passwords for these users although this is only necessary if you need to prevent malicious access. Further restrictions such as preventing anonymous read access or restricting write access to the trunk to a limited set of users can be arranged if necessary.
The FCM team will organise the hosting of your repository. A number of facilities will be set up for you as standard.
svnserve (which we use in preference to
Apache for performance reasons). The FCM team will advise you
of the URL.svnadmin verify
command.Additional hook scripts can be put in place if you have a requirement. The use of hook scripts is discussed in the section Repository Creation and Configuration from the Subversion book.
Note that if you want to use a Subversion repository for your own
individual use there is no need to get the FCM team to host it. You can
simply create your repository and then use a file:// URL to
access it.
You can set up a watch facility to monitor changes in revision log
messages in your repository. An obvious use of this facility is for system
managers to monitor changes of log messages affecting the trunks of their
projects. To set up the facility, you will need to add a
watch.cfg file to the root of your repository. (To avoid
checking out the whole repository, including every branch, make sure that you
checkout the root of your repository non-recursively, i.e. fcm checkout
-N URL.) The watch.cfg file is an INI-type file, with the
basic format:
[repos_base] path/in/repos = list,of,watchers
For example, if your repository is svn://fcm1/FCM_svn/, and you want set up particular users to monitor changes of the log messages affecting several areas within the repository, you may have something like this:
[FCM_svn] FCM/trunk/src = fred,bob FCM/trunk/doc = fred,bob,alice FCM/branches/dev/*/*/src = fred
Later on, if dave attempts to modify the log message of a changeset that affects the path FCM/trunk/src, fred and bob will both be notified by e-mail.
Normally the FCM team will set up your Trac system for you (using a script which helps automate the configuration). This section describes some things you may wish to be configured. This can be done when the Trac system is set up or later if you are unsure what you will require at first.
You will not normally want to allow anonymous users to make changes to your Trac system since this means that changes may not get identified with a userid. The FCM team will normally set up your Trac system such that any authenticated users can make changes. Further restrictions such as restricting write access to named accounts or preventing anonymous read access can be arranged if necessary.
The system manager will normally be given TRAC_ADMIN privileges. This allows them to do additional things which normal users cannot do such as:
For further details please see the section Trac Permissions from the Trac documentation.
By default, each Trac system is configured such that the owner and reporter and anyone on the CC list are notified whenever a change is made to a ticket. If system mangers wish to be notified of all ticket changes then this can also be configured. Alternatively, email notifications can be disabled if they are not wanted.
Milestones are useful for identifying when tickets need to be resolved.
Typically, milestones may be particular system releases or time periods. The
FCM team can configure milestones for you when they set up your Trac system.
However, this is not strictly necessary since milestones can be set up via
the web interface using the admin account (go to the Roadmap
page).
There are lots of other things that can be configured in your Trac system such as:
For further details please see the sections The Trac Configuration File and The Trac Ticket System from the Trac documentation.
The FCM team will organise the hosting of your Trac system. It will be set up on the same server that hosts your Subversion repository and access will be provided via a web server. The FCM team will advise you of the URL. Each night a backup of your Trac system will be taken in case the live system should become corrupted for whatever reason.
When you set up a repository for a new project, you will normally want the FCM team to set up a URL keyword for it in the FCM central configuration file. The name of the project should be a short string containing only word characters.
Individual projects can store revision keywords using the Subversion
property fcm:revision at registered URLs. Using the UM as an
example: if UM is a registered URL keyword, you can add the
fcm:revision property at the head of the UM project by doing a
non-recursive checkout. E.g.:
(prompt)$ fcm co -q -N fcm:um um (prompt)$ fcm pe fcm:revision um
In the editor, add the following and fcm commit:
VN6.3 = 402 VN6.4 = 1396 VN6.5 = 2599 VN6.6 = 4913 VN7.0 = 6163
In a subsequent invocation of fcm, if a revision keyword is
specified for a URL in the UM namespace, the command will attempt to load it
from the fcm:revision property at the head of the UM project.
Revision keywords can also be defined in the FCM central configuration file
if you prefer.
If the project has an associated Trac browser, you can also declare browser URL mapping in the central configuration file. This allows FCM to associate the Subversion URL with a Trac browser URL. There is an automatic default for mapping URLs hosted by the FCM team at the Met Office. External users of FCM may want to adjust this default for their site.
To change the default browser URL mapping, you need to make 3
SET::URL_BROWSER_MAPPING_DEFAULT::<key> declarations in
your $FCM/etc/fcm.cfg file. There are 3 keys to this
declaration: LOCATION_COMPONENT_PATTERN,
BROWSER_URL_TEMPLATE and BROWSER_REV_TEMPLATE. The
LOCATION_COMPONENT_PATTERN is a Perl regular expression, which is
used to separate the scheme-specific part of a version control system URL
into a number of components by capturing its substrings. These components are
then used to fill in the numbered fields in the
BROWSER_URL_TEMPLATE. The template should have one more field than
the number of components captured by LOCATION_COMPONENT_PATTERN.
The last field is used to place the revision, which is generated via the
BROWSER_REV_TEMPLATE. This template should have a single numbered
field for filling in the revision number. This is best demonstrated by an
example. Consider the declarations:
%pattern ^//([^/]+)/(.*)$
%url_template http://{1}/intertrac/source:{2}{3}
%rev_template @{1}
set::url_browser_mapping_default::location_component_pattern %pattern
set::url_browser_mapping_default::browser_url_template %url_template
set::url_browser_mapping_default::browser_rev_template %rev_template
If we have a Subversion URL svn://repos/path/to/a/file, the
LOCATION_COMPONENT_PATTERN will capture the components
[repos, path/to/a/file]. When this is applied to
the BROWSER_URL_TEMPLATE, {1} will be translated to
repos and {2} will be translated to
path/to/a/file. A revision is not given in this case, and so
{3} is inserted with an empty string. The result is
http://repos/intertrac/path/to/a/file. If the revision is
1357, the BROWSER_REV_TEMPLATE will be used to
translate it to @1357, which is then inserted to {3}
of the BROWSER_URL_TEMPLATE. The result is therefore
http://repos/intertrac/path/to/a/file@1357.
For more information on how to set up the keywords, please refer to Repository & Revision Keywords and the Annex: Declarations in FCM central/user configuration file.
The extract and build systems are very flexibile and can be used in lots of different ways. It is therefore difficult to give specific advice explaining how to configure them. However, based on experience with a number of systems, the following general advice can be offered.
Probably the best advice is to look at what has already been set up for other systems. The FCM team can advise on the best systems to examine.
When you create a stable build you should keep an extract configuration
file that can reproduce the build. One easy way to do this is to create your
build using the standard configuration files and the latest versions of the
code. You can then save the expanded extract configuration file which is
created when you run the extract. To help document your stable build you can
use the command fcm cmp-ext-cfg to show what has changed.
Sometimes it is useful to be able to access particular revisions of some
directories from a FCM repository without having to go via Subversion.
Typical examples are namelist or data files used as inputs to a program. The
script fcm_update_version_dir.pl is designed to help with this.
It can be used to maintain a set of extracted version directories from a FCM
repository. The script has the following options and arguments:
-f [--full]-d [--dest] arg-u [--url] argIf an argument is specified, it must be the location of a configuration file. Otherwise, the command reads its configuration from the standard input. The configuration file is a line-based text file. Blank lines and lines beginning with a # are ignored.
Each configuration line must contain the relative path of a sub-directory under the specified source repository URL. If the path ends in * then the path is expanded recursively and any sub-directories containing regular files are added to the list of relative paths to extract.
Optionally, each relative path may be followed by a list of space
separated conditions. Each condition is a conditional operator
(>, >=, <,
<=, == or !=) followed by a
revision number or the keyword HEAD.
The command uses the revision log to determine the revisions at which the relative path has been updated in the source repository URL. If these revisions also satisfy the conditions set by the user, they will be considered in the extraction. In full mode, everything is re-extracted. In incremental mode, the version directories are only updated if they do not already exist.
Example:
(SHELL PROMPT)$ fcm_update_version_dir.pl -u fcm:ver_tr <<EOF namelists/VerNL_AreaDefinition >1000 !=1234 namelists/VerNL_GRIBToPPCode >=600 <3000 namelists/VerNL_StationList elements/* >1000 EOF
N.B.
Some options on working practises and policies are defined in the chapter on Code Management Working Practices. Individual projects should document the approach they have adopted. In addition, each project may also need to define its own working practices and policies to suit its local need. For example each project may need to specify: