  Updated for version 3.8.5 of the OPeNDAP Server Software.

This software is used by two projects: The original OPeNDAP Data
Server, which uses CGI 1.1 in conjunction with a web server such as
Apache, and the new data server (Hyrax) which uses Java Servlets in
combination with the OPeNDAP Back-End Server (BES). This package
contains general purpose handlers for use with either of the two
servers. These are the Usage, ASCII and HTML form handlers. Each takes
input from a 'data handler' and returns a HTML or plain text response
--- something other than a DAP response object.

-------------------------------------------------------------------------

I. Hyrax BES module

II. CGI OPeNDAP Data Server

-------------------------------------------------------------------------
I. Hyrax BES module

  For Hyrax this package includes the library modules that can be
  dynamically loaded into the OPeNDAP Back-End Server (BES) as well as
  help files for the modules. To load these modules into the BES
  simply edit the BES configuration file, its default location in
  /etc/bes/bes.conf. Directions for editing the configuration file
  follow, however, you can run the bes-dap-data.sh script to edit the
  script automatically. Or, if building from source you can use the
  bes-conf target of make to run the script for you.

  To edit the bes.conf script by hand:

  Set the BES user, group and admin email (see the comments in the
  bes.conf file for instructions).

  Change the parameter BES.modules to include the three dap-server
  modules ascii, usage and www as follows (example assumes you have
  installed the netcdf_handler module):

<code>
    BES.modules=dap,cmds,ascii,usage,www,nc
</code>

  And add the following three lines below this:

<code>
    BES.module.ascii=/usr/lib/bes/libascii_module.so
    BES.module.usage=/usr/lib/bes/libusage_module.so
    BES.module.www=/usr/lib/bes/libwww_module.so
</code>

  Also add the following near where the bes_help files are specified.

<code>
    DAP-SERVER.Help.TXT=/usr/share/bes/dap-server_help.txt
    DAP-SERVER.Help.HTML=/usr/share/bes/dap-server_help.html
    DAP-SERVER.Help.XML=/usr/share/bes/dap-server_help.html
</code>

  The next time the BES is started these modules will be loaded and
  the ascii, info_page, and html_form responses will be supported as
  well as the help files for dap-server specifying the syntax of these
  commands.

-------------------------------------------------------------------------
II. CGI OPeNDAP Data Server

PREFACE

  NOTE

      This part of the README is specifically for the CGI version of
      the OPeNDAP Data Server, which is being replaced by a server
      which uses the ESG/OPeNDAP data server back end (aka BES) along
      with a front-end component that uses Java Servlets. This new
      server is name 'Hyrax'.  See opendap.org for more information.
      
      This release is an update of the 3.7.4 release with the security
      patch from 27 April 2007 applied with other fixes applied in
      addition to those in the patch. If you are trying to serve files
      with odd characters in their names and have trouble using this
      server, please contact support at 'support at opendap.org'.

  This is a general guide for the OPeNDAP data servers. There is a
  more detailed guide available online at http://opendap.org/support
  at the link 'Server Installation Guide.'

  The server software is split between two functions; this part of the
  software provides the general capabilities of the server such as
  basic request handling. The server also uses one or more 'handlers'
  for each specific data format. Those handlers are available as
  separate items from our web site or from other sites.

  Version 3.5 and later of the server, described in this file, differs
  in some important ways from previous versions of the server. In
  version 3.5 only one executable file is located in a web daemon's
  CGI binary directory.  In addition, it's no longer necessary to
  locate the data in your web daemon's DocumentRoot directory. Both of
  these changes increase server robustness.

  Though not absolutely essential, it will be useful to understand the
  structure of the software. You can find general descriptions of the
  system and how the pieces fit together in the User's Guide,
  available from the OPeNDAP home page: http://www.opendap.org/.

  This file describes how to use the software once it has been built
  and installed. For instructions on the build and/or installation
  process, see the file INSTALL.

  See NEWS for recent changes to the server software.

  -----------------------------------------------------------------------

<code>
  CONTENTS

  Introduction

  About WWW Servers

  Setting Up a Data Server
  Quick Instructions
  Serving Compressed Files 
  More Sophisticated Installation
    Usage information
    Logging accesses

  About Data Security
  Using A Secure OPeNDAP Server
  Configuring a (Secure) Server

  Server Installation Tips
    Tip 1: Which directory is the cgi-bin directory?
    Tip 2: Which directory is used by your WWW server as the document
           root?
    Tip 3: How can I avoid copying all my data into the document root?
    Tip 4: How do I write a URL for some archive's data?
    Tip 5: How do I add additional descriptive information about my
           dataset?
    Tip 6: How do I set up a server that limits access to only a few
           people?
    Tip 7: Is there a registry of datasets?
    Tip 8: Customizing the DODS Directory response (from Joe McLean)
    Tip 9: Secure servers
    Tip 10: Shorten URLs
</code>

  -----------------------------------------------------------------------

  Introduction

    This is the base software for the OPeNDAP data server. It uses the
    CGI 1.1 protocol and requires Apache or a similar web daemon.

    The software here includes three C++ programs which are installed
    in <prefix>/bin and a collection of Perl software install in
    <prefix>/share/dap-server. Finally, the Perl script nph-dods and
    it's configuration file dap-server.rc are installed in
    <prefix>/share/dap-server-cgi. These last two files are to be
    installed in one of the HTTP daemon's CGI binaries directory (you
    can use the default cgi-bin or set up a new one).
  
    We've added a sample configuration file which can be used with the
    Apache 2.x web server. Look at 'opendap_apache.conf.'

  -----------------------------------------------------------------------

  About WWW Servers

    We have tested the servers extensively with the Apache HTTP daemon
    (httpd).  Other servers will work as along as they support CGI 1.1
    standard. However, the installation instructions will be slightly
    different. Don't let this dissuade you from installing the servers
    with another httpd - the process is very simple. Other groups have
    used the NCSA, CERN and Netscape web servers successfully.

    If you do not have a World Wide Web server running on the machine
    you want to use as your data server machine, the Apache daemon can
    be found at http://www.apache.org/.

  -----------------------------------------------------------------------

  Setting Up a Data Server

  Quick Instructions

    First build and install the Server Base Software. See the file
    INSTALL. You may be able to install a binary package and skip the
    build step.  Once the Base has been installed, you're ready to
    configure the software and add one or more data handlers. The base
    software does not include any handlers (each is available as a
    separate package) so we'll use the NetCDF File handler as an
    example.

    To configure the server you can copy the files nph-dods and
    dap-server.rc from the directory $prefix/share/dap-server-cgi to a
    CGI Binary directory for your web daemon. Or, you can edit your
    web server's configuration file and make that directory a CGI bin
    directory. See SERVER INSTALLATION TIPS section below for help on
    how to locate your web server's CGI Binary diretcory. Make sure
    that the nph-dods script is exectuable. Also see the
    opendap_apache.conf. This can be used in lieu of editing your
    httpd.conf file in many cases, should you want to run your OPeNDAP
    server in its own CGI bin directory.

    Previous versions of the OPeNDAP server required that data either
    be stored in your web daemon's DocumentRoot, one of its
    subdirectories or be symbolically linked so that it appeared so
    (and in the later case a special option had to be set for the
    Apache web daemon). However, the version 3.5 server provides an
    option where the data location can be set using the data_root
    parameter of the dap_server.rc file. See below under "The
    dap_server.rc Configuration File" for details. Note that the old
    behavior where data are located in or under DocumentRoot is still
    supported. Also note that the OPeNDAP server's directory browsing
    functions don't currently work when using data_root.

  The dap-server.rc Configuration File

    The dap-server.rc configuration file is used to tailor the server
    to your site. The file contains a handful of parameters plus the
    mappings between different data sources (typically files, although
    that doesn't have to be the case) and hander-programs. The format
    of the configuration files is:
  
<code>
       <parameter> <value> [ <value> ... ]
</code>

    The configuration file is line-oriented, with each parameter
    appearing on its own line. Blank lines are ignored and the '#'
    character is used to begin comment lines. Comments have to appear
    on lines by themselves (you cannot tack a comment on to the end of
    a line that contains a parameter). 

    The parameters recognized are:

<code>
    data_root <path>
</code>

    Define this if you do not want the server to assume data are
    located under DocumentRoot. The value <path> should be the fully
    qualified path to the directory which you want to use as the root
    of your data tree. For example, we have a collection of netcdf,
    hdf, et c., files that we store in directories named
    /usr/local/test_data/data/nc, /usr/local/test_data/data/hdf, et
    c., and we set data_root to '/usr/local/test_data'. The value of
    <path> should _not_ end in a slash.

    Notes for 'data_root': 

      The OPeNDAP server's directory browsing functions do not work
      when using the data_root option. They do still work when
      locating data under DocumentRoot.

      It's not an absolute requirement that this path be a real
      directory or that your data are in files. Data can reside in a
      relational database, for example. In that case the base software
      will use <path> as a prefix to the path part of the URL it
      receives from the client.).

<code>
    timeout <seconds>
</code>

    This sets the OPeNDAP server timeout value, in seconds. This is
    different from the httpd timeout. OPeNDAP servers run
    independently of httpd once the initial work of httpd is complete.
    This ensures that your OPeNDAP server does not continue
    indefinitely if something goes wrong (i.e., a user makes a huge
    request to a database). Default is 0 which means no time out.

<code>
    cache_dir <directory>
</code>

    When data files are stored in a compressed format such as gzip or
    UNIX compress, the OPeNDAP server first decompresses them and then
    serves the decompressed file. The files are cached as they are
    decompressed. This parameter tells the server where to put that
    cache. Default: /usr/tmp. In the RPM package, this defaults to
    /var/cache/dap-server.

<code>
    cache_size <size in MB>
</code>

    How much space can the cached files occupy? This value is given in
    mega bytes. When the total size of all the decompressed files
    exceeds this value, the oldest remaining file will be removed
    until the size drops below the parameter value. If you are serving
    large files, make sure this value is at least as large as the
    largest file.

<code>
    maintainer <email address>
</code>

    The email address of the person responsible for this server. This
    email address will be included in many error messages returned by
    the server.  Default: support@unidata.ucar.edu.

<code>
    curl <path>
</code>

    This parameter is used to set the path to the curl executable. The
    curl command line tool is used to dereference URLs when the server
    needs to do so. In some cases the curl exectuable might not be
    found be the server when checks default directories. This can be a
    source of considerable confusion because the CGI program run from
    a web daemon used a very restricted PATH environment variable,
    much more restricted than a typical user's PATH.  Thus, even if
    you, as teh server installer have curl on your PATH, nph-dods may
    not be able to find the program unless you tell it exactly where
    to look.

<code>
    exclude <handler> [ <handler> ... ]
</code>

    This is a list of handlers whose regular expressions should *not*
    be used when building the HTML form interface for this server. In
    general, this list should be empty. However, if you have a handler
    that is bound to a regular expression that is very general (such
    as .* which will match all files), then you should list that
    handler here, enclosing the name in double quotes. See the next
    item about the 'handler' parameter. Default: by default, no
    handlers are excluded.

<code>
    handler <regular expression> <handler name>
</code>

    The handler parameter is used to match data sources with
    particular handler programs used by the server. In a typical
    OPeNDAP server setup, the data sources are files and the regular
    expressions choose handlers based on the data file's extension.
    However, this need not be that case. The OPeNDAP server actually
    matches the entire pathname (starting at the HTTP Document Root)
    of the data source when searching for the correct handler to use.
    The regular expressions are Perl regular expressions; the examples
    in the sample dap-server.rc file should get you going.

    Here are the default values for 'handler:'

<code>
    handler .*\.(HDF|hdf|EOS|eos)(.Z|.gz|bz2)*$ /usr/local/bin/dap_hdf_handler
    handler .*\.(NC|nc|cdf|CDF)(.Z|.gz|bz2)*$ /usr/local/bin/dap_nc_handler
    handler .*\.(dat|bin)$ /usr/local/bin/dap_ff_handler
</code>

    "Regular expressions", advanced pattern-matching languages, are a
    powerful feature of Perl and many other computer programs.
    Powerful enough, in fact, to warrant at least one book about them
    (Mastering Regular Expressions by Jeffrey Friedl, O'Reilly, 1997).
    For a complete reference online--not a particularly good place to
    learn about them for the first time--see
    http://www.perldoc.com/perl5.6/pod/perlre.html.

    Briefly, however, the above patterns test whether a filename is of
    the form 'file.ext.comp', where 'comp' (if present) is Z or gz,
    and 'ext' is one of several possible filename extensions that
    might indicate a specific storage format. file.

    If these default rules will not work for your installation, you
    can rewrite them. For example, if all your files are HDF files,
    you could replace the default configuration file with one that
    looks like this:

<code>
    handler .* dap_hdf_handler
</code>

    The .* pattern matches all possible names, and indicates that
    whatever the name of the file sought, the HDF handler is the one
    to use.

    If you have a situation where all the files in a particular
    directory (whatever its extension) are to be handled by the DSP
    service programs, and all other files served are JGOFS files, try
    this:

<code>
    handler \/dsp_data\/.* dap_dsp_handler
    handler .* dap_jg_handler
</code>

    The rules are applied in order, and the first rule with a
    successful match returns the handler that will be applied. The
    above set of rules implies that everything in the dsp_data
    directory will be handled with the DSP service programs, and
    everything else will be handled with the JGOFS programs.

  Serving Compressed Files 

    Here is some important configuration information for people
    serving data from compressed files. If your server is not handling
    compressed files correctly, changing the options as described here
    will likely fix the problem.

    When the OPeNDAP software is used to serve compressed files (e.g.,
    files compressed using gzip), the files are first decompressed and
    then stored in a cache; data served are read from those cached
    files. The location of the cache directory is /usr/tmp by default.
    This can be changed by editing dap-server.rc and changing the
    value of cache_dir. The software is set by default to limit the
    size of this directory to 500 MB. However, if you're serving large
    files, or are experiencing a large volume of traffic, you should
    increase this. To do so, edit the value of of the cache_size, also
    in dap-server.rc. The cache size is given in MB, so changing the
    500 to 1000 would increase the cache size from 500MB to 1GB.

    Finally, the decompression software uses the gzip program to do
    its work.  If your computer does not have the gzip program in its
    /bin directory you'll need to edit the DODS_Cache.pm so that the
    correct version of gzip is used. Look in that file for "/bin/gzip"
    and replace that text with the correct pathname. To figure out
    where gzip is on you computer, type 'which gzip' in a shell. As of
    version 3.5.2 the server will also decompress files compressed
    using bzip2. If the bizip2 utility cannot be found by the
    DOSD_Cache.pm module, look for /bin/bzip2 and replace that with
    the correct pathname for your server host.

  Logging accesses

    Since an HTTP daemon logs all accesses, every access to data
    through your OPeNDAP server is logged. However, to collect
    information about project-wide data use, that information needs to
    be collated. We have built an optional module that scans your
    daemon's logs for just the information about OPeNDAP activity.
    This feature is disabled by default.

    This optional module provides for remote access to this
    OPeNDAP-only information using the URL 'http://<your host>/<your
    server>/stats'. The information can be accessed by your computer
    and some computers at Unidata, URI and OPeNDAP, using tools like a
    browser or wget.

    You can configure the remote stat access module by hand fairly
    easily. In the distributions etc directory, edit the file called
    EXAMPLE_OPENDAP_STATISTICS. This file holds the pathname to you
    server's access and error logs as well as its name. Once you have
    edited the file, save is as 'OPENDAP_STATISTICS' and copy that
    file to the directory that contains the OPeNDAP server (i.e.,
    nph-dods).

    If you are using virtual hosts with the web deamon that runs the
    OPeNDAP server, the remote stats access module can only be
    configured to provide access to one set of log files.

  ABOUT DATA SECURITY

    There are two levels of security which the OPeNDAP server
    supports: Domain restrictions and user restrictions. In
    conjunction with a World Wide Web server, access to the OPeNDAP
    server can be limited to a specific group of users (authenticated
    by password), specific machine(s) or a group of machines within a
    given domain or domains.

  Notes

    DAP 3.4 onward includes support for Digest authentication, which
    significantly increases the robustness of password access.

    DAP versions 3.2 and greater software contains significant
    improvements in the way password authentication is handled. Older
    versions of the DODS clients prompted for the password with each
    and every interaction between client and server. Now credentials
    may be embedded in URLs and are remembered and reused for the
    duration of a session.

  The security features of the OPeNDAP server depends heavily on the
  underlying WWW daemon because we felt this was the best way to solve
  the thorny problem of ensuring only authorized users accessed data.
  By using the daemon's authorization software we are ensuring that
  the security checks used by the server have been tested by many many
  sites. In addition, WWW daemons already support a very full set of
  security features and many system administrators are comfortable and
  confidant with them. The tradeoff with using the web daemon's
  security system for our servers is that two security settings must
  be made for each group of data to be configured and more than one
  copy of the nph-dods CGI program and dap-server.rc configuration
  file may be needed even if you're serving only one type of data.

  Because the security features rely almost entirely on the host
  machine's WWW server, the steps required to install a secure the
  server will vary depending on the WWW server used. Thus, before
  installing a secure server, check over your WWW server's
  documentation to make sure it provides the following security
  features: Access limits to files in the document root on a per user
  and/or per machine basis, and; The ability to place CGI scripts.

  IMPORTANT

    Because security features are used to protect sensitive or
    otherwise important information, once set-up they should be tested
    until you are comfortable that they work. You should try accessing
    from at least one machine that is not allowed to access your data.
    If you would like, we will try to access your data, assuming that
    our machines are among those not allowed, to help you evaluate
    your set-up.

    Since the security features are provided by a WWW server, it is
    highly likely that they are functional and extensively tested.
    While problems with these features have shown up in the past
    (e.g., the Netscape SSL server bug) they are generally fixed
    quickly. Thus there is good reason to assume that your data are
    safe if you choose to set-up your server as a secure one. However,
    *there is a chance* that a defect in the WWW server software will
    allow unauthorized people access; how big that chance is depends
    on the WWW server software you use and how extensively its
    security features are tested. That level of testing is completely
    beyond our control.

  It is important to distinguish securing a server from securing data.
  If data are served, then those data may also be accessible through a
  web browser (see the note about the new data_root configuration
  parameter). If so the data themselves need to be stored in
  directories that have limited access. If all data access will take
  place through the server this limitation can exclude all access
  *except* the local machine. This is the case because some the
  server's directory function requires being able to read the data
  through the local host's web server.

  It bears repeating: If you're serving sensitive information with the
  OPeNDAP server and those data are located under the WWW daemon's
  DocumentRoot, that information is accessible two ways: via the
  OPeNDAP server and through the WWW server. You need to make sure
  *both* are protected.

  About the 'data_root' parameter

    With version 3.5 of the OPeNDAP server, you can use the
    'data_root' parameter in the dap-server.rc configuration file to
    serve data that are *not* accessible using your WWW server. This
    simplifies securing the data but has the drawback that the
    directory response is not supported by the OPeNDAP server.

  About serving both limited- and open-access data from the same server

    In the past it was possible to install two or more OPeNDAP servers
    on a computer and assign different protections to each one.
    However, in practice this has proven to be very hard for to
    configure correctly. In many cases where this feature was used, a
    secure server was setup up for one group of data while an open
    server was set up for another. It was often the case that all the
    data were accessible using the open server! Thus, if you need to
    secure some data, it is best to host all the sensitive information
    on one machine and put other data on a second machine with an
    open-access server. If you *must* run two or more servers from the
    same physical host, we suggest that you configure your web server
    to see two (or more) virtual hosts. This will provide the needed
    separation between the groups of data.

  USING A SECURE OPeNDAP SERVER

    Using a secure sever is transparent if the server is configured to
    allow access based on hosts or domains. Give the URL to a client;
    the server will respond by answering the request if allowed or
    with an error message if access is not allowed.

    Accessing a server which requires password authentication is a
    little different and varies depending on the type of client being
    used. All OPeNDAP clients support passing the authentication
    information along with the URL. To do this add
    `<username>:<password>@' before the machine name in a URL. For
    example, suppose I have a secure server set up on
    `test.opendap.org' and the user `guest' is allowed access with the
    password `demo'. A URL for that server would start out:

<code>
    http://guest:demo@test.opendap.org/...
</code>

    For example,

<code>
    http://guest:demo@test.opendap.org/secure/nph-dods/sdata/nc/fnoc1.nc.info
</code>

    will return the info on the data set fnoc1.nc from a secure
    server. You cannot access the data without including the username
    and password `guest' and `demo'.

    Some clients will pop up a dialog box and prompt for the username
    and password. Netscape, and some other web browsers, for example,
    will do this.  Similarly, some DODS clients may also popup a
    dialog.

  CONFIGURING A SERVER

    In the following I'll use the Apache 1.3.12 server as an example
    (also tested on Apache 2.0.40, 07/25/03 jhrg) and describe how to
    install a server which limits access to a set of users. While this
    example is limited to the Apache server, it should be simple to
    perform the equivalent steps for any other WWW server that
    supports the set of required security features (See ABOUT DATA
    SECURITY).

  Note: 
    I have installed a secured server using dap-server 3.5 and Apache
    2.0.54 and have found that installing the dap-server.rc and
    nph-dods files in their own directory, making that a ScriptAlias
    directory and using the directives shown below works. That is,
    there's no need with Apache 2.x to place the CGI under the
    document root. Two addition bits of information regarding this
    option: 1. The 'Options' directive is not needed since the
    directory is named as a ScriptAlias; and 2. There's no need to
    user the '.cgi' extension on the nph-dods executable.

    I also have found that using the ScriptAliasMatch directive can be
    used to shorten the URL quite a bit. I included the following in
    httpd.conf:

<code>
      ScriptAliasMatch ^/secure(.*) "/usr/local/secure-3.5/nph-dods$1
</code>

    so that a url like http://test.opendap.org/secure/data/nc/fnoc1.nc
    will access the fnoc1.nc dataset using the secured server.

  1. Create a directory for the server.

    To limit access to a  dataset to particular machine, begin by
    creating a special directory for the server. This maybe either an
    additional CGI bin directory or a directory within the web
    server's document root. In this example, I chose the latter.

<code>
    cd /var/www/html/
    mkdir secure
</code>

  2. Establish access limitations for that directory.

    Establish the access limitations for this directory. For the
    Apache server, this is done either by adding lines to the server's
    httpd.conf file or by using a per-directory file. Note: The use of
    per-directory access limit files is a configurable feature of the
    Apache server; look in the server's httpd.conf file for the value
    of the AccessFileName resource.

    I modified Apache's httpd.conf file so that it contains the
    following:

<code>
    # Only valid users can use the server in secure. 7/6/2000 jhrg
    <Directory /var/www/html/secure>
	Options ExecCGI Indexes FollowSymLinks

	Order deny,allow
	Deny from all
	# ALLOW SERVER (IP OF SERVER) MACHINE TO REQUEST DATA ITSELF
	Allow from __YOUR_SERVER_HERE__ 
	Require valid-user
	# ALL VISITORS NEED USERNAME AND PASS BUT NOT SERVER
	Satisfy any

	AuthType Basic 
	AuthUserFile /etc/httpd/conf/htpasswd.users 
	AuthGroupFile /etc/httpd/conf/htpasswd.groups
	AuthName "Secure server"
    </Directory>

    # Protect the directory used to hold the secure data.
    <Directory /var/www/html/sdata>
	Options Indexes

	Order deny,allow
	Deny from all
	# ALLOW SERVER (IP OF SERVER) MACHINE TO REQUEST DATA ITSELF
	Allow from __YOUR_SERVER_HERE__ 
	Require valid-user
	# ALL VISITORS NEED USERNAME AND PASS BUT NOT SERVER
	Satisfy any

	AuthType Basic 
	AuthUserFile /etc/httpd/conf/htpasswd.users 
	AuthGroupFile /etc/httpd/conf/htpasswd.groups
	AuthName "Secure data"
    </Directory>
</code>

    and

<code>
    ScriptAlias /secure/ "/var/www/html/secure/"
</code>

    The first group of lines establishes the options allowed for the
    `secure' directory, including that it can contain CGI programs.
    The lines following that establish that only users in the Apache
    password file can access the contents of the directory, with the
    exception that this server is allowed to access the directory
    without authentication. This last bit is important because OPeNDAP
    servers sometimes make requests to themselves (e.g., when
    generating the directory response) but don't pass on the
    authentication information.*

    Regarding the 'Satisfy any" directive, Brock Murch says: 

      I thought that one needed an "Allow from all" since I want my
      users to connect from anywhere, which would have necessitated a
      "satisfy all" since I needed the passwd authentication as well.
      I didn't know that the "Deny from all" would still allow anyone
      in so long as the AuthType etc was included and authentication
      took place. Since this is the case a "satisfy any" will do as I
      have denied all ip access except for the server itself. The
      second group of lines secure the data itself from accesses which
      bypass the DODS server.

    The ScriptAlias line tells Apache that executable files in the
    directory are CGIs. You can also do this by renaming the nph-dods
    script to nph-dods.cgi and making sure httpd.conf contains the
    line:

<code>
    AddHandler cgi-script .cgi
</code>

    The AuthType directive selects the type of authentication used.
    Apache 2.0 supports 'Basic' and 'Digest' while other servers may
    also support GSS-Negotiate and NTLM. Version 3.4 onward of the DAP
    software supports all these authentication schemes, although only
    Basic and Digest have been thoroughly tested. Configuration of
    Apache 2.0 for Digest authentication is slightly different then
    for Basic authentication, but is explained well in Apache's on
    line documentation.

  3. Copy the server into the new directory.

    Copy the CGI program (nph-dods) and the server configuration file
    to the newly created directory. Note that if you're using the
    extension `.cgi' to tell Apache that nph-dods is a CGI you must
    rename nph-dods to nph-dods.cgi.  If you forget to do that then
    you will get a Not Found (404) error from the server and debugging
    information generated by the DODS server won't appear in Apache's
    error_log even if it has been turned on.

  Server Installation Tips

    The following tips are mostly specific to the Apache server. For
    other web servers, this advice may be only of general, or possibly
    little, help.

    Tip 1: Which directory is the cgi-bin directory?

      Look in the server's `httpd.conf' file for a line like:

<code>
      ScriptAlias /cgi-bin/ /var/www/cgi-bin/
</code>

      The option ScriptAlias defines where CGI programs may reside. In
      this case they are in the directory /var/www/cgi-bin/. With this
      script alias in effect, URLs with `cgi-bin' in their path will
      automatically refer to programs in the specified directory. You
      might want to put the OPeNDAP server in its own CGI directory
      named 'opendap' or 'dap', although that's not a requirement.

    Tip 2: Which directory is used by your WWW server as the document
           root?

      Look in the http.conf file for a line like:

<code>
        DocumentRoot /var/www/html
</code>

      This says that the document root directory is /var/www/html.
      URLs that contain only the machine name and a file name, such
      as:
  
<code>
        http://test.opendap.org/info.txt
</code>

      refer to files in the document root directory.  For example,
      with the above DocumentRoot declaration, and the above URL, the
      file specified is:

<code>
        /var/www/html/info.txt
</code>

    Tip 3: How can I avoid copying all my data into the document root?
           Can't I link to it?

      This tip is old and, while still true, the 3.5 server provides a
      better solution for some sites: Use the data_root configuration
      parameter in dap-server.rc. See the section on the Configuration
      File for more information...

      By default many servers disable soft (symbolic) links that point
      out of the directories that fall under the web server's document
      root. This is not done for inconvenience, it is a security
      feature (really). To disable this feature, and allow links to
      other directories, you'll have to edit the httpd.conf file, and
      add the `FollowSymLinks' option to the directory trees which
      will contain the links to data. This might sound more complex
      than it is.

      For example, suppose that you have a large dataset stored in a
      directory named `/archive1'. Under that directory are many files
      and taken together they are far too large to copy into the WWW
      server's document root. First look in access.conf and see that
      FollowSymLinks is set for the WWW server's document root. It
      will look like:

<code>
        <Directory /usr/local/spool/http>
        Options Indexes FollowSymLinks
</code>

      There may be some blank lines and comments between the two.  If
      `FollowSymLinks' is not on the `Options' line, add it, just like
      in the example above. Once that is done, create a symbolic link
      from within your WWW server's document root to `/archive1' (use
      `ln -s'). You're done.

    Tip 4: How do I write a URL for some archive's data?

      One question about your server you will likely be asked is "How
      do I write one of those URLs for your data?". Here's how to
      answer that question. The URLs have the form:

<code>
        http://<machine name>/<cgi dir name>/<pathname>
</code>

      Suppose that you have copied the HDF server to /var/www/opendap
      and soft linked a set of files in /var/www/html/data/hdf on the
      machine `test.opendap.org'. Also suppose a partial directory
      listing of that is:

<code>
        S2000415.HDF
        S1700101.HDF.gz
        S3096277.HDF.Z
</code>

      A URL that references the first file would be:
  
<code>
        http://test.opendap.org/opendap/nph-dods/data/hdf/S2000415.HDF
</code>

      The part of the URL that reads `opendap/nph-dods' selects the
      server and the section `data/hdf/S2000415.HDF' chooses the file.
      Note that the WWW server's document root is /var/www/html/ and
      the files are stored in a subdirectory within that named
      `data/hdf'. On the Web URLs, are rooted at the DocumentRoot
      directory so the `/var/www/html/' part of the pathname is
      implicit and should not be part of the URL. Since the directory
      `data/hdf' is under the document root directory it *is* included
      in the URL.

    Tip 5: How do I add additional descriptive information about my
           dataset?

      The best way is to use the `usage' service.  This service
      returns information to the client about the server, but it can
      also return an HTML document you write to describe your server
      or any dataset served.

      To describe any special features of a particular data server,
      write an HTML document describing that server, and put it in a
      file named <root>.html in the cgi-bin directory that holds the
      server programs. The only special thing about this file is that
      you should include only those HTML tags that would fall between
      the <body> and </body> tags. Thus it should not contain the
      <html> <head> <body> or their matching counterparts.

      To provide HTML for a class of files you'd create an HTML file
      whose name is based on the names of the data files you want to
      describe. For example, a file that would be used for all the HDF
      files used in the previous section's example would be S.html.
      This file should be located in the directory where the data is
      located.  You could also create an S20004.html and an
      S20005.html to distinguish between the different sets of HDF
      files.

      Users access this information by appending `.html' to a URL. For
      example to get the HTML page for the URL used in the previous
      section, you'd type:

<code>
        http://test.opendap.org/opendap/nph-hdf/data/hdf/S2000415.HDF.html
</code>

      Note: the Usage server will return important information about
      your data set even if you do not write custom HTML files for it.
      IF you do write those files they will be concatenated with the
      default information returned by the usage server.

    Tip 6: How do I set up a server that limits access to only a few
           people?

      Briefly, it is fairly easy in practice to set up servers so that
      only a particular group can access data using them. The specific
      mechanisms you have at your disposal will depend on the WWW
      server you use. See the file README-security (found in the
      DODS/etc directory) for information on installing
      `secure'servers with Apache.

    Tip 7: Is there a registry of datasets?

      You can register your data set(s) with the us using our web
      page. Goto http://www.opendap.org/data/addtolist.html. Our list
      of available datasets can be found under the 'support' heading
      of our web page by following the 'Sources of data' link in the
      left-hand panel. There's also a link to the list of data sources
      on the front-page left-hand panel.

      You can also register your dataset with the Global Change Master
      Directory (http://gcmd.gsfc.nasa.gov/); they maintain a list
      registered DODS/NVODS/OPeNDAP data sets.

      The GCMD has a special portal dedicated to datasets accessible
      using OPeNDAP servers at
      http://gcmd.gsfc.nasa.gov/Data/portals/dods/.

    Tip 8: Customizing the Directory response (from Joe McLean)

      After messing around with Apache for a few hours, I have found
      some configuration directives that may be of use in beautifying
      the appearance of a directory on your user's Browser.

      example: http://ferret.pmel.noaa.gov/cgi-bin/nph-dods/data/PMEL/

      In your Apache httpd.conf file add the following:

<code>
      <Directory /path-to-dods(without cgi-bin or nph-dods)/data>
          Options +Indexes +FollowSymLinks
          IndexOptions FancyIndexing
          IndexOptions +FoldersFirst +NameWidth=* +IgnoreCase
          IndexOptions +SuppressDescription
          DefaultIcon /path-to-icons/dods.gif
          IndexOptions +IconHeight=20 +IconWidth=20
      </Directory>
</code>

      I copied the dods logo from upper left hand corner of
      http://www.unidata.ucar.edu/packages/dods/home/getStarted/
      called it dods.gif and stuck it in my apache icons directory (in
      httpd.conf grep for Icon - something should show you the path)

      Information for Apache

<code>
      IndexOptions: http://httpd.apache.org/docs/mod/mod_autoindex.html#indexoptions

      Options: http://httpd.apache.org/docs/mod/core.html#options

      DefaultIcon: http://httpd.apache.org/docs/mod/mod_autoindex.html#defaulticon
</code>

    Tip 9: Secure servers

      Using the per-directory limit files makes changing limits easier
      since the server reads those every time it accesses the
      directory, while changes made to the httpd.conf file are not
      read until the server is restarted or sent the HUP signal.
      However, using httpd.conf for your security configuration seems
      more straightforward since all the information is in one place.

      If the protections are set up so that it is impossible for the
      server host to access the data and/or the OPeNDAP server itself,
      then an infinite loop can result. This can be frustrating to
      debug, but if you see that accesses generate an endless series
      of entries in the access_log file, it is likely that is the
      problem. Make sure that you have `allow from <server host name>'
      set for both the directory that holds the OPeNDAP server and
      that holds the data. Also make sure that the server's name is
      set to the full name of the host.

      Configuring a secure server can be frustrating if you're testing
      the server using a web browser that remembers passwords. You can
      turn this feature off in some browers. Also, the getdap tool
      supplied with OPeNDAP's libdap (which is required to build the
      server) can be useful to test the server since it will not
      remember passwords between runs.

    Tip 10: Shorten URLs

      Use the ScriptAliasMatch directive in Apache 2.x to shorten
      URLs. Suppose you have a server installed in
      /usr/local/opendap-3.5/share/dap-server-cgi/. The following:

<code>
        ScriptAliasMatch ^/dap(.*) "/usr/local/opendap-3.5/share/dap-server-cgi/nph-dods$1
</code>
 
      will cause a URL like:

      http://test.opendap.org/dap/data/nc/fnoc1.nc
      
      to work where with just the ScriptAlias 

<code>
        ScriptAlias /opendap-3.5/ "/usr/local/opendap-3.5/share/dap-server-cgi/"
</code>

      users would need the more cumbersome:

<code>
        http://test.opendap.org/opendap-3.5/nph-dods/data/nc/fnoc1.nc      
</code>

  More Questions...

    There are several sources of further information about
    DODS/NVODS/OPeNDAP:

      o The OPeNDAP home page: http://www.opendap.org/.

      o The users manual, available in HTML and PDF, on the home page.

      o User support by email: support@unidata.ucar.edu

  Thanks

  * Brock Murch <bmurch@marine.usf.edu> worked out some thorny
    configuration details for securing the Apache/DODS/OPeNDAP
    combination.
    
