PC KA9Q NNTP Client                                CS 13-11-92
~~~~~~~~~~~~~~~~~~~
The nntp commands are:

nntp directory [<spool-directory> [<control-directory>]]

     Allows specification of the spool directory (where the
     news batches will be written) and the control directory
     (where the history, data, kill and get files are kept). 
     The control directory is discussed below.

     Default spool directory: /spool/mail
     Default control directory: /spool/news

nntp groups [<group-list>]

     Allows specification of group list (see discussion
     below).

     Default value: none

nntp addserver <server-name> <poll-interval>
                                [<time-window>] [<group-list>]

     Allows specification of news servers.  The poll-interval
     is the interval in seconds between polls of the news
     server (e.g., 1800 means poll the news server every 30
     minutes).  The time-window specifies a time window for
     polling.  The format is hour:min-hour:min (e.g.,
     18:00-20:00 means only poll between 6pm and 8pm local
     time).  The group-list is discussed below.

     Default value: none

nntp dropserver <server-name>

     Removes a news server from the polling list.

nntp kick <server-name>

     Forces a poll of the news server specified.  Has no
     effect if a poll is already being attempted.

nntp listservers

     Lists the current news servers, the times until the next
     poll and the polling intervals.

nntp batch [on|off [<buffers>]]

     When on, allows requests to be sent to the news server
     before the replies from previous requests have been
     received.  This makes things go faster.  The number of
     buffers determines the number of requests that may be
     outstanding.  Each buffer occupies around 1/2K of memory
     space.

     Default value: off, 2 buffers

nntp newgroups [on|off]

     When on, the news server is asked for a list of new
     groups created since the last successful poll.  The list
     is appended to the file NEWGROUP in the control
     directory.  No attempt is made to remove duplicate
     entries.

     Default value: off

nntp trace [<trace-level>]

     Allows specification of trace information to be sent to
     the screen.  The levels are defined as follows:

          0 - no tracing
          1 - serious errors reported
          2 - transient errors reported
          3 - session progress reported
          4 - actual received articles displayed

     All error messages are sent to the ka9q log file whatever
     the tracing level.

     Default value: 1

nntp safety [on|off]

     When on, output files are closed after each article is
     received.  This gives maximum protection if a machine
     crash occurs but adds extra disk overhead.

     Default value: on

nntp verbose [on|off]

     When on, the following progress messages may be sent to
     the screen:

          "Trying to connect to news server <server> ...."
          "Connected to news server"

          "News available: <n> articles"
          "News available: <n> new groups"
          "News arrived (<n>/<n>): <newsgroups>, article:<msgid>"
          "News duplicate: article <msgid>"
          "News unavailable: article <msgid>"

          "News summary: <n> articles (<n> bytes) in <n> sec
                                              (<n> bytes/sec)"
          "New articles: <n> duplicate, <n> unavailable,
                                    <n> headers, <n> complete"
          "Get articles: <n> invalid, <n> unavailable,
                                                 <n> received"
          "History file: <n> entries, <n> complete scans"
          "Throughput  : <n>/<n> bytes in <n>/<n> sec
                                          (<n>/<n> bytes/sec)"
          "Closing news session"

     When off, only the "News summary" message will be sent to
     the screen.  The session statistics are sent to the ka9q
     log file whatever the verbose setting.

     Default value: on


Control Directory
~~~~~~~~~~~~~~~~~
The following files may be created or read from the control
directory:

     NNTP.DAT

          Used to record the time of the last successful poll
          of each server.  Optionally used to specify the
          newsgroup list for each server.  The format is
          specified below.  Old server entries are not
          automatically removed from this file.

     HISTORY

          Used to hold the message ids of articles received. 
          This information is only required by the NNTP client
          to prevent duplicate articles from being received. 
          Old message ids are not automatically removed from
          this file - it will grow forever if not manually
          pruned.

     NEWGROUP

          Used to hold the names of new groups created on the
          servers.  No attempt is made to prevent duplicate
          names from being added.  Old group names are not
          automatically removed from this file.

     KILL

          Used to hold patterns with which received header
          lines are compared.  If certain patterns match then
          the corresponding article bodies will not be
          fetched.  The format is specified below.  If this
          file exists then all new articles will be fetched in
          two stages: header first then body.

     GET

          Used to hold message ids of articles that should be
          unconditionally fetched (i.e., not subject to the
          KILL file pattern matching).  The message ids of
          articles that are unavailable will not be removed
          from this file.  The file may therefore require
          periodic flushing.

     *.TMP, *.LCK

          Temporary files created during an nntp session. 
          These files should be automatically deleted.  The
          *.LCK files should be manually deleted if there are
          no active programs using files in the control
          directory.


Format of the NNTP.DAT file
~~~~~~~~~~~~~~~~~~~~~~~~~~~
There should be one line for each server.  This line will be
automatically written after a successful nntp session and need
not be created manually.  The line has the format:

     <server-name> <date> <time>

The first character of the server name must be in column one. 
<date> has the form YYMMDD and <time> has the form HHMMSS. 
The year is assumed to be in the closest century and the time
is GMT.  For example:

     news.demon.co.uk 920909 073954

The newsgroups to be received from each server may optionally
be specified on the lines following the line for the server. 
Each line containing a newsgroup specification must begin with
a whitespace character (e.g., space or tab).  One or more
newsgroup entries may be put on a line, separated by
whitespace or commas.  Multiple entries on one line will never
be split across multiple requests.  Blank lines in the middle
of the newsgroup list force the list to be split into multiple
requests.  Blank lines before and after the newsgroup list are
ignored.

Newsgroup entries may be complete names or may contain
asterisks ("*") to allow pattern matches.  In addition, an
exclamation mark ("!") as the first character will negate a
match allowing newsgroups matching certain patterns to be
ignored.  For example, "demon.*,!demon.test" will specify all
groups in the demon hierarchy except demon.test.

General guidance:

1    List all newsgroups one per line (except as noted below). 
     This will minimise the number of requests made.

2    Always list "omit" entries (entries beginning with an
     exclamation mark) on the same line as the entries to
     which they apply.

3    Don't separate the newsgroup list with blank lines. 
     However blank lines are ok before and after the list.

Example:

----------------------------------------
news.demon.co.uk 920909 073954

     demon.* !demon.test
     comp.binaries.eniac
     comp.sys.eniac
     comp.unix.eniac
     alt.humor
     rec.pets rec.pets.gerbils

----------------------------------------

Warning: be careful to leave the server line unchanged.  If
the server line is not recognised then all news available may
be requested.


Format of the KILL file
~~~~~~~~~~~~~~~~~~~~~~~
The kill file should contain one pattern per line.  Each
header line received is compared with every pattern until a
match occurs.  A match must be exact, with the first character
of the pattern matching the first character of the header line
and the last character of the pattern matching the last
character of the header line.  Case is insignificant and
whitespace is always represented by a single space character. 
The pattern may contain special characters as follows:

!    If the first character of a pattern is "!" then the rest
     of the line forms a "keep" pattern ("!" means don't
     kill).  If any header line matches a keep pattern then
     the article body will be fetched.

     If the first character of a pattern is not "!" then the
     whole line forms a "kill" pattern.  If any header line
     matches a kill pattern and no header line matches a keep
     pattern then the article body will not be fetched.

     The "!" character has no significance in any other
     position and will be matched literally.

?    Matches any single character.

*    Matches zero or more characters.

[ ]  Matches any single character that appears within the
     square brackets.  For example, [0123456789] will match
     any single decimal digit.  The characters can be
     specified as a range: [0-9] has the same meaning as the
     previous example.

[^ ] Matches any single character that does not appear within
     the square brackets.  For example, [^abc] will match any
     single character except "a", "b" and "c".

\    Matches the following character literally.  For example,
     \* will match a single asterisk and \\ will match a
     single backslash.

The kill file patterns are all loaded into memory at the start
of the news session - the kill file should therefore be kept
as small as possible.  Offline article selection can be
performed by creating a kill file containing the single
pattern "*".  This will cause only headers to be fetched.  The
message ids of the required articles can then be put in the
get file for retrieval during the next news session.

Example:

----------------------------------------
Lines: *[0-9][0-9][0-9]*
Newsgroups: *rec.pets*
!Newsgroups: *rec.pets.gerbils*
----------------------------------------

The first line should kill any messages with more than 99
lines.  Unfortunately some long messages do not contain a
"Lines:" line so this is not foolproof.

The second line should kill any postings or cross-postings to
the rec.pets hierarchy.

The third line will keep any postings to the rec.pets.gerbils
hierarchy regardless of any matches with the other kill
patterns.

Note that the header lines of killed articles are retained. 
The full posting can be fetched later by putting the message
id into the get file.


Specifying newsgroups
~~~~~~~~~~~~~~~~~~~~~
There are three methods of specifying the newsgroups to be
received.

If there is a newsgroup list associated with the server in the
NNTP.DAT file then that list will be used.  The list may be of
any length.

If there is not a recognisable group list in the NNTP.DAT file
then any groups specified with an NNTP ADDSERVER command will
be used.  There is a maximum length for a group list specified
in the NNTP ADDSERVER command of 488 characters (allowing one
comma between each entry).

If there is no group list in the NNTP.DAT file and there are
no groups specified with an NNTP ADDSERVER command then any
groups specified in an NNTP GROUPS command will be used.  The
maximum length of the group list here is also 488 characters.

If there are no groups specified anywhere then all groups will
be requested.

Note that the group list specified with an NNTP GROUPS command
replaces any existing list while the group list specified with
an NNTP ADDSERVER command is appended to any existing list.


Example settings for autoexec.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     nntp dir \spool\articles
     nntp groups demon.*,!demon.test
     nntp addserver news.demon.co.uk 1800
     nntp trace 2
     nntp batch on 4
     nntp safety off

The list of newsgroups required is included in the NNTP.DAT
file.  The NNTP GROUPS command is used as a back-up (if the
NNTP.DAT file becomes unreadable, only the demon groups will
be requested).
