NewsLink is an external module for Synchronet BBS Software v3.1+. It is used to link one or more Synchronet sub-boards (aka message areas) with one or more USENET (or other NNTP network) newsgroups on one or more News Servers.

News Servers use the Network News Transfer Protocol (NNTP) to transfer messages to and from a News Client (aka newsreader). Popular News Clients include Microsoft Outlook Express, Netscape Communicator, Mozilla, Pine, Xnews, and many others. NewsLink emulates a News Client based on the NNTP standard. News Servers normally listen for incoming connections on TCP port 119 and usually require an authenticated user login.

NewsLink is written in 100% JavaScript and the source code (if you're interested) is contained in the file newslink.js in the Synchronet exec directory. Current version is always available via git.

If you have installed Synchronet v3.10 or v3.11 from scratch (not an upgrade from an earlier version), then some of the following steps may have already been completed for you.

1. Go into SCFG->External Programs->Timed Events and created a timed event for NewsLink (if one doesn't already exist). Example:

 [NEWSLINK Timed Event]
  1: Internal Code                   NEWSLINK
  2: Start-up Directory
  3: Command Line                    ?newslink
  4: Enabled                         Yes
  5: Execution Node                  1
  6: Execution Days of Month         Any
  7: Execution Days of Week          All
  8: Execution Frequency             48 times a day
  9: Requires Exclusive Execution    No
 10: Force Users Off-line For Event  No
 11: Native (32-bit) Executable      <doesn't matter>
 12: Use Shell to Execute            <doesn't matter>
 13: Background Execution            <doesn't matter>
 14: Always Run After Init/Re-init   No

You may change the Execution Frequency to suit your needs.

2. Go into SCFG->Message Areas and create a new Message Group for your USENET

 messsage areas (assuming you're linking to a USENET News Server). Example:

 [USENET Group]
 1: Long Name                  USENET Newsgroups
 2: Short Name                 USENET
 3: Internal Code Prefix       USENET_
 4: Access Requirements

Note: At this point you may want to login to your News Server with a regular News Client (aka newsreader), retrieve a list of available newsgroups and make a list of the newsgroups you would like linked into your BBS's message areas. An alternate option is to use the provided exec/getnewsgrouplist.js script to retrieve a list of newsgroups from your News Server and save those to a file (e.g. newsgroup.lst). Example:

$ jsexec getnewsgrouplist.js user password news.example.com > newsgroup.lst

(Note: the getnewsgrouplist.js script supports alternate ports and group name filtering options, if you need them)

3. Now go into SCFG->Message Areas->USENET->Sub-boards and create some sub-boards to link to newsgroups on the News Server. Example:

 [alt.bbs.allsysop Sub-board]
  1: Long Name                  alt.bbs.allsysop
  2: Short Name                 alt.bbs.allsysop
  3: QWK Name                   allsysop
  4: Internal Code              USENET_ALLSYSOP
  5: Newsgroup Name             alt.bbs.allsysop
  6: Access Requirements
  7: Reading Requirements
  8: Posting Requirements       REST NOT G
  9: Operator Requirements
 10: Moderated Posting User
 11: Maximum Messages           5000
 12: Purge by Age               Disabled
 13: Duplicate Checking         Enabled (10000 message CRCs)
 

An alternate option here would be to import the newsgroup.lst file if you have one, using the SCFG “Import Message Areas” option and the “USENET Newsgroup List” file format.

4. For each NewsLinked sub-board, confirm Network Options->Internet (UUCP/NNTP) is set to “Yes”.

5. Edit your ctrl/newslink.cfg file:

• Change the “SERVER” line to specify the host name or IP address of the News Server you are linking with. If the News Server is listening on a non-standard TCP port (e.g. other than 119), you will need to add a “PORT” line specifying the TCP port number of the server.
• If you wish to use TLS/SSL, uncomment the “TLS” line. This will change the default port to 563.
• If the News Server requires an authenticated user login, change the “USER” and “PASS” lines to specify the username and password you use to login to the above News Server. If the “USER” line is missing or commented-out (with a semicolon), then no user authentication will be attempted (sometimes called “anonymous login”).
• For each sub-board you configured in step 3, add an “AREA” line, specifying each sub-board's “Internal Code” (from SCFG->Message Areas).

Example: AREA USENET_ALLSYSOP
(An alternate option here is to use the AUTO_AREAS keyword which will automatically link all sub-boards flagged as Internet/NNTP networked in SCFG)

• By default, NewsLink will only import articles (aka messages) posted *after* the first scan of the newsgroups (i.e. it will not import *any* articles during the first run). If you want NewsLink to import *all* existing articles for *all* linked newsgroups (potentially many *thousands* of messages, depending on the News Server), add the line “FLAGS I” (case insensitive). WARNING: Using this feature may cause the initial NewsLink execution to take a long time to download all the existing articles (potentially many thousands) that may be stored on the News Server.
• Remove or comment-out (with a semicolon) on the DISABLED line (if present).

6. Now you may force the NEWSLINK timed event to run. In SBBSCTRL (Win32 GUI), you can do this by clicking BBS->Force Timed Event->NEWSLINK, or for the console or NT services versions of Synchronet, you can touch or create the file data/newslink.now to force the event to run.

7. Watch your events log for the NewsLink execution progress and details. Remember, if the 'I' flag was not specified (globally or per-newsgroup) in your ctrl/newslink.cfg, then no articles (aka messages) will be imported during the first run and only newly-posted articles will be imported on subsequent runs.

Each “AREA” line in the NewsLink configuration file can specify one or more “flag characters” to enable special behaviors. These same flag characters may be specified globally (for all linked areas) using the “FLAGS” line.

The supported flags characters are:

x   do not add tearlines & taglines to exported messages
n   do not add "From Newsgroup" text to imported messages
t   do not add tearline to imported messages
a   convert extended-ASCII chars to ASCII on imported messages
r   remove "Newsgroups:" header field from imported messages
b   decode single-part (uuencoded or yenc-encoded) binary attachments
i   import all articles on first run (not just new articles)
s   no subject filtering (do not check against text/subject.can)

To specify one or more flag characters for a specific area, you must use the following “AREA” line syntax:

  AREA <sub-code> <newsgroup> <flags> [attachment_dir]

where:

• <sub-code> is the sub-board's complete internal code (including the group's code prefix, if there is one).
• <newsgroup> is the name of the newsgroup as it is represented on the News Server. If this field is missing or is less than 2 characters in length, the newsgroup name from SCFG is used.
• <flags> is the optional list of flag characters.
• [attachment_dir] is the optional destination directory for decoded binary file attachments (only used with the 'b' flag)

Examples:

  AREA USENET_ALLSYSOP alt.bbs.allsysop nib /sysop_attachments
  AREA USENET_ALLSYSOP - asr

ctrl/newslink.cfg is just the default NewsLink configuration file name. If you specify the path to a different file on the ?newslink command-line, NewsLink will use that configuration file instead. This allows you to have multiple NewsLink timed events configured with different configurations for different News Servers.

Other supported NewsLink command-line options:

Most SPAM messages on public NNTP networks (e.g. USENET) are directed at multiple newsgroups. For this reason, the newslink.cfg file supports a max_newsgroups_per_article value which defaults to 5.

If you find NewsLink is filtering valid messages with an error in the log like:

!Too many newsgroups (7)

try increasing this value.

If you find NewsLink is allowing SPAM through that is addressed to multiple newsgroups, try reducing this value. Set max_newsgroups_per_article to 0 to disable this feature.

NewsLink also uses the text/subject.can filter file to filter (ignore) articles with specific text in the subject and the ctrl/twitlist.cfg file to ignore messages from specific users or domains.

NewsLink will yield time-slices based on the lines_per_yield (default: 5) and yield_length (default: 1) values.

If you find NewsLink is consuming a large portion of your CPU cycles, try decreasing the lines_per_yield value or increasing the yield_length value.

NewsLink can be stopped by “touching” (creating or updating) the semaphore file: data/newslink.stop

With Synchronet v3.11, it is also possible to run NewsLink externally from Synchronet (or as a background timed event) by using JSexec.

• newslink.cfg
• JSexec
• NNTP Service
• USENET
• Modules