NewsLink - NNTP Gateway
NewsLink is an external module for Synchronet BBS Software v3.1+. It is used to link one or more Synchronet sub-boards (aka message areas) with one or more USENET (or other NNTP network) newsgroups on one or more News Servers.
News Servers use the Network News Transfer Protocol (NNTP) to transfer messages to and from a News Client (aka newsreader). Popular News Clients include Microsoft Outlook Express, Netscape Communicator, Mozilla, Pine, Xnews, and many others. NewsLink emulates a News Client based on the NNTP standard. News Servers normally listen for incoming connections on TCP port 119 and usually require an authenticated user login.
interested) is contained in the file
newslink.js in the
exec directory. Current version is always available via git.
If you have installed Synchronet v3.10 or v3.11 from scratch (not an upgrade from an earlier version), then some of the following steps may have already been completed for you.
1. Go into SCFG->External Programs->Timed Events and created a timed event for NewsLink (if one doesn't already exist). Example:
[NEWSLINK Timed Event] 1: Internal Code NEWSLINK 2: Start-up Directory 3: Command Line ?newslink 4: Enabled Yes 5: Execution Node 1 6: Execution Days of Month Any 7: Execution Days of Week All 8: Execution Frequency 48 times a day 9: Requires Exclusive Execution No 10: Force Users Off-line For Event No 11: Native (32-bit) Executable <doesn't matter> 12: Use Shell to Execute <doesn't matter> 13: Background Execution <doesn't matter> 14: Always Run After Init/Re-init No
You may change the Execution Frequency to suit your needs.
2. Go into SCFG->Message Areas and create a new Message Group for your USENET
messsage areas (assuming you're linking to a USENET News Server). Example:
[USENET Group] 1: Long Name USENET Newsgroups 2: Short Name USENET 3: Internal Code Prefix USENET_ 4: Access Requirements
Note: At this point you may want to login to your News Server with a regular
News Client (aka newsreader), retrieve a list of available newsgroups
and make a list of the newsgroups you would like linked into your
BBS's message areas.
An alternate option is to use the provided
exec/getnewsgrouplist.js script to retrieve a list of newsgroups from your News Server and save those to a file (e.g.
$ jsexec getnewsgrouplist.js user password news.example.com > newsgroup.lst
getnewsgrouplist.js script supports alternate ports and group name filtering options, if you need them)
3. Now go into SCFG->Message Areas->USENET->Sub-boards and create some sub-boards to link to newsgroups on the News Server. Example:
[alt.bbs.allsysop Sub-board] 1: Long Name alt.bbs.allsysop 2: Short Name alt.bbs.allsysop 3: QWK Name allsysop 4: Internal Code USENET_ALLSYSOP 5: Newsgroup Name alt.bbs.allsysop 6: Access Requirements 7: Reading Requirements 8: Posting Requirements REST NOT G 9: Operator Requirements 10: Moderated Posting User 11: Maximum Messages 5000 12: Purge by Age Disabled 13: Duplicate Checking Enabled (10000 message CRCs)
An alternate option here would be to import the
newsgroup.lst file if you have one, using the SCFG “Import Message Areas” option and the “USENET Newsgroup List” file format.
4. For each NewsLinked sub-board, confirm Network Options->Internet (UUCP/NNTP) is set to “Yes”.
5. Edit your
- Change the “SERVER” line to specify the host name or IP address of the News Server you are linking with. If the News Server is listening on a non-standard TCP port (e.g. other than 119), you will need to add a “PORT” line specifying the TCP port number of the server.
- If you wish to use TLS/SSL, uncomment the “TLS” line. This will change the default port to 563.
- If the News Server requires an authenticated user login, change the “USER” and “PASS” lines to specify the username and password you use to login to the above News Server. If the “USER” line is missing or commented-out (with a semicolon), then no user authentication will be attempted (sometimes called “anonymous login”).
- For each sub-board you configured in step 3, add an “AREA” line, specifying each sub-board's “Internal Code” (from SCFG->Message Areas).
(An alternate option here is to use the
AUTO_AREAS keyword which will automatically link all sub-boards flagged as Internet/NNTP networked in SCFG)
- By default, NewsLink will only import articles (aka messages) posted *after* the first scan of the newsgroups (i.e. it will not import *any* articles during the first run). If you want NewsLink to import *all* existing articles for *all* linked newsgroups (potentially many *thousands* of messages, depending on the News Server), add the line “FLAGS I” (case insensitive). WARNING: Using this feature may cause the initial NewsLink execution to take a long time to download all the existing articles (potentially many thousands) that may be stored on the News Server.
- Remove or comment-out (with a semicolon) on the
DISABLEDline (if present).
6. Now you may force the NEWSLINK timed event to run. In SBBSCTRL (Win32 GUI), you can do this by clicking BBS->Force Timed Event->NEWSLINK, or for the console or NT services versions of Synchronet, you can touch or create the file data/newslink.now to force the event to run.
7. Watch your events log for the NewsLink execution progress and details.
Remember, if the 'I' flag was not specified (globally or per-newsgroup)
ctrl/newslink.cfg, then no articles (aka messages) will be imported
during the first run and only newly-posted articles will be imported on
Each “AREA” line in the NewsLink configuration file can specify one or more “flag characters” to enable special behaviors. These same flag characters may be specified globally (for all linked areas) using the “FLAGS” line.
The supported flags characters are:
x do not add tearlines & taglines to exported messages n do not add "From Newsgroup" text to imported messages t do not add tearline to imported messages a convert extended-ASCII chars to ASCII on imported messages r remove "Newsgroups:" header field from imported messages b decode single-part (uuencoded or yenc-encoded) binary attachments i import all articles on first run (not just new articles) s no subject filtering (do not check against text/subject.can)
To specify one or more flag characters for a specific area, you must use the following “AREA” line syntax:
AREA <sub-code> <newsgroup> <flags> [attachment_dir]
- <sub-code> is the sub-board's complete internal code (including the group's code prefix, if there is one).
- <newsgroup> is the name of the newsgroup as it is represented on the News Server. If this field is missing or is less than 2 characters in length, the newsgroup name from SCFG is used.
- <flags> is the optional list of flag characters.
- [attachment_dir] is the optional destination directory for decoded binary file attachments (only used with the 'b' flag)
AREA USENET_ALLSYSOP alt.bbs.allsysop nib /sysop_attachments AREA USENET_ALLSYSOP - asr
Multiple Servers/Config Files
ctrl/newslink.cfg is just the default NewsLink configuration file name. If you
specify the path to a different file on the
?newslink command-line, NewsLink
will use that configuration file instead. This allows you to have multiple
NewsLink timed events configured with different configurations for different
Other supported NewsLink command-line options:
| ||enable debug output|
| ||reset import pointers (import all)|
| ||update import pointers (import none)|
| ||reset export pointers (export all)|
| ||update export pointers (export none)|
| ||no e-mail addresses|
| ||no mangling of e-mail addresses (disable anti-SPAM measures)|
| ||un-mangle e-mail addresses when importing|
| ||import a fixed number (n) of articles/messages (default is 500)|
Most SPAM messages on public NNTP networks (e.g. USENET) are directed at
multiple newsgroups. For this reason, the newslink.cfg file supports a
max_newsgroups_per_article value which defaults to 5.
If you find NewsLink is filtering valid messages with an error in the log like:
!Too many newsgroups (7)
try increasing this value.
If you find NewsLink is allowing SPAM through that is addressed to multiple
newsgroups, try reducing this value. Set
disable this feature.
NewsLink also uses the
text/subject.can filter file to filter (ignore)
articles with specific text in the subject and the
ctrl/twitlist.cfg file to
ignore messages from specific users or domains.
NewsLink will yield time-slices based on the
If you find NewsLink is consuming
a large portion of your CPU cycles, try decreasing the
value or increasing the
NewsLink can be stopped by “touching” (creating or updating) the semaphore file:
With Synchronet v3.11, it is also possible to run NewsLink externally from Synchronet (or as a background timed event) by using JSexec.