Synchronet v3.19b-Win32 (install) has been released (Jan-2022).

You can donate to the Synchronet project using PayPal.

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
module:newslink [2010/02/24 15:32] – created digitalmanmodule:newslink [2022/09/29 09:37] (current) – [Background] Change from CVS to git nelgin
Line 1: Line 1:
-====== NewsLink ======+====== NewsLink - NNTP Gateway ====== 
 +{{:module:rob_s_iphone_sept-2015_185_2_.jpg?500|A newsstand sign I saw in an airport, long after I named this module}} 
 +====== Background ====== 
 + 
 +NewsLink is an external module for Synchronet BBS Software v3.1+. 
 +It is used to link one or more Synchronet sub-boards (aka message areas) 
 +with one or more USENET (or other NNTP network) newsgroups on one or 
 +more News Servers. 
 + 
 +News Servers use the Network News Transfer Protocol (NNTP) to transfer 
 +messages to and from a News Client (aka newsreader). Popular News Clients 
 +include Microsoft Outlook Express, Netscape Communicator, Mozilla, Pine, 
 +Xnews, and many others. NewsLink emulates a News Client based on the NNTP 
 +standard. News Servers normally listen for incoming connections on TCP 
 +port 119 and usually require an authenticated user login. 
 + 
 +NewsLink is written in 100% JavaScript and the source code (if you're 
 +interested) is contained in the file ''newslink.js'' in the 
 +Synchronet ''[[dir:exec]]'' directory. Current version is always available via [[dev:git]]. 
 + 
 + 
 +====== Step-by-Step Installation/Configuration ====== 
 + 
 +If you have installed Synchronet v3.10 or v3.11 from scratch (not an upgrade 
 +from an earlier version), then some of the following steps may have already 
 +been completed for you. 
 + 
 + 
 +1. Go into [[:util:SCFG]]->External Programs->Timed Events and created a timed event 
 +for NewsLink (if one doesn't already exist). Example: 
 + 
 +   [NEWSLINK Timed Event] 
 +    1: Internal Code                   NEWSLINK 
 +    2: Start-up Directory 
 +    3: Command Line                    ?newslink 
 +    4: Enabled                         Yes 
 +    5: Execution Node                  1 
 +    6: Execution Days of Month         Any 
 +    7: Execution Days of Week          All 
 +    8: Execution Frequency             48 times a day 
 +    9: Requires Exclusive Execution    No 
 +   10: Force Users Off-line For Event  No 
 +   11: Native (32-bit) Executable      <doesn't matter> 
 +   12: Use Shell to Execute            <doesn't matter> 
 +   13: Background Execution            <doesn't matter> 
 +   14: Always Run After Init/Re-init   No 
 + 
 +You may change the Execution Frequency to suit your needs. 
 +    
 +2. Go into SCFG->Message Areas and create a new Message Group for your USENET 
 +   messsage areas (assuming you're linking to a USENET News Server). Example: 
 + 
 +   [USENET Group] 
 +   1: Long Name                  USENET Newsgroups 
 +   2: Short Name                 USENET 
 +   3: Internal Code Prefix       USENET_ 
 +   4: Access Requirements 
 + 
 +Note: At this point you may want to login to your News Server with a regular 
 +News Client (aka newsreader), retrieve a list of available newsgroups 
 +and make a list of the newsgroups you would like linked into your 
 +BBS's message areas. 
 +An alternate option is to use the provided ''[[dir:exec]]/getnewsgrouplist.js'' script to retrieve a list of newsgroups from your News Server and save those to a file (e.g. ''newsgroup.lst''). Example: 
 +  $ jsexec getnewsgrouplist.js user password news.example.com > newsgroup.lst 
 +(Note: the ''getnewsgrouplist.js'' script supports alternate ports and group name filtering options, if you need them) 
 + 
 + 
 +3. Now go into SCFG->Message Areas->USENET->Sub-boards and create some 
 +sub-boards to link to newsgroups on the News Server. Example: 
 + 
 +   [alt.bbs.allsysop Sub-board] 
 +    1: Long Name                  alt.bbs.allsysop 
 +    2: Short Name                 alt.bbs.allsysop 
 +    3: QWK Name                   allsysop 
 +    4: Internal Code              USENET_ALLSYSOP 
 +    5: Newsgroup Name             alt.bbs.allsysop 
 +    6: Access Requirements 
 +    7: Reading Requirements 
 +    8: Posting Requirements       REST NOT G 
 +    9: Operator Requirements 
 +   10: Moderated Posting User 
 +   11: Maximum Messages           5000 
 +   12: Purge by Age               Disabled 
 +   13: Duplicate Checking         Enabled (10000 message CRCs) 
 +    
 +An alternate option here would be to import the ''newsgroup.lst'' file if you have one, using the SCFG "Import Message Areas" option and the "USENET Newsgroup List" file format. 
 + 
 +4. For each NewsLinked sub-board, confirm Network Options->Internet (UUCP/NNTP) is set to "Yes". 
 + 
 +5. Edit your ''[[dir:ctrl]]/[[config:newslink.cfg]]'' file: 
 + 
 +  * Change the "SERVER" line to specify the host name or IP address of the News Server you are linking with. If the News Server is listening on a non-standard TCP port (e.g. other than 119), you will need to add a "PORT" line specifying the TCP port number of the server. 
 +  * If you wish to use TLS/SSL, uncomment the "TLS" line.  This will change the default port to 563. 
 +  * If the News Server requires an authenticated user login, change the "USER" and "PASS" lines to specify the username and password you use   to login to the above News Server. If the "USER" line is missing or commented-out (with a semicolon), then no user authentication will be attempted (sometimes called "anonymous login"). 
 +  * For each sub-board you configured in step 3, add an "AREA" line, specifying each sub-board's "Internal Code" (from SCFG->Message Areas). 
 +Example: ''AREA USENET_ALLSYSOP''\\ 
 +(An alternate option here is to use the ''AUTO_AREAS'' keyword which will automatically link all sub-boards flagged as Internet/NNTP networked in SCFG) 
 +  * By default, NewsLink will only import articles (aka messages) posted *after* the first scan of the newsgroups (i.e. it will not import *any* articles during the first run). If you want NewsLink to import  *all* existing articles for *all* linked newsgroups (potentially many *thousands* of messages, depending on the News Server), add the line "FLAGS I" (case insensitive). WARNING: Using this feature may cause the initial NewsLink execution to take a long time to download all the existing articles (potentially many thousands) that may be stored on the News Server. 
 +  * Remove or comment-out (with a semicolon) on the ''DISABLED'' line (if present). 
 + 
 +6. Now you may force the NEWSLINK timed event to run. In SBBSCTRL (Win32 GUI), 
 +you can do this by clicking BBS->Force Timed Event->NEWSLINK, or for the 
 +console or NT services versions of Synchronet, you can touch or create the 
 +file data/newslink.now to force the event to run.  
 + 
 +7. Watch your events log for the NewsLink execution progress and details. 
 +Remember, if the 'I' flag was not specified (globally or per-newsgroup) 
 +in your ''[[dir:ctrl]]/[[config:newslink.cfg]]'', then no articles (aka messages) will be imported 
 +during the first run and only newly-posted articles will be imported on 
 +subsequent runs. 
 + 
 +====== Area Flags ====== 
 +Each "AREA" line in the NewsLink configuration file can specify one or more 
 +"flag characters" to enable special behaviors. These same flag characters may 
 +be specified globally (for all linked areas) using the "FLAGS" line. 
 + 
 +The supported flags characters are: 
 + 
 +  x   do not add tearlines & taglines to exported messages 
 +  n   do not add "From Newsgroup" text to imported messages 
 +  t   do not add tearline to imported messages 
 +  a   convert extended-ASCII chars to ASCII on imported messages 
 +  r   remove "Newsgroups:" header field from imported messages 
 +  b   decode single-part (uuencoded or yenc-encoded) binary attachments 
 +  i   import all articles on first run (not just new articles) 
 +  s   no subject filtering (do not check against text/subject.can) 
 + 
 +To specify one or more flag characters for a specific area, you must use the 
 +following "AREA" line syntax: 
 + 
 +    AREA <sub-code> <newsgroup> <flags> [attachment_dir] 
 + 
 +where: 
 +  * <sub-code> is the sub-board's complete internal code (including the group's code prefix, if there is one). 
 +  * <newsgroup> is the name of the newsgroup as it is represented on the News Server. If this field is missing or is less than 2 characters in length, the newsgroup name from SCFG is used. 
 +  * <flags> is the optional list of flag characters. 
 +  * [attachment_dir] is the optional destination directory for decoded binary file attachments (only used with the 'b' flag) 
 + 
 +Examples: 
 + 
 +    AREA USENET_ALLSYSOP alt.bbs.allsysop nib /sysop_attachments 
 +    AREA USENET_ALLSYSOP - asr 
 + 
 + 
 +====== Multiple Servers/Config Files ====== 
 + 
 +''[[dir:ctrl]]/[[newslink.cfg]]'' is just the default NewsLink configuration file name. If you 
 +specify the path to a different file on the ''?newslink'' command-line, NewsLink 
 +will use that configuration file instead. This allows you to have multiple 
 +NewsLink timed events configured with different configurations for different 
 +News Servers. 
 + 
 + 
 +====== Command-line Options ====== 
 + 
 +Other supported NewsLink command-line options: 
 + 
 +^ Option ^ Description ^ 
 +| ''-d'' | enable debug output | 
 +| ''-ri'' | reset import pointers (import all) | 
 +| ''-ui'' | update import pointers (import none) | 
 +| ''-re'' | reset export pointers (export all) | 
 +| ''-ue'' | update export pointers (export none) | 
 +| ''-ne'' | no e-mail addresses | 
 +| ''-nm'' | no mangling of e-mail addresses (disable anti-SPAM measures) | 
 +| ''-um'' | un-mangle e-mail addresses when importing | 
 +| ''-ix //[n]//'' | import a fixed number (n) of articles/messages (default is 500) | 
 + 
 + 
 +====== Controlling SPAM ====== 
 + 
 +Most SPAM messages on public NNTP networks (e.g. USENET) are directed at 
 +multiple newsgroups. For this reason, the newslink.cfg file supports a 
 +''max_newsgroups_per_article'' value which defaults to 5. 
 + 
 +If you find NewsLink is filtering valid messages with an error in the log like: 
 +  !Too many newsgroups (7) 
 +try increasing this value. 
 + 
 +If you find NewsLink is allowing SPAM through that is addressed to multiple 
 +newsgroups, try reducing this value. Set ''max_newsgroups_per_article'' to ''0'' to 
 +disable this feature. 
 + 
 +NewsLink also uses the ''[[dir:text]]/[[config:subject.can]]'' filter file to filter (ignore) 
 +articles with specific text in the subject and the ''[[dir:ctrl]]/[[config:twitlist.cfg]]'' file to 
 +ignore messages from specific users or domains. 
 + 
 + 
 +====== Yielding ====== 
 + 
 + 
 +NewsLink will yield time-slices based on the ''lines_per_yield'' (default: ''5'') 
 +and ''yield_length'' (default: ''1'') values.  
 + 
 +If you find NewsLink is consuming 
 +a large portion of your CPU cycles, try decreasing the ''lines_per_yield'' 
 +value or increasing the ''yield_length'' value. 
 + 
 + 
 +====== Stopping NewsLink ====== 
 + 
 +NewsLink can be stopped by "touching" (creating or updating) the semaphore file: ''[[dir:data]]/newslink.stop'' 
 + 
 + 
 +====== JSexec ====== 
 +With Synchronet v3.11, it is also possible to run NewsLink externally from 
 +Synchronet (or as a background timed event) by using [[util:JSexec]]. 
  
 ===== See Also ===== ===== See Also =====
 +  * [[config:newslink.cfg]]
 +  * [[util:JSexec]]
 +  * [[:service:nntp|NNTP Service]]
 +  * [[network:USENET]]
   * [[:module:|Modules]]   * [[:module:|Modules]]
  
 +{{tag>usenet nntp jsutil}}

Trace: