Differences

This shows you the differences between two versions of the page.

--- faq:misc [2018/02/06 16:19] – [FTN MSGID] Added link to FTA-1006 (Thanks, Mark) digital man
+++ faq:misc [2018/06/18 17:09] – [FTN MSGID] fixed typos and beautification digital man
@@ Line 133: / Line 133: @@
 Because, for the message-IDs to be useful for any intended purpose (e.g. dupe-checking, reply-linking) they **must** be //unique// for every message created on a BBS.
-When using [[https://www.ietf.org/rfc/rfc0822.txt|Internet-standard message-IDs]] (defined in 1982), this is not a problem because the entire message identifier is basically free-form text (between ''<'' and ''>'' delimiters), leaving the entire data portion completely up to the implementer to add whatever data they feel necessary to guarantee a unique ID for any particular message created at any given time. No data within an Internet-standard message-ID is expected to be interpreted or used in any way, other than to uniquely identify a particular message. The data could be a large number or just a string of meaningless symbols and so long as it is unique, it's doing its job rather perfectly. Usually, an Internet-standard message-ID will contain globally unique addresses (e.g. public IP address or host name) so as to prevent random collisions with the IDs generated on a nearly unlimited number of other systems on the Internet.
+When using [[https://www.ietf.org/rfc/rfc0822.txt|Internet-standard message-IDs]] (defined in 1982), this is not a problem because the entire message identifier is basically free-form text (between ''<'' and ''>'' delimiters), leaving the entire data portion completely up to the implementer (e.g. programmer) to add whatever data they feel necessary to guarantee a unique ID for any particular message created at any given time. No data within an Internet-standard message-ID is expected to be interpreted or used in any way, other than to uniquely identify a particular message. The data could be a large number or just a string of meaningless symbols and so long as it is unique, it's doing its job rather perfectly. Usually, an Internet-standard message-ID will contain a globally unique addresses (e.g. public IP address or host name) so as to prevent random collisions with the IDs generated on a nearly unlimited number of other systems on the Internet, but there is no //requirement// that an address is included in the identifier.
-However, for FidoNet, [[http://ftsc.org/docs/fts-0009.001|the de facto standard]] (defined in 1991) imposed some restraints on the contents of the message-ID (between ''^AMSGID: '' and ''\r'' delimiters): 2 fields separated by a single space: ''origaddr'' and ''serialno''. The definition of the ''serialno'' field is thus:
+However, for FidoNet, [[http://ftsc.org/docs/fts-0009.001|the de facto standard]] (defined in 1991) imposed some restraints on the contents of its message-IDs (the data between ''^AMSGID: '' and ''CR'' delimiters): 2 fields separated by a single space: **''origaddr''** and **''serialno''**.
+The definition of the ''serialno'' field is thus:
 <file>
 ... may be any eight character hexadecimal number, as long as it is unique -
 no two messages from a given system may have the same serial number within a three years.
 </file>
-My interpretation of this requirement is that the author thought that 32-bits (8 hex digits) of data were enough to uniquely identify every message generated on a single FidoNet node for at least 3 years. Even if that were true, as an experienced BBS software developer, I have these issues with that concept:
+My interpretation of this requirement is that the author of the specification was asserting that 32-bits (8 hex digits) of data (~4.2B unique numbers) was enough to uniquely identify every message generated on a single FidoNet node for at least 3 years. Even if that assertion were true, as an experienced BBS software developer, I have these issues with that concept:
-  - 3 years is not long enough. I have message threads on my BBS dating back 20 years or more and I certainly expect and my software depends on the message-IDs being unique so long as the messages are stored in the message bases. If the messages are stored for a 100 years, they should still be unique then. So forget the 3 year clause, that's no more valid a limitation than Bill Gates' 640 Kilobytes((supposedly a myth, but a good one)).
+  - 3 years is not long enough. I have message threads on my BBS dating back 20 years or more and I certainly expect and my software depends on the message-IDs being unique so long as the messages are stored in the message bases. If the messages are stored for a 100 years, their identifiers should still be unique then. So forget the 3 year clause, that's no more valid a limitation than Bill Gates' 640 Kilobytes((supposedly a myth, but a good one)).
-  - A pseudo-randomly chosen 32-bit number is obviously **not** a good choice, but it //would// probably work for the majority use case. I fear that some inexperienced or lazy programmers might have chosen this solution and no one would be the wiser until... someday there's a random ID collision and someone might notice and complain. But probably not.
+  - A pseudo-randomly chosen 32-bit number is obviously **not** a good choice, but it //would// probably work for the majority use case. I fear that some inexperienced or lazy programmers might have chosen this simple "solution" and no one would be the wiser until... someday there's a random ID collision and someone might notice and complain. But probably not.
-  - A 32-bit ''time_t'' (Unix time, in seconds since Jan-1-1970) would seem an obvious choice for the ''serialno'' field except for the fact that today's systems can generate many thousands of messages per second if they need to. Even the multinode systems of the 80's and 90's could easily generate multiple messages within the same second. Many FidoNet programmer's fatally chose a ''time_t'' value as their ''serialno'' as their source of //uniqueness// and you can see the proof of this in the MSGIDs that show up in FidoNet messages //to this day//((If this is your software doing this, you should fix or replace it)). Again, nobody is wise to this design flaw until there is a "random" collision one day... and maybe someone notices. But usually not.
+  - A 32-bit ''time_t'' (Unix time, in seconds since Jan-1-1970) would seem an obvious choice for the ''serialno'' field except for the fact that today's systems can generate many thousands of messages //per second// if they need to. Even the multinode systems of the 80's and 90's could easily generate multiple messages within the same second. Many FidoNet programmers fatally chose a ''time_t'' value as their ''serialno'' and only source of //uniqueness// and you can see the proof of this in the MSGIDs that show up in FidoNet messages //to this day//((If this is your software doing this, you should fix or replace it)). Again, nobody is wise to this design flaw until there is a "random" collision one day... and maybe someone notices. But usually not.
-  - A system to exclusively register monotonically-incrementing message serial numbers would be error prone (e.g. the sysop deletes the data file tracking the last assigned serial number) and unnecessarily bottle-neck message generation (e.g. importing thousands of messages from some other networking technology, which would need unique FidoNet-style message-IDs assigned to each)
+  - A system to exclusively register monotonically-incrementing message serial numbers would be error prone (e.g. the sysop deletes the data file tracking the last assigned serial number) and unnecessarily bottle-neck message generation (e.g. importing thousands of messages from some other networking technology, which would need unique FidoNet-style message-IDs assigned to each).
   - It's obviously still a problem if 2 //different// systems generate the same message-ID ''serialno'' value, unless there's some //other// source of "uniqueness" in the Message-ID...
@@ Line 155: / Line 157: @@
   - It's not limited to a specific number of characters or a specific character set.
   - It's rather vague about what constitutes "a valid return address".
-  - It's only a "[[http://ftsc.org/docs/fta-1006.002|should]]" clause, which in spec-lawyer-speak, means: "you don't //have// to" (in contrast to "shall" or "must").
+  - It's only a "should" clause, which in spec-lawyer-speak, means: "you don't //[[http://ftsc.org/docs/fta-1006.002|have]]// to" (in contrast to "shall" or "must").
 So the ''origaddr'' is where Synchronet and its utilities (e.g. [[util:SBBSecho]]) adds unique data to insure that all messages generated by a correctly-configured Synchronet system have //unique// FTN message-IDs... forever. The data we place in the ''origaddr'' is currently thus:
   <msgnum>.<subcode>@<ftn_addr>
-where the ''<msgnum>'' is the monotonically-incrementing message number for the message in this particular message database (//echo area//), ''<subcode>'' identifies the message database where the identified message is stored, and ''<ftn_addr>'' is the 3D or 4D FTN-style address of the FidoNet-style node that created the message. Example:
+where
+  * ''<msgnum>'' is the monotonically-incrementing message number for the message in this particular message database (//echo area//)
+  * ''<subcode>'' identifies the system-local message database where the identified message is stored
+  * ''<ftn_addr>'' is the 3D or 4D FTN-style address of the FidoNet-style node that created the message
+Example:
 .sync@1:103/705
-The result **is** a "//valid return address for the originating network//", in that a FidoNet NetMail sent to this address will normally reach the sysop of the originating node (in the example case, me). But the //originating// network might **not** be an FTN network. There is **nothing** in FTS-9 that specifies the form or content of the ''origaddr'' field. In fact, it mentions the use of "double-quotes" and how they must be escaped if part of the address. When is the last time you saw quotes of any kind in an FTN address? Clearly the author was leaving this definition open to non-FTN addresses. And again, it's only a "should" clause anyway. This field could just be the GPS coordinates of my birthplace and would still be completely within the specification.
+The result **is** a "//valid return address for the originating network//", in that a FidoNet NetMail sent to this address will normally reach the sysop of the originating node (in the example case, me). But the //originating// network might **not** be an FTN network. There is **nothing** in FTS-9 that specifies the form or content of the ''origaddr'' field. In fact, it mentions the use of "double-quotes" and how they must be escaped if part of the address. When is the last time you saw quotes of any kind in an FTN address? Clearly the author was leaving this definition open to non-FTN addresses. And again, it's only a "should" clause anyway. This field could only contain the GPS coordinates of my birthplace and would still be completely within the requirements of the specification.
-The serial number (''serialno'') field generated by Synchronet and its utilities is a mathematical sum of the area-unique ''<msgnum>'' and a number derived from the date and time the message was created so as to maximize the use of the allocated 32-bits. The serial number will by unique among all the messages generated by a Synchronet-system in a single message area (echo), but is not guaranteed to be unique, by itself, among //all// the FTN-connected message areas of a single Synchronet-system. The ''serialno'' field alone does **not** constitute a //unique// message identifier (so don't try to use it as one).
+The serial number (''serialno'') field generated by Synchronet and its utilities is a mathematical sum of the area-unique ''<msgnum>'' and a number derived from the date and time the message was created so as to maximize the use of the allocated 32-bits. The serial number will by unique among all the messages generated by a Synchronet-system in a single message area (echo), but is not guaranteed to be unique, by itself, among //all// the FTN-connected message areas of a single Synchronet-system. The ''serialno'' field //alone// does **not** constitute a unique message identifier, so don't try to use it as one!
 **But it doesn't //look// like other other FTN MSGID's!?!**
-This is true. But before [[person:digital man|I]] decided on the makeup of the Synchronet-generated FTN MSGID's I collected and collated many thousands of FidoNet messages (from the Z1 backbone) and found that there was already a lot of variety in the //look// of the ''orginaddr'' field of the MSGIDs used over the last 20+ years: Most were simple 3D or 4D FTN-style addresses, but many had 5D-style addresses with ''@fidonet'' as the domain, sometimes ''@fidonet//.org//'', and many others just had what appeared to be an Internet-style host name (no ''@'' or FTN-style address at all). Others combined Internet-style host names with other seemingly random or incrementing numbers or symbols.
+This is true. But before [[person:digital man|I]] decided on the makeup of the Synchronet-generated FTN MSGID's I collected and collated many thousands of FidoNet messages from the Zone 1 backbone echoes and found that there was already a lot of variety in the //look// of the ''orginaddr'' field of the MSGIDs used over the last 20+ years: Most were simple 3D or 4D FTN-style addresses, but many had 5D-style addresses with ''@fidonet'' as the domain, sometimes ''@fidonet//.org//'', and many others just had what appeared to be an Internet-style host name (no ''@'' or FTN-style address at all). Others combined Internet-style host names with other seemingly random or incrementing numbers or symbols.
 So Synchronet/SBBSecho did not introduced some "new thing" into FidoNet. Indeed, I would just be the next thoughtful programmer that has solved this problem in his or her own way because the original specification (FTS-9) was short-sighted((RFC-822... anyone, anyone?)) and **never fixed**.
-If you have or know of someone that has software that has a problem with the Synchronet-generated FTN MSGID's, I am sorry. That same software, if it exists, would have a similar problem, whatever that is, with many other unique FTN MSGID's which existed long before Synchronet and SBBSecho started sending messages with unique message-IDs into FidoNet-style networks.
+If you have or know of someone that has software that has a problem with the Synchronet-generated FTN MSGID's, I am truly sorry. That same software, if it exists, would have a similar problem, whatever that is, with many other unique FTN MSGID's which existed long before Synchronet and SBBSecho started sending messages with unique message-IDs into FidoNet-style networks.
-**But //why// don't you change it to look like other other FTN MSGID's!?!**
+**But //why// don't you change it to look like other FTN MSGID's!?!**
 Because:
   - That would defy the uniqueness of the FTN MSGID's generated by Synchronet, and that's the **entire point** of their existence.
   - If I were motivated to change how the MSGIDs look, it would be to make them **more** unique, not //less//.
-  - The MSGIDs are **already compliant** with the requirements of the //de facto standard// specification (FTS-9) - [[http://ftsc.org/docs/fts-0009.001|read it]].
+  - The MSGIDs are **already compliant** with the requirements of the //de facto standard// specification ([[http://ftsc.org/docs/fts-0009.001|read it]]), so they don't need to be changed.
-  - The MSGIDs are not //that// dissimilar to unique MSGIDs that have been included in FidoNet messages for **over 20 years**.
+  - The MSGIDs are not //that// dissimilar to unique MSGIDs that have been included in FidoNet messages posted by other compliant-software for **over 20 years**.
   - You should fix or replace **your software** instead. :-P

Trace:

Differences

Navigation

Search

Toolbox