News Servers And Usenet For Noobs.

Getting Started

Frequently people ask "What are news servers?" and "What is Usenet?".

Usenet is one of the oldest messaging systems on the net, and has been around since 1980 or so. In concept, it is like a message board that you can email messages to, allowing others to respond. The messages are divided into groups, or newsgroups, which are organised into hierarchies. So, for example, in the comp hiearchy you might find a group comp.lang.basic, the comp hierarchy being devoted to computer-related groups.Some others:

sci - science - e.g: sci.space.history
rec - recreation - e.g: rec.arts.disney
za - South Africa - e.g: za.politics
and so on.....

There are many groups within most hierarchies ( some servers carry over 100,000 groups ):

Messages are posted by people with an interest in the group:

Servers

Usenet runs on a protocol named NNTP - Network News Transfer Protocol - which runs by default on Port 119. Normally your ISP will run a Usenet, or News, Server. These servers are configured to talk to other news servers, so that a message posted to one server, will be relayed over to other news servers that do not have that message, and hence the messages propagate around the net. Note that the messages are ( normally ) actually copied from server to server and that the server holds a local copy of the messages it has received.

Binaries

Usenet has a reputation for carrying binary files - mp3's, executables, etc. Now, since the system was designed to take text messages, to post a binary it is necessary to encode the binary so that it looks like text ( low ascii ) and split it into parts small enough to be normal messages. Many news servers do not accept messages over a pre-defined size. So if you are looking at the available messages via a news reader, you might see your favourite mp3 split into a lot of pieces like this:

But don't despair - your news reader client can normally sort all these parts out for you.

Most binaries reside in the alt hierarchy.

Servers part ii

A full Usenet feed is three to four terabytes per day - a heavy load. So many news servers retrieve only the text groups, and not the binaries.

Messages are retained for only a certain period - this is referred to as newsgroup Retention. A binary group retention of 30 days or more is excellent! Text group retention can be weeks or even years, because of the limited size of the messages.

Completeness. Because binaries are posted in multiple parts, it happens that not all parts of a file propagate correctly. This stuffs you if you are trying to download a binary. It is important that your news server gets all the parts.

Whew! Complicated - What Do I Do?

It is not so bad!

To access all this, and, believe me it is worth it, you need two things - a client program for your PC/Mac/whatever, and a news server.

In South Africa, the latter is the hard part. Telkom and Verizon have pathetic news servers - don't even bother! IS, on the other hand, appear to offer a live feed via their news server to a big commercial news feed overseas ( I suspect Giganews ). You may also pay to subscribe to one of the big overseas news servers yourself, but be aware that this is counted as international traffic cap-wise. Take a look at:

Giganews
Easynews
Newshosting

On the client program side it is very much a matter of personal preference. Some clients are oriented towards binaries, others are general purpose. If you have a Windows PC I suggest looking at Forte Agent, Newsbin Pro, Grabit.

If you choose Easynews, you need nothing much more than a browser and a good download manager ( ReGet strongly suggested ) since Easynews has a very useful facility to allow you to choose and zip your downloads, and then get them via HTTP. Since, in South Africa, the default HTTP port (80) is often given priority by ISPs who shape other ports into oblivion, this is very useful!

If you choose a standard NNTP reader, once it is installed and configured to your chosen news server, you normally first download a full list of the groups that the server carries. Be patient - maybe do it at night. It can take an hour or so! Once that is done, you can download the currently available message headers for the groups you are interested in - so you can see what messages are available. This can take time depending on the groups. There might be zero or half-a-million headers in your alt.binaries.erotica.piggies! Once you have the headers, you can download the parts, and your newsreader will normally have a way of decoding and assembling the parts into usable files. I suggest you RTFM on this!

Par Files

Sadly, messages can become corrupted on any net transfer system. Many people who post binaries on Usenet include lovely things called Par files to help you get around corrupt or missing messages. According to Wikipedia, par files are part of "an error-correction system that can be applied to a collection of files to allow recovery when one or more of the files is lost". The scheme uses parity to do this. It is always advisable to download any par files on offer and run them through ( on Windows ) Quickpar to ensure your downloads are correct.

Nice things, par files!

NZB Files

It can be a big schlep downloading group names and daily headers via NNTP. The Newsbin writers therefore created a format for a small ascii file that contains all the original identifiers of a posting ( e.g: complete album in mp3 format ) with the subject information. A compatible news client such as Newsbin can use these little files to download the posting from your local news server. There are also sites that carry these, that you can search and get the nzb files from! Look at Newzleech for a partially free one.

Personal Taste

How you go about dealing with Usenet is very much a matter of personal preference and taste. If you are on an uncapped account I recommend an Easynews subscription - $10 for 20Gb normally. If you are on a capped account and need local content, use the IS news servers if you are on IS. They are a bit up and down, but you can access them for free. If you are capped with SAIX, well, see how far you can push that cap early every month!

Getting to grips with Usenet is well worth the effort.