Download audio file in Large Hi-Fi format OGG MP3
Download audio file in Small Lo-Fi format OGG MP3Speex

Closing Music:Elena Rose - Answer Me

Featured Project:Cyberpunk Radio

Gopher

DeepGeek

29 November 2009

Abstract: This is a script for a podcast. This podcast details the Gopher protocol, a protocol for finding and sending text documents over the Internet, that pre-dated the World Wide Web.

1 Introduction

I must have first heard about the Gopher protocol years ago, but I was very interested to learn that it was still being used today, and is the subject of an attempt to revive the format. I heard about this through Klaatu’s podcast “The Bad Apples,” when he read viewer mail about a project called “Overbite.” The Overbite project is a Firefox plug in gopher browser, which is superior to the gopher browser that is built in to Firefox. By The Way, the built-in gopher browser was requested removed due to the fears that it may cause a security hole in the future, and that it is not needed any more.

Gopher was developed at the University of Minnesota in 1991, the Universiy’s mascot is the Gopher, hence the name.

2 Pre-WWW Interweb

So let’s step back, both topically and in time, to the pre World Wide Web Interweb. Back then, it was actually the Internet, as the “Web” was not yet invented.

What we now call the “Interweb” was “The Internet,” and before that it was “Arpanet.” Arpanet was an inter-networking of computers for educational research and military development. Only after the Arpanet grew did it morph into the Internet. World Wide Web services was a bomb that changed the Internet from a text-based network of computers to a network of computers that could serve information for the masses. This required graphical and later multimedia support, not to mention improvements to the speed of the backbone.

This is key. I read somewhere during my research on this, that in the bad old days the ’net’s backbone functioned at the speed of a modern day internal modem. Thus, the text-based nature of the ’Net in those days. Can you imagine how slow a web page full of images would be? I remember dial-up and images on web pages were slow, full fledged pictures were a challenge, and an mp3 would take an hour for a five minute song.

Which is why gopher worked. It was stripped down for text. When you came into a server, that server had a set of files that were the instructions for the menu links. The files you wanted themselves were simple text files. It is also one of those protocols you can operate from telnet, and it will make sense as a terminal application.

3 Gopher’s strange siblings

Needless to say, we don’t navigate around the Internet with menu systems. Menu systems are nice, but information can be buried way to deeply on too many systems. So let’s look briefly at Gopher’s brothers and sisters of the time.

Archie was a way of searching anonymous FTP servers. Archie was originally built out of a system that would log into FTP servers every so often, and take a file listing. When you searched with Archie you searched for a specific file name. In Archie’s first incarnation, this search was merely a Unix grep through a large amount of FTP server directories.

When I say “Archie,” I know you’re saying “Veronica and Jughead.” Oddly enough, Archie’s programmer was not fond of the comic books and did not mean to pay any homage to the comic book character. One of the programmers, Alan Emtage, saw it as a way of searching archives, but the file names at the time were limited to six digits, he therefore shortened the name to “Archie” by removing the “v” from “Archive.”

I’m sure by now, dear listener, that you’re saying “what, you had to know the file name?” Yes, you had to know the file name. I managed to find a still alive archie search web-page in Poland, and so I tried a search for “jargon.” “The Hacker’s Dictionary” used to be known as “The Jargon File.” It was still there! How did you get file names? By emailing friends and reading newsgroups.

Now, we all know about Google, the big problem of the Internet in the early days was finding the document. Different ways to get the most pertinent document were wanting. Building indexes of the words in documents works, but not like Google’s technique of giving websites weight based on what links to them, but let’s keep it pre Google and talk about searching the insides of documents.

Jughead, later renamed Jugtail because of copyright concerns, searched the text of documents for a single server. I could only find one Jugtail site in operation on the web, and there was a software page at nongnu.org that had no downloads. Jughead was originally written by Rhett Jones in 1993 at the University of Utah.

Veronica searches the insides of documents over several servers. I have found two of these, so they are still out there. In this way, veronica can be said to be the predecessor of indexing-style search engines. Veronica was originally developed at the University of Nevada, Reno by Steven Foster and Fred Barrie.

4 Fooling Around With Gopher Clients

There are a large number of gopher clients out there, but the large number is because of gopher coming from an era when there was less standardization of computers. There have been gopher clients written for a huge variety of systems, by which I mean all manner of all computers with different OS’s, such as old Commodore computers, various DOS computers, mainframes.

There is also gopher in web browsers. Just as a web browser will give you an all-in-one experience with email plug-ins, ftp access, etc. So there is also a native gopher browser in firefox.

So I would like to give my impressions of a few that I have tried.

Firefox has a built in one, which I found less than satisfactory for the reason that it was “hard wired” to port 70, the default port of gopher. Veronica searches, and indeed some gopher sites, do deviate from port 70. Other than that, it works fine.

You can also get the Overbite plug-in for Firefox. This thing is, IMHO, GUI based gopher browsing at it’s finest. It is not hard wired to a port, so when you go off port 70, it keeps going. It also allows for the latest advances in gopher, so you can view images and and download podcasts over gopher with it, not to mention that it supports gopher menus linking out into the World Wide Web.

From the command line I also tried “gopher,” the original client that started it. Standard black and white, so it was just too bland for my tastes. I shied away from it for my tastes. While I am quite comfortable with the command line, I do basically do my day to day computing in a GUI except when the CLI is more efficient.

I found the lynx browser to be much better for the CLI environment than gopher. Links uses colored text to denote links and headers, for a very visually attractive text based browser experience. I also note that lynx handles HTTP downloads well so I am sure it will be just as good as gopher based downloads too.

5 Gopher Advances

You may have noticed that I used phrases like “latest advances,” and “HTTP downloads.” Gopher has evolved over the years, going into it’s third generation of features. Now, gopher has ways of handling links to all sorts of binary files, as well as being able to link to the World Wide Web. You can download audio, video, and stills via gopher, but they are not rendered in-line as they are in web pages.

6 Modern Gopher Servers

There are many Gopher servers, but the most well known and used ones are bucktooth and pygopherd. Bucktooth is written in Perl, and is not currently available as a Debian package, so I wont review it.

However, I have had some fun experimenting with Pygopherd. This Gopher daemon is written in Python, and has some really interesting functions to it. In it’s default state, if functions as a Gopher server should, waiting for Gopher clients to attach to it. However, this server has a feature which is amazing to me, and that is if it is attached to by a Web Browser or a WAP phone, it takes the Gopher pages and re-formats them on the fly to be in the appropriate protocol. This creates a situation that makes Gopher more universal. Not only can you use it via Web Browser or WAP phone, but when on-line search engines like Google come along, they don’t get gopher pages that they may not understand, instead they get web pages that they understand quite well. Thus, your gopher pages get indexed by the search engines, allowing them to be found when they contain information users of these search engines want. Not only that, but you don’t have to worry about such a search engine user needing the right client, as Pygopherd will re-format your gopher pages to however their system needs them. Optionally, you can even tell Pygopherd to publish a variety of Unix/Linux mailbox formats, so perhaps some sort of remote mail reading method, while not private, is a definite possibility.

7 The Viability of Gopher Today

This talk of a Gopher site serving a variety of clients, both Gopher based and non-Gopher based, makes this an excellent place to segue into discussing Gopher in the Modern Day.

Many people who could talk to you about Gopher would tell you that is has been superseded by the web. If you look at Gopher as a protocol for getting text-based articles to people then I suppose it may have been.

But looked at another way, Gopher does not look the same. If you look at Gopher as a hybrid version of FTP then it has a whole other aspect. Then it becomes a way of moving information where you can transfer files between computers, and do so without the problems with getting through a firewall that FTP has. On top of that, you can actual look at certain things, like text, or in the case of GUI browsers, images, instead of downloading first, then viewing. On that note, Gopher is a pretty sweet deal.

However, the lack of inline graphic design leaves gopher disabled for the purposes of Internet commerce.

What a strange Duck! Where does that leave us?!

I would say, that I would not ever advocate not using the HTTP. However, as far as non-commercial hobbyist use goes, Gopher should still be considered as a possible solution. It gives you a way of doing all things you may need to do over the Interweb in one place, and if you user base is right, the bandwidth savings may make it viable to run this off a home server (and in that case, your ISP would probably never think of filtering or looking for your traffic on port 70.)

One thing I have noticed is that all the Gopher Pages I have viewed have loaded incredibly fast. I really can’t tell if this is because perhaps the server is not heavily loaded or if the protocol is actually faster, but so far so good. They say that you can do anything over HTTP you can do over Gopher. But to make that article load fast in one file request, you would have to write a static HTML page that did not chain to any CSS sheets externally, and did not use imagery. If you did those things, you would have the same speed as a text page on gopher, but whether or not the server load is a factor, is a question I just can’t answer. Some of these servers are on older computers in people’s homes, which means a slow computer for a server, but maybe your the only request it is handling at the moment.

Where can one find Gopher today?! There are about 130 servers on-line now according to Wikipedia. But if you do the right Google search you just may see a result on a web-page running on port 70, you never know. There is also a big west-coast co-operative on the web called sdf.lonestar.org. This organization has a long history reaching back to the days of SDF being an anime BBS, but now their free shell and free Gopher hosting has made them a scene for techies experimenting with Gopher today. Be sure to check out the show transcript if you want to see a link to them as well as other links.

And wither did Gopher go. What caused it’s decline. Well, it turns out the University of Minnesota began charging licensing fees, and IBM pressured Tim Berners-Lee to put HTTP into the Public Domain.


Sources: