Running Open Genera 2.0 on Linux Systems

Introduction

Throughout the early 1980s to early 1990s, Symbolics Inc. manufactured a series of workstations running an advanced Lisp environment called "Genera". The combination of specialized hardware with a powerful software system made these Lisp machines quite unique. For instance, it performed array bounds checking at the hardware level and manipulated data structures directly in hardware, rather than treating memory as a flat array like other general-purpose computers (still the case today). Symbolics provided workstation-level features for the Genera environment including large high-resolution bitmap displays, overlapping windows, and mouse control. In the early 1980s, long before the Macintosh appeared, this was cutting-edge technology.

In the early 1990s, Symbolics took a bold step by releasing "Open Genera" as a general-purpose software environment on Digital Equipment Corporation (DEC) Alpha workstations. They created a "Virtual Lisp Machine" (VLM) that ran under OSF/1 (later Digital UNIX) and emulated the complete Symbolics Lisp machine architecture.

The VLM source code was written in Lisp and generated Alpha assembly instructions at runtime. In the early 2000s, Brad Parker came up with an ingenious idea to see if he could modify the Alpha assembly generator to produce C code fragments instead. It worked, and the result was VLM ported to 64-bit Linux.

We now enter a gray area. Symbolics dissolved in the mid-1990s, and its intellectual property situation is currently unclear. Copyrights to VLM and Genera itself are held by different individuals. There is no longer any "Symbolics Inc." company.

Given these considerations...

This document aims to guide you through installing VLM and Symbolics Open Genera on modern 64-bit Linux systems. If you decide to proceed, note that you may or may not be infringing on others' copyrights. Enjoy yourself and stay safe!

Essential Reading

If you're unfamiliar with Genera, you need to read the following:

1. Genera Concepts (pdf)
2. Genera User's Guide (pdf)

Beyond the above two, I strongly recommend reading "Lisp Lore: A Guide to Programming the Lisp Machine"

Detailed Installation Guide

Installing Linux

VLM should run on any modern 64-bit Linux system. The commands below are specifically targeted at Ubuntu, and I have tested them on Ubuntu 15.10 through Ubuntu 18.04 LTS.

I chose Ubuntu simply because I'm most familiar with it. These instructions also apply to Debian with minor variations (if any). Similarly, they can be used for CentOS or Fedora, though this is beyond the scope of this document.

Installing Required Packages

You need curl to download required packages, so let's install it first. (wget would work fine too)

$ sudo apt-get install curl

Creating Directory for VLM and Genera

The directory can be placed anywhere; I put it in my home directory just for convenience.

$ mkdir $HOME/lisp_environment

Installing VLM

VLM is actually just a binary executable called vlm\_executable. The entire virtual Lisp machine is contained in this single file. Source code can be found in several places, such as GitHub, where you can compile from source (if pre-built binaries don't work on your machine), or download pre-compiled binaries directly.

There are two versions of pre-compiled binaries: vlm\_executable compiled on Core-i7 machines, and vlm\_executable-i5 compiled on Core-i5 machines. Some users report that the Core-i7 binary doesn't work on older I5 machines, so if you get illegal instruction errors, try the Core-i5 binary.

Core i7 Binaries

$ cd $HOME/lisp_environment
$ curl -L -O https://archives.loomcom.com/genera/vlm_executable
$ chmod a+x vlm_executable

Core i5 Binaries

$ cd $HOME/lisp_environment
$ curl -L https://archives.loomcom.com/genera/vlm_executable-i5 -o vlm_executable
$ chmod a+x vlm_executable

Downloading World and Debugger Files

Genera itself is released as two components:

1. A "World" file containing the entire running state of the Lisp machine serialized to disk. This file is named Genera-8-5-xlib-patched.vlod (can be renamed)
2. Files on the local UNIX filesystem exported via NFS. Thece files are located under /var/lib/symbolics

The following steps will download the required files and place them in the correct locations.

$ cd $HOME/lisp_environment/
$ curl -L -O https://archives.loomcom.com/genera/worlds/Genera-8-5-xlib-patched.vlod
$ curl -L -O https://archives.loomcom.com/genera/worlds/VLM_debugger
$ curl -L -O https://archives.loomcom.com/genera/worlds/dot.VLM
$ mv dot.VLM .VLM
$ cd /var/lib
$ sudo curl -L -O https://archives.loomcom.com/genera/var_lib_symbolics.tar.gz
$ sudo tar xvf var_lib_symbolics.tar.gz

Next, we want the owner of the symbolics directory to be the user running Genera (i.e., your own account). In typical Ubuntu installations, your UID and GID are both 1000, but this isn't always the case, so be sure to use the correct username and group name here. Also note that these should match the UID and GID used in the NFS setup below.

$ sudo chown -R <user>:<group> symbolics

Finally, edit the .VLM file, modifying this line:

vlm.worldSearchPath: /home/user/lisp_environment

Change it to the actual directory where you placed the Genera files.

Note: If you decide to use IP addresses differant from this tutorial, be sure to modify this line accordingly:

vlm.network: tap0:INTERNET|192.168.3.2;gateway=192.168.3.1

Setting Up hosts File

Edit /etc/hosts to match the internal network for the VLM installation. These IP addresses can be changed to anything you prefer; indeed, very sophisticated networking is possible, but for simplicity, I suggest sticking with these addresses. If you use different IP addresses, you'll at least need to edit your .VLM file.

You can change the hostnames lisp-vlm and lisp\_machine below to whatever you like, but remember to use the correct names when setting up the Lisp machine later!

Genera's VLM software will use 192.168.3.2, and the UNIX (Linux) host will create a tun0 interface bound to 192.168.3.1.

192.168.3.1    lisp-vlm
192.168.3.2    lisp_machine

Configuring time and daytime Services

Genera synchronizes clocks from UNIX via the old-fashioned time and daytime services. On modern Linux systems, these services are disabled by default (or not even installed), so you need to install and enable them.

sudo apt-get install inetutils-inetd

Then edit /etc/inetd.conf

sudo vi /etc/inetd.conf

Replace with the following content:

time      stream  tcp  nowait root internal
time      dgram   udp  wait   root internal
daytime   stream  tcp  nowait root internal
daytime   dgram   udp  wait   root internal

Restart the service

$ sudo systemctl restart inetutils-inetd.service

Confirm it's working via telnet localhost 37 and telnet localhost 13

Installing and Configuring NFS

Genera accesses files on the local host (Linux) via NFS.

First, install NFS:

$ sudo apt-get install nfs-kernel-server

Next comes configuration. You should modify the anonuid and anongid values below according to your actual user UID and GID. In typical Ubuntu installations, the primary user's UID and GID are both 1000, but verify this yourself.

Edit /etc/exports

/       lisp_machine(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)
# If your home is a separate partition, you need to export it separately. Otherwise you won't be able to access it.
/home   lisp_machine(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)

Restart NFS service

$ sudo systemctl restart nfs-kernel-server

Then, verify the filesystem is correctly exported by running

$ sudo exportfs -rav

Ubuntu Notes for 17.04 and Later

OpenGenera uses the very old NFSv2 protocol. Ubuntu 17.04 and later versions don't enable NFSv2 by default, requiring additional work to enable NFSv2.

Edit /etc/default/nfs-kernel-server. You need to change these two lines:

RPCNFSDCOUNT=8
RPCMOUNTDOPTS="--manage-gids"

To:

RPCNFSDCOUNT="--nfs-version 2 8"
RPCMOUNTDOPTS="--nfs-version 2 --manage-gids"

Edit /etc/nfs.conf, in the \[nfsd\] section, add these two lines:

udp=y
vers2=y

Finally, restart NFS:

$ sudo systemctl restart nfs-kernel-server

After restarting NFS service, run the following command to confirm version 2 is enabled (you should see +2 in the version list):

sudo cat /proc/fs/nfsd/versions

Proceed to the next step upon success.

If all efforts fail, that's bad news. NFSv2 modules are no longer compiled into newer Linux kernels by default. In this case, either compile your own Linux kernel or download a world implementing NFSv3 protocol from http://www.jachemich.de/vlm/genera.html. Note that this world's Genera version is 8.6, not 8.5. However, I haven't found 8.6 system sources yet.

Creating tap0 Interface

We'll allow Genera to access the host's NFS-exported filesystem through addresses assigned to the TUN/TAP interface. This is straightforward on Linux.

# sudo ip tuntap add dev tap0 mode tap
# sudo ip addr add 192.168.3.1/24 dev tap0
# sudo ip link set dev tap0 up

Genera is assigned the address 192.168.3.2 and expects to find NFS files through 192.168.3.1.

Remember! This operation needs to be performed every reboot. Ways exist to make this permanent, but I'll leave that as an exercise for the reader.

Edit /etc/network/interfaces, adding the following code:

allow-hotplug tap0
auto tap0
iface tap0 inet manual
  pre-up ip tuntap add tap0 mode tap
  pre-up ip addr add 192.168.3.1/24 dev tap0
  up ip link set dev tap0 up
  post-down ip link del dev tap0

Starting Genera

Now you're ready for the first startup!

Unfortunately, Ubuntu 17.10 requires special steps, so I've split this into two branches for explanation.

Ubuntu 17.04 and Earlier, Ubuntu 18.04 and Later (Not Ubuntu 17.10)

Run directly from $HOME/lisp\_environment:

$ ./vlm_executable

Ubuntu 17.10 Only

Ubuntu 17.10 defaults to Wayland instead of X11. Wayland is usually X11-compatible, but it's incompatible with OpenGenera because OpenGenera uses quite old X11 protocols.

Luckily, we can solve this using the pre-installed Xephyr X11 server in Ubuntu 17.10.

Create a script named launch\_genera.sh in your ~/lisp\_environment directory with the following content:

#!/bin/bash

Xephyr -br -reset -terminate -ac -noreset -screen 1280x1024 :3 &

DISPLAY=:3.0
export DISPLAY

./vlm_executable -coldloadgeometry 640x480+0+0 -geometry 1280x1024+0+0

Logging In and Defining Site

You should now see the initial world, which looks like this:

Important! If you input incorrectly, use the delete key to backspace, not backspace. This is standard Genera behavior!

The world you just started is an initial world. You can already do many things with it, but you'd better define a site and save it as a working world. That way, you can keep the initial world as backup in case you want to start over later.

Note that defining a site writes files to NFS, located under lisp-vlm:/var/lib/symbolics/sys.sct/. If you really want to restart, you need to delete /var/lib/symbolics and then re-extract the tar to that directory.

Logging In

When Genera starts, you should see a prompt like this:

Please login.
Command:

The first thing you need to do is log in. There's a special global user named lisp-machine, which is what we'll log in as.

At the Command prompt, type:

login lisp-machine

You'll see that after typing login, Genera prompts you for a username. This is just one of Genera's many user-friendly features.

Defining Your Site

Next, you need to define a site. At the Command prompt, type:

Define Site

Genera responds and automatically fills (site name), where you can enter your site name. The site name should be a single word (I use "Dragon" as my site name), then press enter.

Now, click on

Namespace Server Name: the name of the primary namespace server

Type lisp\_machine then press enter. Genera responds and adds multiple editable fields.

Click on the following field:

Unix Host Name: the name of the DEC-AXP host on which Open Genera is running

This time, enter lisp-vlm

After completing these two steps, click "END" (or press the End key) to save changes. Now you've defined your own site!

The dialog should look like this:

Now it's time to save the world.

Saving World

One-time Setup

After defining the site, the IP address for host "lisp_machine" is incorrect. It's set to the external IP address of the Linux host. You need to change it to 192.168.3.1 to avoid issues when reloading your saved world and prevent many troubles.

Type

Edit Namespace Object Host lisp_machine

Change the address to

INTERNET 192.168.3.2 tap0

Now type

Save Object

After the prompt appears, answer "Yes". Finally, type quit to exit editing and return to the top-level LISP listener.

You must complete this before saving the world, otherwise you won't be able to access the local network when restoring from the world.

Saving World

Now you're ready to save your working world.

At the Command prompt, type:

Save World

After pressing space, you'll receive the following prompt:

Save World (on file [default LISP-VLM:Genera-8-5.vlod])

That's the default world name; you can change it to anything. For example, I named my world LISP-VLM:Genera-8-5-working.vlod. After typing, press enter.

You'll be asked to confirm; type Y and press enter.

At this point, Genera will restart. But it's not ready for use yet! You need to shut down Genera, edit the .VLM file, then restart again.

Shutting Down Genera

Type

Halt Genera

You'll be asked for several confirmations. When prompted, confirm again whether you really want to quit Lisp (if the debugger window doesn't pop up, look for a hidden window and type Y there).

Now you're back in Linux. Next is editing the .VLM file.

Editing .VLM File

Open your .VLM file, find and comment out the line vlm.world: Genera-8-5-xlib-patched.vlod, then add this line:

vlm.world: Genera-8-5-working.vlod

(Modify according to your saved world filename)

At this point, you can restart Genera by typing ./vlm\_executable.

Congratulations! You're in a functioning Genera world—time to explore.

Useful Genera Knowledge

Shutting Down

To shut down, type

Halt Genera

You'll be asked for several confirmations. After Genera's window closes, you must confirm once more in the cold start window.

Keyboard Mapping

Symbolics' Lisp machines had remarkable keyboards filled with modifier keys.

When Symbolics released VLM, they realized there was a problem. DEC Alpha workstations used PS/2-style keyboards, so they had to map all the modifier keys to PS/2 keyboard codes somehow and build a converter box to use their keyboards with DEC hardware.

Thus, the following mapping was born. By following this mapping, you can use regular 101 or 104 PC keyboards. (Note: "KP" refers to numeric keypad keys)

The following lists common key mappings and shortcuts:

Select        F1
Network       F2
Function      F3
Suspend       F4
Resume        F5
Abort         F6
Super L       F7        s-
Hyper L       F8        h-
Scroll        F9
ClearInput    F10
Complete      F11
Help          F12

Local         Windows
Shift         Shift     sh-
Control       Ctrl      c-
Meta          Alt       m-
Return        Enter

Switching between activities:

Select(F1):
 =          select Key Selector
 C          Converse
 D          Document Examiner
 F          File System Maintenance Operations
 I          Inspector
 L          Lisp
 M          Zmail
 N          Notifications
 p          Peek
 Q          Frame-Up
 T          Terminal
 X          Flavor Examiner

Function(F3) S     Return to previous activity

C-M-y      Previous command, after entering command history mode, you can repeatedly press M-y to display previously executed commands

Scrolling

After some time away, I always forget how to scroll. It's actually simple:

Mouse Button	Description
Left button	Moves the line where the mouse cursor is to the screen top (scroll up)
Right button	Moves the current top-of-screen line to the mouse cursor position
Shift-left button	Moves the cursor line to the screen bottom
Middle button	Displays content corresponding to the percentage position of the cursor relative to the window

Keyboard scrolling is more convenient: F9 scrolls down, Alt-F9 scrolls up

Setting Who-Calls

When a site is defined, Genera automatically calls (si:enable-who-calls :new) for you. But we want to enable who-calls on all functions, not just new ones, so we must set it manually:

(si:enable-who-calls :all)

This takes a few minutes.

Finding Functions

To search all functions containing "crypt" in their names, simply type:

(apropos "crypt")

Who Calls

After completing the Who Calls setup above, you can find all callers of rpc::encrypted-password.

(who-calls 'rpc::encrypted-password)

Authentication

UNIX authentication seems to be defined in RPC::UNIX-AUTHENTICATION-MIXIN. See the RPC::USERNAME-AND-PASSWORD-VALID-P function specifically.

Here's the initial definition:

(defun-in-flavor (username-and-password-valid-p unix-authentication-mixin)
    (username password)
  (declare (values username-valid-p password-valid-p))
  (if (null username)
      (values nil nil)
      (multiple-value-bind (encrypted-password defaulted-p)
          (let ((*unix-authentication-allow-defaulting* t))
            (username->password (unix-name-lookup-access-path) username))
        (cond ((and defaulted-p
                    (not (string-equal username "anonymous"))
                    (not (string-equal username "lisp-machine"))
                    (not (string-equal username "nobody")))
               (values nil nil))
              ((or (zerop (string-length encrypted-password))
                   (and password
                        (string= (unix-crypt password encrypted-password)
                                 encrypted-password)))
               (values t t))
              (t
               (values t nil)))))))

We can hack this function to overcome the need for crypt-style passwords in NIS authentication. We can do several things here:

1. Implement MD5, SHA256, or SHA512 encryption
2. Add some type of no-op to hack (unix-crypt)
3. Figure out Genera's Flavor system and properly implement our own login mixin (the correct way?)

What I personally did was modify the function to let me log in with password "xyzzy". This is very silly, but it works. It's also an introduction to hacking Genera.

Here's my modification, note (string= "xyzzy" password)

(defun-in-flavor (username-and-password-valid-p unix-authentication-mixin)
    (username password)
  (declare (values username-valid-p password-valid-p))
  (if (null username)
      (values nil nil)
      (multiple-value-bind (encrypted-password defaulted-p)
          (let ((*unix-authentication-allow-defaulting* t))
            (username->password (unix-name-lookup-access-path) username))
        (cond ((and defaulted-p
                    (not (string-equal username "anonymous"))
                    (not (string-equal username "lisp-machine"))
                    (not (string-equal username "nobody")))
               (values nil nil))
              ((or (zerop (string-length encrypted-password))
                   (string= "xyzzy" password)
                   (and password
                        (string= (unix-crypt password encrypted-password)
                                 encrypted-password)))
               (values t t))
              (t
               (values t nil)))))))

Appendix A: Genera's XLIB Patches

The Genera-8-5-xlib-patched.vlod used in this tutorial is already patched, so this section is omitted.