The ideal operating system
Further musings on Lisp machines
So I don’t know if this is a flex, but I’ve used a lot of operating systems. I’m not talking about Linux distros, I played around a little and settled on Debian, I’m talking about weird experimental stuff you run on a ThinkPad or in a VM. Weird operating systems are something I love playing with and thinking about. In fact, my most-read post on this blog is about one of my musings.
In that post (which you should read as background into how I’m thinking about all this), I discuss why I think our modern UNIX systems are inadequate and why LISP machines could be a viable alternative. Now in that post, I don’t go over too much of how it would look to the user, and over the last several weeks I’ve been thinking about that and I think I have a decent, theoretically viable idea that could be the next big leap in computer software.
So let’s start at the bottom of the stack and work our way up. Let’s say we have a kernel that runs on anything that gives us a LISP machine, now for the sake of this post it’s a microkernel and all of it (except part of the kernel, you nerds) is written in LISP and the only thing it can and will ever have to run is lisp programs. With this hypothetical kernel x, we can start building the LISP user space
Now I didn’t touch on this too much in my first LISP post, but I hate files. Well fine, I hate local files, all we do is copy them, duplicate them, accidentally delete them, and manage them between devices. This is in a world where we have 10 gig networking. I could go on and on about how much time I waste at my job moving files, but you know. I’m willing to bet most of the people who will read this are gainfully employed somewhere where moving files around is a headache. Sure in we have SMBs and git for helping manage it, but it hardly solves the problem, many data centers have machines with no local storage but often that's a game of tricking your OS into thinking the storage is local. Let’s scrap all of it, this new 21st-century, user space doesn’t need local files and it’s not going to have them.
So how would data storage work? Well to look at that we’re not going to look at file storage we’ll be looking at block-level storage, Just keep that in mind as we move through this section. I want us to look at a rather obscure piece of software called plan9.
For the uninitiated, Plan 9 is an operating system written by the OGs of Unix who got tired of UNIX before anyone else did. The idea is to instead of having a network of UNIXes make the network its own UNIX. If that sounds confusing, I’d recommend reading more about it, sadly I don’t have the space to get into it here.
A major part of this was centralizing all the files into file servers and one of the later servers, designed purely for archival purposes, lives rent-free in my head. venti is a hashed-based archival file server. https://9fans.github.io/plan9port/man/man7/venti.html for more on that. Now for our new system, I think this is ideal and eliminates a lot of real-world issues and could be easily abstracted as a traditional file system for compatibility or if simply best fits a given use case, but as we will see later on I don’t think traditional files are ideal in our new user space.
Now the hash file system idea is where this started, but with Venti it’s restricted to a local network or a centralized server (that could go down) and if we abolish local storage we need the network storage to be accessible anywhere. So what if we had a global, decentralized, block/hash-based storage system…. Well, we do and while I don’t think it’s a perfect piece of software I think it’s a great jumping-off point.
I think we need to eliminate files as an abstraction for regular users, especially in this new completely interconnected operating system world. I think there is a good option that is already emerging in its own way and when applied to an operating system and the computer as a whole could take its usability to the next level.
Enter IPFS (Interplanetary File System), the proof that my ideas for file systems aren’t completely insane. Now IPFS has its flaws, particularly how the company tries to push its separate Filecoin network, but IPFS as a concept is, in my opinion, the future of data storage.
If we could picture an IPFS where users and companies might run their own hosts (through the incentive of having local caches for faster speeds) and built it around the concept of diskless machines we could build a global system where any computer can be your home computer, data is shareable between people without the complexity of cross-device or network file management and an open system where we don’t have to worry all too much about storage capacity (I can go into my theory on why in a different post, this one is already running long).
Now you may be wondering how we go about building an interface built with a global file network in mind. I think it would call for a radical transformation in how the computer would operate, one that would represent data in a completely new way, and would enable the user to be as productive as possible.
I'm gonna call it a Notebook User Interface, people who use applications like Notion, and Org Mode will probably see what I’m thinking here. It’s an interface built on large interconnected notebooks and with something like Notion, you can make collaborative notebooks that all integrate into one workspace. This makes applications like Notion incredibly powerful just as a notes app and a task manager. Now imagine you can run code, embed applications, and host “websites” all in one interface that is built into the OS.
Now think about that, let your mind run wild with the possibilities this could bring, sure some apps will need to be full screen, but with this model most apps, programs, and websites can work as if they are one seamless interface because it is. With the underlying global filesystem, your data and the data published by others are on a level playing field. This interface let’s that be expressed in a fast and productive for the users casual and advanced.
When you combine all of this your computer becomes a terminal and processing engine for your “Notebook”. You can easily share the documents and use encryption as access control. The interface is already proven to be incredibly powerful interface and if properly integrated with a global filesystem and powerful Lisp-based system could change the way computers work and truly be a modern operating system.
So yeah that’s my latest set of operating system ideas. Hope you enjoyed and I’d love to talk to any and all of y’all about this on Twitter or in person if you're in the Birmingham AL area. Happy hacking!