Introduction

Lately, I have been wanting to finally wanting to dive deep into a dynamic language. In the past, I have written a little about Ruby because of Ruby On Rails. While I do like Ruby on Rails, I am not exactly a huge fan of Ruby. It is not bad, but it will still missing something. I have heard quiet a bit about Elixir, but never thought about it until recently. It is a dynamic language like Ruby, but with more a more functional nature. There is also some overlap in the Rust and Elixir fans. So I decided to give it a go being since I do like Rust and Swift, with Swift having some overlap in language features and concept with Rust. Also, I had heard some great things about Phoenix, a web framework inspired by Ruby on Rails, but in Elixir.

At first glance, Elixir looked pretty nice. But you can’t really get a feel for a language until you actually start to use it for something. So, being as there is homebrew written in Ruby, I decided to embark on a project to make a little system package manager in Elixir.

Why a package manager?

First a foremost, it has been something I’ve wanted to try my hand at for a long time. Since I was a young lad trying to bumble my way through LFS (Linux From Scratch). Outside of just doing apt install or yum install or pacman -Ss, I hadn’t looked too deeply in package management at the time until LFS and also subsequently checking out Slackware. But that was many moons ago. But, seems like a fun project for Elixir.

Also, as noted, I have a little experience packaging software. In the past, I packaged, or wrote a port in the ports tree, for a few packages in Ravenports (I have some articles about this). Not too long ago, I packaged comtrya for homebrew. You can learn a lot about software development and software by packaging. Just seems like the next level to go for learning more about packaging and distributing software on UNIX-y systems is to now write my own package manager.

In comes spellbook. A witch-y theme package manager written in elixir.

What do I want it to do?

Alright, so now, what does it need to do? Well quiet a bit of it, for anyone with experience using a system package manager, is obvious. It needs to install, uninstall, and search for software. That is kind of the low hanging fruit, so I won’t spend much time on that. But specifically how do I want it to do these things?

Right now, I have mostly only thought through the installing. I want to have a ports tree. For those not familiar, a ports tree is a directory of recipies for building packages from source. These recipies cover things like downloading a package’s source code and then building it from source. So right now, I am ignoring installing binaries from a repository or grabbing binaries from github or other source. I am putting that off to later. So right now, it will be limited to building from source.

Okay, homebrew kind of does this! First, so what. Technically this is a “competitor” with homebrew, but I am mostly doing this out of fun and for learning. But, there is a future feature that I want to support that it doesn’t look like homebrew has. But, it is a feature of Ravenports. As a side tangent, I would love to use Ravenports on macOS (my daily driver now), but it can’t support any newer versions because Ravenports relies on nullfs, which is no longer very viable on macOS. At least that is what I remember when I was looking into a few years ago. But that feature is the concept of variants.

Variants in Ravenports are like supporting multiple versions of the same package, but with different feature sets. For those familiar with ebuilds, portage, and gentoo, variants in Ravenports is sort of like customizing your FLAGS. For an example for those not familiar, lets take a package like git. If you build git in the most basic way possible, you get the full package. But if you change some build variables, you can get a lighter version of git. In Ravenports, you can get the git-full or git-lite package. Its your choice. Another example is less. You can build less so it uses colors in it’s output, or for less to be colorless. I want to eventually support this idea of variants. But that is a future feature.

What does a port look like

Before going further, lets take a look at a current working sample of a port definition. Or, as it is called in spellbook (a package/port), a spell definition.

defmodule Htop do
  @behaviour Spellbook.Spell

  @impl true
  def name(), do: "htop"
  @impl true
  def version(), do: "3.4.1"
  @impl true
  def homepage(), do: "https://htop.dev"
  @impl true
  def license(), do: "GPL-2.0"
  @impl true
  def type(), do: "bin"
  @impl true
  def checksum(), do: ""
  @impl true
  def deps(), do: []
  @impl true
  def source(), do: "https://github.com/htop-dev/htop/archive/refs/tags/3.4.1.tar.gz"

  @impl true
  def install(args) do
    System.cmd("sh", ["autogen.sh", "--prefix=#{args.prefix}"], cd: args.cwd)
    System.cmd("sh", ["configure", "--prefix=#{args.prefix}"], cd: args.cwd)
    System.cmd("make", [], cd: args.cwd)
    System.cmd("make", ["install"], cd: args.cwd)

    :ok
  end
end

Each spell is a module and it conforms to the Spellbook.Spell behavior, which is sort of like an interface. Quiet a bit of it is just package information like the name, version, license information, and where to get the source.

Then a spell definition needs to implement an install/1 function, these are the build instructions for building the software.

Building the port

To build a port right now, a function is ran that does a few things. First, it creates the ‘environment’ for building. It does this by creating a temp directory and downloding the package there, then it extracts it’s contents. Afterwards, it will construct some environment variables and also create the information that will be injected into install/1 via the args parameter, then the method is called. The method executes it’s instructions. Returning from that, need to ‘install’ it into the right place(s). So this would be some bin folder as an example, which it does by iterating over the build artifacts and creating symlinks to the appropraite spots. That is atleast the tldr of it. So far at least.

Output from a build

 ~/ElixirProjects/spellbook/ [main*] ./spellbook cast ~/ElixirProjects/StandardBookOfSpells/htop.exs
Casting spell: /Users/toddmartin/ElixirProjects/StandardBookOfSpells/htop.exs
Warming up to perform cast...
Casting /Users/toddmartin/ElixirProjects/StandardBookOfSpells/htop.exs
Build directory: /var/folders/vf/7r2g86991c788ppg6zq147r80000gn/T/htop-3.4.1
Now on to something else
Downloading https://github.com/htop-dev/htop/archive/refs/tags/3.4.1.tar.gz to /var/folders/vf/7r2g86991c788ppg6zq147r80000gn/T/htop-3.4.1/3.4.1.tar.gz

22:13:14.923 [debug] redirecting to https://codeload.github.com/htop-dev/htop/tar.gz/refs/tags/3.4.1
Download complete
Extracing /var/folders/vf/7r2g86991c788ppg6zq147r80000gn/T/htop-3.4.1/3.4.1.tar.gz....
Extraction complete: /var/folders/vf/7r2g86991c788ppg6zq147r80000gn/T/htop-3.4.1
autoreconf: export WARNINGS=all
autoreconf: Entering directory '.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: configure.ac: not using Intltool
autoreconf: configure.ac: not using Gtkdoc
autoreconf: running: /opt/homebrew/Cellar/autoconf/2.72/bin/autoconf --force
autoreconf: running: /opt/homebrew/Cellar/autoconf/2.72/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
autoreconf: Leaving directory '.'
Successfully installed htop
Linking...
[lib/spellbook/linker.ex:12: Spellbook.Linker.link_spell/2]
base #=> "/opt/spellbook/Spells/htop/3.4.1"

Creating symlink /opt/spellbook/Spells/htop/3.4.1/bin/htop -> /opt/spellbook/bin/htop
Creating symlink /opt/spellbook/Spells/htop/3.4.1/share/man/man1/htop.1 -> /opt/spellbook/share/man/man1/htop.1
Creating symlink /opt/spellbook/Spells/htop/3.4.1/share/pixmaps/htop.png -> /opt/spellbook/share/pixmaps/htop.png
Creating symlink /opt/spellbook/Spells/htop/3.4.1/share/icons/hicolor/scalable/apps/htop.svg -> /opt/spellbook/share/icons/hicolor/scalable/apps/htop.svg
Creating symlink /opt/spellbook/Spells/htop/3.4.1/share/applications/htop.desktop -> /opt/spellbook/share/applications/htop.desktop
Done casting...

As shown above, the build artifacts are in /opt/spellbook/Spells/htop/3.4.1, following a conventions of /opt/spellbook/Spells/<PACKAGE_NAME>/<VERSION>. When symlinking, it will see that there is a bin directory in /opt/spellbook/Spells/htop/3.4.1/bin. It will iterates over the files there and create symlinks in /opt/spellbook/bin.

Going full witch-y with it

Here is a the help menu from spellbook.

 ~/ElixirProjects/spellbook/ [main] ./spellbook
Magical system package manager 0.0.1
Todd Martin
A magical system package manager

USAGE:
    spellbook
    spellbook --version
    spellbook --help
    spellbook help subcommand

SUBCOMMANDS:

    cast            Cast (install) a spell (package)
    dispel          Dispel (uninstall) a spell (package)
    scry            Search for a spell
    grimoire        List casted spells
    reveal          Reveal information about a spell
    renew           Renew your spellbook shelf
    empower         Upgrade a casted spell

As shown above, to install we cast. To list all casted spells, we use the subcommand grimoire which will list all of our casted spells.

 ~/ElixirProjects/spellbook/ [main] ./spellbook help cast
Cast (install) a spell (package)

USAGE:
    cast SPELL
    cast --version
    cast --help

ARGS:

    SPELL        Package

And this is what the /opt/spellbook directory looks like.

 /opt/spellbook/ ls
bin     etc     lib     sbin    share   Shelves Spells

Conclusion

This is still very early days with a lot more left to implement. But all in all, I am enjoying my time with Elixir and as I get further in, I plan to post more about this project. And as I ‘live’ in Elixir a little but more, I will post more about my experience with that. Afterall, it is hard to really know if you like something until you’ve lived in it for a bit.