Ubik Devlog #3

Welcome to the third devlog! If you’re new here, Ubik is a distributed bug tracker I’ve been working on. It’s an experiment in trying to take features from centralized services like Github and making them distributed in the spirit of Git itself. This time, I have updates on evolving the Checks feature that I introduced last time and a small adjustment to refreshing data while Ubik is running.

Multiple checks per commit

Tests aren’t the only thing that can run in your CI pipeline - you might run a code style checker, a security vulnerability analysis tool, etc.

To reflect that reality, Ubik now supports running multiple checks per commit. Once the checks for a commit have finished, you can expand the logs for each check to dig into the details. Checks can also be marked “optional”, meaning the commit won’t be marked as failed if that check fails.

It doesn’t look like much, but I think it provides a good opportunity to talk about how the Bubbletea library works.

A Bubbletea primer

Ubik is based on Charm’s Bubbletea library:

The fun, functional and stateful way to build terminal apps. A Go framework based on The Elm Architecture. Bubble Tea is well-suited for simple and complex terminal applications, either inline, full-window, or a mix of both.

The main idea of the “Elm Architecture” on which Bubbletea is based is that what you see on the screen should be directly derived from the application’s state. That’s achieved through the use of three concepts:

  1. The “Model”, where the application’s state is stored.
  2. The “View”, which uses the data stored in the Model to determine which characters should be drawn in the terminal.
  3. An “Update” function that mutates the model and triggers the View to update.

If you’ve used React and Redux, the concepts may be familiar. There are a couple other Bubbletea specifics that will be important for explaining the Checks feature, so let’s orient ourselves by looking at a simple Bubbletea program:

package main

import (
	"fmt"
	"os"

	tea "github.com/charmbracelet/bubbletea"
)

// This program will print a string to the screen that displays the current count. You can increase or decrease the number by hitting "+" and "-", or quit the program by pressing "q".

type Model struct {
	count int
}

// We're not using this at the moment, but Init returns an initial command for the application to run
func (m Model) Init() tea.Cmd {
	return nil
}

func (m Model) View() string {
	return fmt.Sprintf("The current count is %d", m.count)
}

func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
	switch msg := msg.(type) {
	// was the msg a keypress?
	case tea.KeyMsg:
		// which key?
		switch msg.String() {
		case "+":
			m.count++
		case "-":
			m.count--
		case "q":
			return m, tea.Quit
		}
	}

	return m, nil
}

func main() {
	p := tea.NewProgram(Model{count: 0})
	if _, err := p.Run(); err != nil {
		fmt.Printf("Error: %v", err)
		os.Exit(1)
	}
}

Here it is in action.

The Model and View are fairly straightforward; a place to put state and the string that gets displayed in the terminal. The Update method is where things get interesting. From the README:

The update function is called when ”things happen.” Its job is to look at what has happened and return an updated model in response. It can also return a Cmd to make more things happen.

In our case, when a keypress happens, Bubbletea calls our Update method with a tea.Msg that contains information about that keypress. The method looks at which key was pressed, updates a copy of the model, and returns the new model. You can see in the case of hitting q, we’re returning the same model alongside a tea.Cmd to make the next thing happen, which is to quit the application.

A tea.Cmd is just a function that returns a tea.Msg, which the Update method will consume. You can define your own commands and messages:

type incrementMsg string
type decrementMsg string

func incremented() tea.Msg {
	return "incremented!"
}

func decremented() tea.Msg {
	return "decremented!"
}

func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
	switch msg := msg.(type) {
	case incrementMsg:
	  // do something with the message
	case decrementMsg:
	  // do something with the message
	case tea.KeyMsg:
		switch msg.String() {
		case "+":
			m.count++
			return m, incremented
		case "-":
			m.count--
			return m, decremented
		case "q":
			return m, tea.Quit
		}
	}

	return m, nil
}

Bubbletea will take every tea.Cmd that you return from Update and run them in their own goroutine. This means we can keep using the application while computation is happening in the background (e.g. file or network I/O). When each tea.Cmd is finished, Bubbletea calls our Update method with the returned tea.Msg. With that, we should now have what we need to understand how the Checks feature works.

Using Bubbletea to build the Checks feature

What needs to happen:

  1. Given a list of checks for a commit, Ubik should run each check concurrently in the background. The check will be marked as “running” while the check is executing. It’s important to note that the check should execute in the context of the codebase at a specific commit, so it can’t just run on the current working directory. We’re trying to determine if a given commit is safe to ship to production, not if the latest state of your filesystem happens to produce passing tests.
  2. Output from the check’s shell command needs to be captured so that we can look back at the logs and determine what went wrong in the case of a failure.
  3. Once a single check is finished, its status should be reported in the UI.
  4. Once all checks for a commit have finished, the commit should have an overall status that reflects the status of its individual checks.

The first one will be the most interesting to discuss here.

Running checks in the background

To keep the app running smoothly while checks are performed in the background, we’ll use a tea.Cmd, which we learned about earlier. Let’s return one when the key to run checks is pressed:

func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
	// ...
	switch {
	case key.Matches(msg, keys.RunChecks):
		commit := // find the commit from the list
		var cmds []tea.Cmd
		// the checks returned from commit.NewChecks have
		// a "running" status by default. This is only
		// updated once we have the result of a check
		checks := commit.NewChecks()
		commit.LatestChecks = checks

		for _, check := range commit.LatestChecks {
			cmds = append(cmds, RunCheck(check))
		}

		// tea.Batch tells Bubbletea to take a collection of commands and
                // run them each in their own goroutine.
		return m, tea.Batch(cmds...)
	}
}

func (m Model) View() string {
	// there's a bunch of manual string-building in this method
	// where we conditionally render a "running" status for the
	// commit if not all of its individual checks are finished
}

To run the check itself, we need to figure out how to ensure it executes in the context of a particular commit. My first attempt at this used git worktrees to create a snapshot of the repo’s working directory for a commit. This worked well enough, but it’s not the intended use-case for worktrees; it’s a tool meant to make working on multiple branches at once easier. From the docs:

A git repository can support multiple working trees, allowing you to check out more than one branch at a time. With git worktree add a new working tree is associated with the repository, along with additional metadata that differentiates that working tree from others in the same repository. The working tree, along with this metadata, is called a “worktree”.

Worktrees come along with extra metadata and git refs that we don’t need, that need to be cleaned up when a check is finished. I wanted a cleaner solution, and it turns out git has one: git archive. It creates a compressed snapshot of the working directory without git metadata - it’s just the working directory as it existed at a specific commit. Here’s some of the code in Ubik that makes that work:

// our custom tea.Msg
type checkResult Check

func RunCheck(check Check) tea.Cmd {
	return func() tea.Msg {
		result, err := executeCheckUsingArchive(check)
		check.Output = result
		check.FinishedAt = time.Now().UTC()
		if err != nil {
			check.Status = failed
			return checkResult(check)
		}
		check.Status = succeeded
		return checkResult(check)
	}
}

func executeCheckUsingArchive(check Check) (string, error) {
	// create a temporary directory
	tempDir, err := os.MkdirTemp("", "check-archive-")
	if err != nil {
		return "", fmt.Errorf("failed to create temp directory: %w", err)
	}
	// make sure to clean it up after we're done
	defer os.RemoveAll(tempDir)

	archiveCmd := exec.Command("git", "archive", "--format=tar", check.CommitId)
	archive, err := archiveCmd.Output()
	if err != nil {
		return "", fmt.Errorf("failed to create archive: %w", err)
	}

        // extract the archive into the temporary directory
	extractCmd := exec.Command("tar", "-xf", "-")
	extractCmd.Dir = tempDir
	extractCmd.Stdin = bytes.NewReader(archive)
	if err := extractCmd.Run(); err != nil {
		return "", fmt.Errorf("failed to extract archive: %w", err)
	}

	// the command (e.g. "go test", "rake") runs in the context of the extracted git archive's directory
	check.Command.Dir = tempDir
	output, err := runCommandWithOutput(check.Command)
	if err != nil {
		return output, fmt.Errorf("command execution failed: %w", err)
	}

	return output, nil
}

As you can see, RunCheck returns a tea.Cmd which will run our check in a goroutine. That tea.Cmd returns a checkResult message which can then be handled in the Update method.

I spent a good deal of time getting multiple checks working as expected, but now I feel like there’s a solid base on which to build out some other ideas that I’m excited about. I think Checks are going to be a core part of Ubik and enable interesting new ways to streamline the process of building software by yourself or with a small group of peers.

Refresh on focus

In the future, you might use Ubik alongside a git client like lazygit to stage changes, switch branches, or commit. Since checks operate on git commits, Ubik needs to be aware of changes that happen to the git repository while it’s already running.

I thought about using or implementing a file watcher that would refresh Ubik’s state when the git repo changed. Lazygit went through this same thought process, but opted for a much simpler approach: when the terminal pane that is running lazygit is focused, lazygit re-queries the git repository for its data and refreshes the UI. No need for a persistent process to watch for changes. Using the same approach in Ubik required a tiny addition to the Update method thanks to Bubbletea’s recent addition of focus/blur support:

func (m Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
	switch msg := msg.(type) {
	// fires when the terminal pane is focused
	case tea.FocusMsg:
		// getIssues and getCommits are tea.Cmd functions that
                // fetch data from the git repo
		return m, tea.Batch(getIssues, getCommits)
	}
}

Back to work - see you in the next one!

Last updated:

← Back