← Back to Kevin's newslettersPublished: 2023 September 3

I’m traveling this month! Hit me up for a coffee/beer conversation in:

This newsletter in two parts: software/UI, then hardware/electronics.

Interfaces for prototyping hardware

While prototyping scientific hardware, I often find myself simultaneously iterating on both the domain model and its corresponding user interface.

For example, a software-controlled pump might start with “on/off” control:

struct Pump {
  is_running: bool
}

Exposing this data model with an HTML <button> or <input type="checkbox"> is enough for initial hardware testing.

Say the first experiments show that switching the pump on/off is too violent (the sudden start/stop shears cells or explodes pipes or whatever), so we add target speed and acceleration parameters:

struct Pump {
  target_speed: Quantity<AngularVelocity>,
  max_acceleration: Quantity<AngularAcceleration>,
}

These new fields can be exposed in the interface as numeric sliders or text inputs which accept exact values of various units, e.g., “600 rpm” or “5 radians/sec”. This enables more quick benchtop experiments to determine reasonable pump speeds.

Perhaps we then add some sensors to our apparatus to measure the current pump speed and system pressure, which enable us to write closed-loop control algorithms. Only a single control algorithm can be active at a time, so we model them as variants of an enumeration:

struct Pump {
    current_speed: Quantity<AngularVelocity>,
    current_pressure: Quantity<Pressure>,
    control: FlowControl,
}

enum FlowControl {
    ConstantSpeed {
        target_speed: Quantity<AngularVelocity>,
        max_acceleration: Quantity<AngularAcceleration>,
    },

    ConstantPressure {
        max_pressure: Quantity<Pressure>,
    },

    ConstantFlow {
        target_flow: Quantity<VolumeRate>,
    },

    Brake,

    FreeSpin,
}

A user interface might show the read-only current speed and pressure as numbers, perhaps next to line charts visualizing the last few minutes.

The flow control mode could be specified via a drop down (<select>) or tabs, with the variant-specific fields appearing after selection. Alternatively, if we prioritize discoverability over saving screen space, we could show all of the modes (and their associated field controls) at once, with radio buttons to select the active mode and gray out the others.

When I’ve done this sort of data model iteration in the past, I’ve just manually rewritten the UI to match. That usually means duct-taping together something in a web browser to round-trip the appropriate Serde JSON representation with the backend system.

Handwritten UIs allow for maximum flexibility: I can choose (or custom-build) the most appropriate input controls for each field, adding additional domain context like min/max speed validation, slider step sizes, etc.

However, such UI takes a ton of time to develop and usually incurs coordination costs (as the UI and data model are often implemented by separate people and across tech stacks / deployment mechanisms).

I’m curious about other points of the tradeoff space here, especially in the context of “internal tools” where the implementers are also the end-users. Such author-operators are jointly optimizing “ease of use” with “ease of implementation”; they might prefer an ugly, “good enough” UI built in 10 minutes over a “bulletproof” one built over a week.

Generating UI from a data model

A potential approach to quick, “good enough” internal tools is to generate UI automatically from an underlying data model.

I’ve seen this done in a few places:

However, while there’s plenty of prior art on the idea of generating UI from data definitions, empirically it’s not something I actually do.

What properties would I find compelling for prototyping scientific hardware?

UI implementation thoughts

Most of my scientific hardware work has been in Rust, as it is suitable for full Mac/Windows/Linux systems, microcontrollers, and can be used for low-jitter applications like real-time control loops.

The standard Rust field definitions with the Units of Measurement crate’s types:

struct Apparatus {
    current: Quantity<Velocity>,
    target: Quantity<Velocity>,
}

don’t sufficiently specify the constraints and units our ideal UI would support. We could try and encode such things with macros:

struct Apparatus {
    #[read_only, display(meters_per_second)]
    current: Quantity<Velocity>,

    #[min("0 m/s"), max("10 m/s"), display(meters_per_second)]
    target: Quantity<Velocity>,
}

which is the approach taken by crates like serde_valid.

The advantage of this approach is that it might neatly into Rust ecosystem tooling. Ultimately that’s an empirical question, and I’m not sure whether one (read: me) could generate the validation and UI code at compile time without devolving into an mess of macro and build.rs cleverness, with all the associated obtuse error messages and poor IDE support.

And macros strike me as the only sensible expressive-but-concise route within Rust’s code generation facilities — I shudder to think of designing generic-filled type machinery like Quantity<Dimension, MinValue, MaxValue, DisplayUnit> (and then asking people to use it!)

There may be a path using NewTypes — following “parse, don’t validate” and replacing Quantity<Velocity> with some kind of BoundedQuantity<Velocity> that guarantees the 0–10m/s constraint, but it’s not clear to me exactly what that would look like. I also worry about the compile-time costs (generating unique types per field) and backend ergonomics (having to write state.some_field.value.0.get::<Thing>() to extract the value you actually care about from the types).

We can, of course, step outside of Rust’s facilities and write a program that “compiles” our data model into the necessary Rust and UI (HTML/JavaScript) code. Defining our own lil’ DSL would maximize expressiveness, though with some initial cognitive overhead compared to “vanilla” Rust.

In this case, I’d be inclined to explore building on something like Clojure’s Malli library, which does validation, error messages, and can even generate conforming data (useful for checking that a schema actually (dis)allows what you intended; not to mention load and visualization testing):

(-> [:map
     [:id int?]
     [:size [:enum {:error/message "should be: S|M|L"}
             "S" "M" "L"]]
     [:age [:fn {:error/fn (fn [{:keys [value]} _] (str value ", should be > 18"))}
            (fn [x] (and (int? x) (> x 18)))]]]
    (m/explain {:size "XL", :age 10})
    (me/humanize
      {:errors (-> me/default-errors
                   (assoc ::m/missing-key {:error/fn (fn [{:keys [in]} _] (str "missing key " (last in)))}))}))
;{:id ["missing key :id"]
; :size ["should be: S|M|L"]
; :age ["10, should be > 18"]}

Beyond model and UI definition, there’s also a question about how this’ll work at runtime: Should it be more like a “library” (a pile of functions to be called by your code) or a “framework” (which defines an entire lifecycle that calls your code)?

Arduino sketches are a wonderful framework: You write setup() and loop() functions, then some Magic Arduino Stuff happens and your code is running on a microcontroller.

We’d need some kind of start_webserver() call to kick things off, with platform specific variations (threads vs. async runtime, std. vs. no-std, etc.). Then maybe all we need are two channels (message queues): Incoming<Command> and Outgoing<Status>, with the Command and Status types being rendered in the UI as controls and data visualizations, respectively. The API contract is essentially “we’ll render in the UI and log any statuses you give us, and we’ll pass along any valid commands from the UI”.

Having the author reify a distinct Command type (rather than generating such types automatically for every non-read-only struct field) may be a bit more verbose, but it strikes me as better to make things explicit using Rust’s existing concepts rather than introduce new options. For example, one could specify whether the UI should support setting multiple fields at once by making all of the command fields optional:

struct PumpCommand {
    speed: Option<Quantity<AngularVelocity>>,
    control: Option<FlowControl>,
}

or whether only a single field can be set at a time:

enum PumpCommand {
    SetSpeed(Quantity<AngularVelocity>),
    SetControl(FlowControl),
}

Having this level of detail may be required to appropriately expose the underlying hardware constraints.

Overall, while there are lots of details to sort out, I suspect this architecture would have been sufficient support all of the hardware prototyping needs I’ve run across so far in my career.

I’ll have a go at building something and let you know how it works out in practice.

Other UI solutions

On hardware

The space of “prototyping hardware” ranges from:

I’ve multiple friends wish for something between bare PCB breakout boards and full-on PLC hardware. I’m sympathetic: It’d be great to have something more convenient and robust than a mess of wires on a breadboard without having to drop thousands of bucks for an electrical cabinet full of factory-grade components. Especially since PLCs are “just” microcontrollers made robust in a plastic housing with extra protection circuitry, a bit of networking (fieldbus), and software.

Using the koji fermentation controller as an example, the author spent about $500 for a Click PLC with a few GPIOs and temperature sensors. Why that instead of just reading an $8 K-type thermocouple breakout with a $5 microcontroller? They write:

In summary, we use PLCs because they are:

  1. Extremely rugged
  2. Customizable, modular and easily repaired or replaced
  3. Serviceable and programmable by any competent technician

However, PLCs are no more than simple, real-time computing systems. Your average PLC is probably weaker in raw compute power than a Raspberry Pi, and multiple times the cost.

How much of this physical robustness, modularity, and system-level convenience can be engineered at a cheaper price point? Is it simply “wrapping” cheap breakout boards with:

?

I mean, yeah, I’d love something convenient like Click or ODot components for $101 rather than $102.

If this were feasible, why haven’t SparkFun, SeeedStudio, or Adafruit done it? Perhaps the hardware margins just aren’t there. Or those companies aren’t interested in developing the fieldbus and UI necessary for the value proposition.

It may be doable as an open source labor of love — like a moth to the flame, I can’t help but sketch it out.

Requirements for a “hobbyist PLC” system

Nice to haves / open questions

I’d be curious to hear about other prior art in this broad space. I only learned about PLCs after years in Arduino “maker” world, and I wouldn’t be surprised if I’m missing entire other adjacent communities of practice.

There seem to be plenty of electrical engineers designing modular PCBs (e.g., Hexabitz, PURE modules) but I’m not aware of any attempts that include PLC-like physical enclosure and fieldbus standards.

The closest “open source 3d-printable” physical standard I’ve seen is the Gridfinity storage system.

Roadmap

The ideas I’ve discussed above seem (to me, anyway) tantalizingly doable. There’s still a ton of details to work out, but in broad strokes I suspect the technical core could be built in a month or two of focused work:

Please convince me not to start this.

Misc. stuff