back

Richer AI Interfaces

More expressive responses from LLMs

exploration artefacts

@gumbee/agent
A powerful agent framework with rich widget support
@gumbee/structured
Progressively parse a structured JSON with a flexible schema

The way AIs respond today feels unnaturally biased towards text. And from a technical standpoint, it makes sense. The path of least resistance for an LLM is to generate markdown and for us to style it as best as possible.

While tinkering with my custom AI browser, I’ve been thinking a lot about what it would mean to make AI responses richer. Not just longer or more detailed, but more expressive. Here you can explore a short demo of the exploration.

Requirements

When building the rich interface system, I had a few requirements in mind.

Design fit
The system needs to be able to conform to our product's design language.
Ease of use
The system should make it easy for us to create rich interfaces for different use cases and domains.
Scalability
The system should be able to scale while remaining cost efficient and performant.
Reliability
The system should be able to deal with the non-deterministic nature of LLMs.

See It In Action

Here are two short demos for different use cases to explore the capabilities.

Choose a Use Case
Loading ChatGPT Demo...

What’s Next

I’m still working on ways to imbue the LLM with more creativity and thoughtfulness when it comes to structuring the response and assembling the widgets. That includes coming up with good widget sets that expose lots of degrees of freedom.

I’m also curious if LLMs can produce better results by generating JSX trees directly, rather than JSON that we later map to the DOM. While I wouldn’t want to expose the full complexity of the DOM (e.g by generating raw React components), leveraging the LLM’s extensive training on JSX is an interesting path to explore.

If you found this interesting and have some thoughts, reach out. I’m curious to hear your feedback and/or ideas.