Opionated tokens

The new version of opinionated workflows

In the SaaS era, one way to think about what people were really paying for when they purchased a SaaS product was an opinionated workflow. While a lot of software could at the end of the day be reduced to a CRUD app on top of a database, or a fancy version of something that could be done in a spreadsheet, immense value was created by simply being opinionated about the right set of interfaces and the right data model for a given user in a particular industry.

This mental model, for the most part, does not map well to AI products because of how quickly model advances alter the need for opinionated workflows. An obvious example of this is how rapidly model advances utterly upended coding workflows - the transition from autocomplete to next edit to agentic IDE to agent orchestrator to software factory being so rapid that any workflow innovation in one stage was obviated within 3-6 months. See also how the UX of most leading vertical AI startups - e.g. Harvey in law - is now virtually indistinguishable from Claude or ChatGPT.

To be clear - this does not mean there is not value in applied AI products. I just think it means we need a different framework for how to think about what people are really paying for when they buy an AI software product on top of base model.

Increasingly, the framework I use is that you are buying opinionated tokens. Specifically, you are willing to pay a surplus on top of the base models for an opinionated answer to - what is the right combination of models to use, in what scenarios, across what use cases, to achieve the optimal quality/cost/latency tradeoff for my use case?

It is extremely easy in most cases to get a v0 of a workflow using the baseline model products. You can run a financial due diligence in Claude rather than use Rogo. You can draft a contract with ChatGPT rather than use Legora. You can run a QA agent with Claude Code on Github Actions rather than use Factory. This, to me, maps very closely to how a lot of people could use case use a spreadsheet + google docs as a starting point for any given SaaS workflow.

Where the rub comes though is - what happens when this new AI workflow becomes your norm, and the cost starts being concerning, and the quality isn’t quite there in some scenarios. You might start to realize you can use Haiku here vs. Opus there, or that you can write a skill to sub-divide a given task into three tasks to make the system more reliable, but it is ultimately impossible for you as an individual (or corporation) to:

1. Benchmark all possible models

2. Against all possible use cases

3. In all possible combinations

4. With all forms of configuration (prompting, context management, fine tuning, etc)

5. At a weekly frequency given how quickly the model environment changes

Yet, doing the above well can in many cases result in order of magnitude improvements in token efficiency - or the overall cost/quality tradeoff of the system you are building. And thus, the ROI is clearly there to pay someone else to be responsible for this decision for you, particularly if this is a non “core” use case for your business.

This example from Factory is a good one - where the system being smart simply about model routing in a task gives you 60% relative cost savings for the same quality on various coding tasks

A good example of this amongst companies I work with is Extend, which is an AI document processing platform. It is undeniably true that in many cases, simply passing your document through a frontier model will do a not bad job of parsing or extracting the data you need. However, if you are building production document pipelines on top of complex documents, what you’ll invariably realize is that the juice you can squeeze from a more complex system is substantial. Extend’s platform offers an opinionated set of ways to combine parsing models, layout detection models, OCR models, vision models, frontier models, and similar to drive massive performance deltas vs. just using single frontier models.

The results are dramatic. Ultimately, what the customer is buying is the belief that it will be better for Extend to be responsible for token optimization in this category rather than them. The margin they pay on top of tokens to Extend pays for itself.

Extend Politax Split benchmark - an intelligent harness drives massive improvements over more naive base model choices

I thus think that the set of primitives that really matter for an given applied AI company that sits on top of a bunch of frontier models are:

1. Benchmarks & Evals - The tooling to measure & understand model performance across tasks in a given domain

2. Router - A system to dynamically route queries in that domain across different models

3. Ensembler - A system that composes sets of models to solve a given task more efficiently than any single model

4. Domain specific specialty models - Tooling to do domain specific training of models, especially niche specialty models that need to do one thing very well in that domain. Ideally this is coupled with a unique way to access domain-specific data or specialty reward signals you get from people using your product.

5. Domain specific model heuristics - Essentially, learned experience about the right general way of deconstructing tasks into models or sets of models in a given domain. This can look like prompt engineering, skills, or just system architecture - but it’s an implicit mapping of the domain to models.

The above come together to let you deliver opinionated tokens. I find this to be a much more useful and helpful way to think about building applied AI businesses than workflow differentiation.

I also find that this is increasingly the right way to sell such products - virtually every application layer AI company we work with increasingly gets compared to an in-house DIY with base model products, and if you can’t clearly demonstrate why your opinionated tokens drive much higher ROI than the naive solution, you won’t close the customer, even if your workflow is more opinionated and domain specific. It is too easy to work around those sorts of minor deltas given how the flexible base model products like Claude CoWork are.

But what is undeniable is if you show someone how, for a use case where they are trying to make it work with a Claude Skill, you can give them 30% higher quality at 50% lower cost by composing tons of models together, and that the right way to do that today is different than it was even 2 weeks ago. So either they can spend their time on these types of weekly optimizations, or they can let you.

While it is true that other things can matter to differentiate applied AI companies - such as domain-specific integrations, specialty business models, or secondary non-AI functionality - I think that if you don’t have opinionated tokens at your core, it will ultimately be very difficult to win long term.