Discussion about this post

User's avatar
Mithun Madhusudan's avatar

Great read as always. Context is king - and its now a question of how that context gets passed to a model most efficiently (harness engineering or tuning at the edge?)

David Wilkens's avatar

Own your workflow.

Orchestration, durable execution and work flow management is key. Models are just an input that need to be optimized for the job. In the long run will it matter where the token comes from in the same way that we don't care where an electron comes from? The ability to efficiently match a model to the job isn't quite there yet, someday it will and maybe it wont matter.

It would seem that open-source models will eventually climb the S-curve and be good enough for most applications. They just haven't had $100b to throw at the RL that the closed models have had. Our view of the horizon has gotten so short that we think the SOTA models will be necessary for everything. I can't imagine this will continue to be the case. The economics will dictate it and there will be a relentless reduction in the cost of tokens for inference.

Something else to consider is that a tax will come for tokens to account for the payroll taxes that will make a human worker less competitive than the future agent or for the agent that is doing work where a human operator is not available because of the demographics.

No posts

Ready for more?