Really excited to share our first benchmarks @ThriveHoldings: individual tax prep and helpdesk ticket resolution. The first two in a larger suite based on real world workflows across our businesses.
I like to think about our product philosophy as “1000 evals.”
Evals are how we actually have a POV on what the models are doing well for our operators and where the product needs to improve.
The problem is what "good" looks like keeps moving as models get better and as agents can take on more work. To allow the evals to evolve with the work, we structure the UX such that the product is the grader. A user’s corrections to output become graded examples, unedited = ground truth.
M&A, entity tax, and other core workflows coming soon!
http://evals.thriveholdings.com