Legal tech firms use outdated models with low reasoning
12 posts across 5 communities in the last 30 days; 33% from r/LocalLLaMA.
“That's not Anthropic's fault though. They don't have the ability to release any of that because of copyright. Universities can take advantage of various exemptions (sections 107, 108, and 110), and even then they can't release most of what they scan.”
“> The old social contract was that the writer had done more work than they were asking of the reader, but this is torn to shreds with LLM-generated text.The problem is that your biases kick in long before you read the stuff. And I do not see a solution for this.I write my devlog…”
“the ones who agree the problem is real are never the ones holding the budget. the ones holding the budget just ask for a price and a name to sue if it breaks. law firms dont buy inference, they buy whatever the IT guy already resells them.”
Several users express concern that legal technology companies are employing older AI models, specifically mentioning 'frontier models' and noting their limitations in reasoning ability. This results in inaccurate comparisons and a perceived disadvantage when competing against more advanced systems like those from Anthropic. The posts suggest Thomson is using these outdated models.
SOURCES (27)
“That's not even a good argument, that's literally just RNG and a pool of choices. I meant make decisions with judgement, which is not debatable in LLMs. Look at the recent Jev decision model, that's literally all it does. [re: CISA and NIST advisory about…”
“I don't quite get what's the argument you're making here. What's in contention is whether or not the 0.2% is in agreement.Are WE the 0.2%?Because if not, what's the point talking about the merits of AI's competence in of itself? Accomplishes nothing, we're he outsiders. And if we are, then either you're fishing for disagreement and the post you replied to is actually correct, or you get unanimous support to your statements, and establish... Nothing, the in-group could be delusional in a way that”
“What a waste. Thomson Reuters should realize that their "copyrighted editorial material" is easily replicated. It's like suggesting, "because you can extract an opinion and summarize it, you are therefore copying our intellectual property." Agents will be dismembering Thomson Reuters IP six ways to Sunday before the end of the year.”
“That's a very lazy attempt at a strawman argument. You can do much better than that lol”
“They’re called f-a-c-t-s. Add education to the list of things you guys need. Cheers!”
“I agree. You still need to understand "scaffolding' underneath to verify and control the outcome but it takes care of tedious parts.”
“That's not Anthropic's fault though. They don't have the ability to release any of that because of copyright. Universities can take advantage of various exemptions (sections 107, 108, and 110), and even then they can't release most of what they scan.”
“This isn’t even how it works. The key is in recognizing good arguments - which at the trial court typically have to do with the facts, which are being developed in real time to an extent - the law is really pretty secondary.”
“> The old social contract was that the writer had done more work than they were asking of the reader, but this is torn to shreds with LLM-generated text.The problem is that your biases kick in long before you read the stuff. And I do not see a solution for this.I write my devlog by hand and then run it through an LLM for a quick sanity check. Grammatical errors and spelling mistakes in stuff you publish in this age of AI is lazy. But I don't do it when I comment on HN/Reddit etc because this is”
“It literally is. It's just that the trainee is a machine instead of a human being.”
“the ones who agree the problem is real are never the ones holding the budget. the ones holding the budget just ask for a price and a name to sue if it breaks. law firms dont buy inference, they buy whatever the IT guy already resells them.”
“Most motions are either a discretionary judgment call by the court or there is likely an objectively correct answer to a dispute based on a relatively small body of relevant case law. Sometimes there is a case of first impression where looking at a wider body of persuasive authority makes sense and issue framing is more important. However, there is a finite body of case law and a finite number of structures for an argument and differences amount to style which doesn’t matter much. AI, as you des”
“what part of the premise do you think breaks down? I’m not suggesting courts currently allow unlimited filings. I’m asking what happens when the amount of legal analysis that can be generated before a filing increases by orders of magnitude while the court’s attention remains finite.”
“Your argument continues to be poor. Your predicates simply do not support your conclusion.”
“this is actually not how frontier labs are training their models any longer. they are long past the stage of stealing everything and uploading it into their model. I'm sure all the frontier models have already stolen everything that is in westlaw. nowadays, frontier labs are having real people grade AI outputs and are using that to improve quality of output. there are companies that hire law students at 60/h to grade AI outputs. the thing that westlaw has over frontier labs is actual lawyer&”
“I think they are VC backed but yes that’s interesting. I can’t imagine a good lawyer agreeing to work there”
“My understanding is they are farming out the legal work to agents supervised by lawyers in India.”
“I understand. I'm just saying competition lawyers are notoriously conservative about AI usage. When I say you'll have better luck with experts, I mean expert witnesses, not lawyers who are experts at competition law”
“(I'm sorry if this goes against the community rules. If it does, I'd appreciate subreddit recommendations!) I'm running a blind evaluation comparing several AI systems' performance on competition law Q&A, both case-specific and theoretical questions. I'm looking for practitioners or academics with genuine competition law expertise to score anonymized transcripts against a structured rubric (legal accuracy, citation quality, reasoning, etc.). What's involved: \- \~3 ho”
“The thing that I find hilarious is that if it indeed cost $20M, maybe you could have paid 10 mathematicians $200K per year for 10 years to do nothing but work to solve that problem and this might have happened faster...”
Which they could arguably recreate. In fact, are harnesses patentable?
“I think the outcome will be very obvious when the the time comes to settle the bet, but I agree that it's not easy to write those terms today.But speaking of Stack Overflow, that's a good point to examine more closely. Simply looking at Stack Overflow's usage trend [1] should have been a strong clue that a seismic shift was under way. A couple of things could be responsible for it, though:1) Stack Overflow's movers and shakers have finally assholed themselves into irrelevance. That's possible”
“I would enthusiastically recommend the OpenLaw MCP. It's free, essentially BAILII (or your local equivalent) but made LLM-friendly. You can connect it to Claude Code or your favourite harness. It doesn't have the depth of Westlaw, but it's 90% of the way for 0% of the price.”
“Just saying cuz they used outdated frontier models and used them with low reasoning, supplied snowdon and The company's strongest factuality comparison gave Thomson access to Westlaw and Practical Law while rival models searched the open web. Of course it won that one. You don't get credit for beating someone in a race where you gave yourself a car”
“Okay, then give us the actual market thesis. Your original claim was that TR is not a model-training company. When presented with evidence of its model-training capability, you shifted to saying it cannot compete with frontier labs. Compete on what axis? TR does not need to outperform Anthropic on every general-purpose benchmark. It needs a model that performs well enough on relevant tasks that, combined with its corpus, tool environments, distribution, and control over inference costs, produces”