Ira Rothken @rothken

We need to be able to have consistent LALA language and logging of AI agent workflows so we can teach it to lawyers and law students, including those who are not computer scientists, so they can implement in a consistent manner their duty to supervise the AI. Every AI legal platform should output this on demand.

We need LALA, a Lawyer AI Language for Agents. An AI can draft a 100-page contract or brief in two minutes. Supervising it well enough to satisfy the duty of care takes hours. So why do AI vendors sell the legal profession the two minutes? For any serious legal AI task, the human lawyer in the loop matters far more than whether one AI platform runs a few minutes faster than another. An AI might draft a 100-page agreement or brief from dozens of source documents in a minute or two. But a real human-in-the-loop review — the kind that actually satisfies a lawyer’s duty of supervision — takes a large multiple of that time. The AI generation was never the bottleneck. The supervision is. That’s where the time goes, and where the liability lives. And right now, that supervision is largely broken — because there’s no standard for how to do it. Each lawyer essentially invents their own paradigm for reviewing AI work, with no shared materials, no common method, and no agreed record of what “enough” review even looks like. We often hand a lawyer the AI’s polished output and call it “human in the loop.” But reading the output doesn’t always reveal what the AI quietly skipped — the controlling statute it never surfaced, the evidence or argument it dropped, the case it cited but never actually read correctly. To truly supervise it, the lawyer may have to reconstruct what the machine considered and ignored, across a workflow that can run differently every time. In some instances, that asks lawyers to be two people at once: the reviewing attorney AND a forensic AI scientist debugging a black box that changes on every run. Most lawyers aren’t computer scientists. So what we sometimes get is “human in the loop” as theater — a signature in a box, with no robust basis to validate what occurred. The stream of lawyers sanctioned for AI-hallucinated citations is just the visible tip of it — the failures we happen to catch because a judge did. To be fair, major platforms are improving but methods are balkanized. So let’s redefine “AI-native.” The real win isn’t the AI’s speed alone — it’s the speed and quality of the AI combined with genuinely good ways for lawyers to supervise and guide it, so that total productivity AND total quality come out net better. An AI that’s fast but unsupervisable isn’t AI-native. It’s just fast. The supervision layer is part of the product, not an afterthought bolted on at the end. Here’s a fix — and I think the whole profession benefits from it. Call it LALA, a Lawyer AI Language for Agents. It does three things. First, it generates consistent, structured log files — the same record every time — so the duty of supervision and any later audit finally rest on something real instead of a signature in a box. Second, it lets lawyers share AI agent workflows: write the steps, the checkpoints, and the human sign-offs once, as a plain-language file, and hand it to anyone. Third, it runs on a free, open interpreter, so no one has to buy a platform just to use it. The workflow is portable, but the tools it calls aren’t free by magic. To actually run one, you still need your own access to the pieces it reaches for — the LLM, a legal database like Westlaw, any vendor service. LALA is the portable recipe and the free interpreter is the kitchen; you still bring your own ingredients. What’s shared and standardized is the method — and the method is the hard part. The access you can already buy. The method is portable and readable, and the benefits run across the whole profession. Lawyers and Law students could learn AI supervision once and use it for years. Legal aid groups and small firms could inherit a workflow built by one of the largest firms and run it on the free interpreter — swapping in a free or low-cost model and a free case-law source where the original used premium ones. The method transfers even when the budget doesn’t, putting real access-to-justice firepower in the hands of those who need it most.

@ValsAI Further supporting why the lawyer duty of supervision of AI is the gating issue - we need to make more accurate, predictable, and efficient the human lawyer in the loop. Fast AI is illusory if lawyer review is optics over substance.

Ira Rothken @rothken

3 days ago

We need LALA, a Lawyer AI Language for Agents. An AI can draft a 100-page contract or brief in two minutes. Supervising it well enough to satisfy the duty of care takes hours. So why do AI vendors sell the legal profession the two minutes? For any serious legal AI task, the

5 3 35 4K 55

@MaxJunestrand @ycombinator @CooleyLLP @WeAreLegora Very good and helpful use case.

That is because AI is largely probabilistic, so there is some percentage risk of errors like hallucinations & lawyers have a duty to supervise AI - over delegation to AI can lead to a parade of horribles from sanctions to malpractice to unauthorized practice of law complaints. One should look to a total productivity of AI metric that includes the required gestalt amount of human in the loop. That is the gating issue on AI productivity enhancements.

@RafieFaruq Multi factorial - the more complex the task, the more probabilistic the analysis, the more at stake, the higher the error rate, the higher the malpractice and ethics risk, the higher the risk of violating court rules, are some of the factors.

Legal AI needs a Lawyer Analysis Metric (LAM): how much human review counts as sufficient supervision of AI for a given task. Review every citation? Wise, but it erases AI’s time savings. Review nothing? Sanctions. How about the LAM for a given contract review redlined by AI? We’re regulating this line without ever measuring it. It will vary by task. As AI improves, LAM should fall — if liability and ethics rules let it.

@gabepereyra This is very good work. There is an obvious tension in clear view. AI legal automation speeds things up and there is a lawyer duty of supervision line for human in the loop that is hard to measure that is key to the calculus on AI total productivity gains.

Ira Rothken @rothken

2 weeks ago

Legal AI needs a Lawyer Analysis Metric (LAM): how much human review counts as sufficient supervision of AI for a given task. Review every citation? Wise, but it erases AI’s time savings. Review nothing? Sanctions. How about the LAM for a given contract review redlined by AI?

12 3 40 4K 32

LAM should decrease over time

@curl_justin It matters but other legal data privacy risks remain absent point to point encryption.

Ira Rothken @rothken

2 weeks ago

Mythos is getting pushback from the legal AI folks for not having ZDR, and that critique matters. It’s also ironic that most legal SaaS tools that call LLM APIs, after pages of privacy‑protective language, still reserve the right to retain, analyze, and disclose lawyer–client

3 0 23 3K 18

It is a challenge as upstream and downstream are both complex. Microsoft Azure Cloud, Google Cloud, and Amazon AWS cloud are likely in the loop and they have the gestalt terms - as they should to stop bad folks - and nearly everything runs through them. So Harvey, Clio, and Legora are still likely subordinated to what the big cloud providers and ISPs choose to do when snoopers want to snoop unless “point to point” or user controlled encryption is used.

Mythos is getting pushback from the legal AI folks for not having ZDR, and that critique matters. It’s also ironic that most legal SaaS tools that call LLM APIs, after pages of privacy‑protective language, still reserve the right to retain, analyze, and disclose lawyer–client data when they believe it’s necessary to comply with law or protect their own rights and safety. That’s not a heckle—those clauses are prudent—but it shows just how much trust lawyers already place in their SaaS providers who are in the AI agent workflows.

@gabepereyra Sounds like that functionality is in the Venn diagram with Ediscovery and investigations. Why should law firms create a massive hybrid search - vector store for Ediscovery with search and AI prompting and another for discovery requests and brief writing in Harvey.

@gabepereyra @harvey I agree with you on early testing. Sometimes you just want the gist.

@WeAreLegora Supervision requires AI agent conduct and human lawyer clickstream logging which is dual use evidence - it shows good supervision or over delegation by a lawyer. This is especially important to discern what the AI legal agent does while you are not looking.

Ira Rothken @rothken

2 weeks ago

Welcome to the clickstream monitoring era of lawyers using AI. If lawyers use AI agents, every workflow step requires logging —- the need for quality control, risk reduction, and supervisory obligations require your firm to know what the agent was doing behind the scenes while

14 3 70 46K 73

This logging point covers one person firms too. If a lawyer uses AI agent workflows for client work it is axiomatic that they log in a secure manner the conduct of the agent, inputs and outputs, and clickstream. Saying I don’t know seems inconsistent with the duty to supervise AI.

Welcome to the clickstream monitoring era of lawyers using AI. If lawyers use AI agents, every workflow step requires logging —- the need for quality control, risk reduction, and supervisory obligations require your firm to know what the agent was doing behind the scenes while you weren’t looking, and to make sure you’re not over-delegating to the AI. Ironically, AI work is loggable by default - the clickstream already exists whether you want it or not. So it’s no longer “should we log this,” it’s “do we dare delete it” - and deleting it looks far worse than never having it. Insurers will require it before the bar does. Regulators will likely require judges to log AI interactions and workflows and to store them. But the same clickstream that is evidence you supervised also suggests when you didn’t - one minute to approve a 60 page brief or contract is not a good look for a lawyer under oath - and these logs may be discoverable and not privileged in certain contexts. Don’t log and you can’t defend your work. Log and you’ve built a forensic record of your own lapses. Either way, the lawyer’s judgment is no longer invisible the way it used to be. You’re not just adopting AI - you’re putting a clickstream analyzer on yourself and your law firm. This AI logging era will be particularly challenging to the design evolution of platforms like Harvey and Legora.

@CorpCounsel Yep and lots of logging and monitoring of the AI agent workflows and human lawyer in the loop to mitigate over delegation, retain proof of supervision, and avoid hallucinations.

Ira Rothken @rothken

2 weeks ago

Welcome to the clickstream monitoring era of lawyers using AI. If lawyers use AI agents, every workflow step requires logging —- the need for quality control, risk reduction, and supervisory obligations require your firm to know what the agent was doing behind the scenes while

14 3 70 46K 73

@gregisenberg AI Native means not over delegating law practice to the AI, properly keeping human lawyers in the loop (and avoiding hallucinations), and tracking agent workflows for compliance. It’s now the lawyer clickstream monitoring era.

Ira Rothken @rothken

2 weeks ago

Welcome to the clickstream monitoring era of lawyers using AI. If lawyers use AI agents, every workflow step requires logging —- the need for quality control, risk reduction, and supervisory obligations require your firm to know what the agent was doing behind the scenes while

14 3 70 46K 73

@circle Something like this swarmcontract.com