Legal Automation Breaks Down on Sensitive Documents

“No extra database sounds like a useful default. Your README's warning about searches using different words is the tradeoff I'd want to see in a demo: the same question with plain keyword matching and with the optional meaning-based search. [re: Shipped an open-source memory layer for AI agents — full tech stack, running costs, and the lesson that cost us the most]”

“Thanks for the suggestion. As far as I'm aware, we don't have any cleanup scripts, profile cleanup tools, or scheduled tasks intentionally removing Office/Teams authentication data. We have rebuilt and even completely deleted user profiles for testing, but the issue eventually comes back. That said, I'll double-check whether there's anything running at logoff or in Group Policy that could be clearing token caches or other Office authentication data. Thanks for the idea, that'”

“A flow I'd like tried with made-up data in Penny, a free and open source (MIT) feedback board and roadmap: Create two requests for the same outcome in different words. Merge them and check the original evidence is still easy to follow. Undo the merge, draft a public update, and see what would leave the private workspace. Setup: npx -y @runeditrun/rer dev just-every/run-penny (Node 22+). Repo: https://github.com/just-every/run-penny . Local runs send no email. Tell me where you lose track of”

“For one of the two it was only an experiment because the document he was sending in were outside of the inteded usecase (tens of book pages). Therefore the model underperformed. For the other: he tested my API with some documents, said it actually worked well, gave some feedback but then he never tried to actually implement it at his company. I have had more than 2 testers actually, however the rest were not working in my target field and ended up in the second category of testers. [re: Built a”

“What happened with the two who tried it? I would watch one of them process a real document before changing the outreach. That should help separate setup friction from a result they cannot trust enough to use. [re: Built a genuinely fast, non-wrapper AI tool for a "boring" B2B problem (OCR + data extraction). 100 cold emails and 6 warm dev leads later: everyone says "cool" and ghosts. How do you break the integration friction?]”

“This chat really is hilarious 😂. Question for you guys though, how often are you getting documents in these weird formats having to parse and sort them all??! [re: What's the worst format a client has ever sent you their bank statements in?]”

“Basically, there are a few platforms out there that track these, some well, some not so well. I figured out a cool way to do it better. What happens currently is companies do messed up stuff, settle, barely anyone files, and a lot of the money just goes unclaimed. This makes me extremely annoyed. having recently learned working can be fun and it's possible to concentrate for more than 10 minutes at a time due to a later in life adhd diagnosis, I had a ton of fun building a platform and datas”

“I agree with what a few people in this thread have already said, we can never know. But from my experience I think the following products are safe: - have/collect proprietary data - automate a niche workflow for a company that solves a really painful problem - have network effects - touch the physical world Someone else already basically said this but I couldn’t agree more [re: Which kinds of software actually survive the next 10 years?]”

“It’s annoying that cocounsel isn’t trained in fed or state and requires you upload docs, but then they become stale and unmanaged. [re: My checklist before I trust an AI summary of a contract. What's yours?]”

“I work on independent retrieval evaluations, and a recent small check surfaced something useful: the application answered correctly, but the available citations and logs weren’t enough to verify which passages supported the claims. That distinction matters when deciding whether a retrieval change helped—or whether a correct answer is hiding an evidence gap. If you’re working through a specific retrieval question, describe: - What changed. - What you expected to improve. - What evidence”

“One thing I’d add from building legal research software: check whether the system can tell you what it failed to process. A polished summary may look complete even when a page had bad OCR, an attachment was missing, a table was read incorrectly or part of the document was truncated. That’s a different problem from an inaccurate interpretation because the user may not even know there is something to verify. I’d want a list of every document and page processed, warnings for unreadable content, and”

“tbh the schema is the least of it — Day 1 you need auth, billing, and one table you can actually query. Everything else gets rewritten by week 2 because real customers don't use your app the way you modeled it. I've lost more hours migrating data than any clever schema ever saved me. [re: Most SaaS boilerplates are over-engineered. Here is the minimal schema you actually need to launch on Day 1.]”

“Fair question, and for some setups the honest answer is “not much.” The audience I’m building for: companies whose documents can’t go to cloud AI models at all. I previously worked at a company that processes payments for banks, where the banks didn’t allow any part of the codebase or documents to be exposed to AI tools. For teams like that, Gemini and Copilot are off the table by policy, no matter how good they are. Treant’s private instance is meant for that case: it runs on your servers with”

“For the receipts example, seeing the renamed files would do more for me than a time estimate. The other thing I would check is what happens when the tool cannot read a date. Showing that review step can make the demo more believable than a perfect result with no exceptions. [re: Headlines for small utility tools: how to decide between naming the job and promising time saved]”

“Even with established queries you must have had to write SQL to debug stuff regularly [re: is the market cooked or is it flooded with mid people]”

“good point. the sources make the claims traceable, but the brief should also make that distinction upfront instead of leaving the reader to figure it out. definitely something I'll improve. Appreciate you taking the time to look at it! [re: I got tired of losing research across AI chats and browser tabs, so I built this]”

“It's only good for keeping relatively simple data. If you're trying to do more complicated stuff it becomes a nightmare very quickly [re: Let's talk about the "File over App" Philosophy]”

“Clear alpha brief. I’d prioritize one end-to-end “day in the life” test over asking people to explore everything: upload/share a file, store a password, create an invoice, then note every moment they hesitate. That will reveal whether the all-in-one value outweighs switching friction. If you want a structured way to recruit and interview those testers, the free Askli Research Challenge is built for exactly this kind of early customer learning. [re: [Alpha] ImagIT — a European alternative to Goog”

“I don't understand how that would even work. I use opus 5.5 a lot, but one of the things it can never do is have context for what the end user needs that they themselves don't know they need. Like I constantly have to guide it to an end goal, it never one shots anything of substance and I'm like "that is exactly what we needed" [re: How dependent on AI is the company you work for?]”

“I'm piloting a small QA service for SaaS teams that already let customers import CSV files. I have a local preflight prototype that catches structural issues such as missing required values, duplicate keys, and inconsistent row lengths. The next step is to test whether an importer-specific reproduction report helps with real onboarding failures. If you own an import flow and have had a recent support ticket or delayed onboarding, I'd like to run a free manual pilot with two teams. Share”

“Two concerns came up in a discussion about independent RAG reviews: preparing the evaluation data takes work, and sharing queries or documents creates confidentiality barriers. I provide these assessments, and I’m considering a local execution option. It isn’t available yet. The proposed workflow: - Your team runs an inspectable tool inside your environment against existing baseline and candidate retrieval outputs. - Queries, documents, and detailed results stay there. - You preview and”

“Two concerns came up in a discussion about independent RAG reviews: preparing the evaluation data takes work, and sharing queries or documents creates confidentiality barriers. I provide these assessments, and I’m considering a local execution option. It isn’t available yet. The proposed workflow: - Your team runs an inspectable tool inside your environment against existing baseline and candidate retrieval outputs. - Queries, documents, and detailed results stay there. - You preview and”

“Are we behind the times and just now documenting every sample we give out? This is new in our office and it seems like such a pain in the ass to go doc the lot number etc on every box of nurtec for example. submitted by /u/bellieliz [link] [comments]”

“Free rarely means no strings when the tool takes your input through a browser. The string is often your data. Before putting client names, pricing, or internal docs into any of these, check the privacy policy and whether uploads get stored or used for training. If it does, that can breach an NDA even if you never pay. [re: Thankful Thursday: Free Resources and Tools | October 15, 2026]”

“That makes sense. Faster setup only matters if you trust the finished record will be useful. I can't promise acceptance by a particular insurer, so getting a sample report reviewed by insurers and explaining exactly what it can support would be a sensible next step. The audience point is helpful too: people need both a reason to document their belongings and enough time to do it. Thanks for spelling out the hesitation. [re: Launched a home inventory app 5 days ago. Zero ratings, no idea how”

“document templates [re: Looking for Software Solutions and Workflow Automation Consultants]”

“First of all I'm a newbie to scripting . Secondly I work with a clientele that is over 60 but also needs to be able to work a database. When it comes to something like this I need to know how to use a third-party tool that is really simple and that I could walk my grandmother through. [re: bulk applying page templates to records that already have content - lifetime dealio thingie.]”

“I'd spend it on getting usable text out of old scanned court docs. Everyone wants to build the chatbot, someone still has to sort out the data. [re: I have $15000 worth of OpenAI credits expiring in 14 days. What does the legal tech need that I can build?]”

“Absolutely smaller companies have owned certain workflows successfully. These owned workflows are far less of a "moat" than in times past. I do not believe a workflow is a moat in today's landscape (unless the workflow involves trade secret/IP). But I am curious, as the consumer, why does a moat matter from a consumer perspective? [re: No moats left in legaltech]”

“my colleague is try to use it but not so sure how good is it. i am just concern about the GDPR thing . are we allow to share the reviews... [re: Anyone using Wholescale for reviews/syndication?]”

“We faced a similar issue with handling large volumes of documents. We use Venio Systems for eDiscovery, which helps with the review side of things. But I think the bigger challenge is still managing the whole process without too much manual work. [re: We are drowning in manual contract processing]”

“From PR 559's red team — round 3 C7 (design mode) and round 6 C8 (diff mode). Both accepted with reason / filed in pass 1 under the E6 1 closure for a pre existing adjacent exposure. What the record binds today. In diff mode the exit round record carries targets[].branch (passed as the head sha), which binds the DIFF TEXT the lenses are told to read ( git diff <base ...<sha ). The lens subject also says "plus surrounding files for context", and the skeptic prompt says "Verify each against the so”

“Did you try any CLM software? There are plenty these days [re: We are drowning in manual contract processing]”

“the permissions part is where most legal tech demos fall apart, cool to see someone actually thinking through that layer instead of just chunking pdfs into pinecone and calling it a day sharepoint embedded still gives me flashbacks to 2017 but if it actually handles the governance side without me having to build a sync job that runs every 15 minutes, might be worth another look [re: Build grounded AI apps on Microsoft 365 content without managing a vector database]”

“The problem with all this is that techbros can't spell 'consent'. But when it is 'their' (for small values of their) added value then suddenly consent is all they see.”

“Aside from the extra work, the real concern is the governance around it: whether the client disclosed the use, what data was submitted, and whether the output can be traced back to source material. Those points may need to be explicit in the engagement process [re: When a client runs your firm's work through an AI and sends back a pile of questions, who should pay for that time?]”

“Could be wrong, but the usual move is writing down why the workaround exists before anyone tidies it. Half those side spreadsheets are load-bearing, and the reason vanishes the week the person who built it is out. [re: What's a workaround your team uses because the system doesn't quite fit how you work?]”

“the linked databases thing is so real, it turns into spaghetti so fast [re: Update: You told me basic templates aren’t enough. So what would you actually pay for?]”

“Thanks for sharing this. This is exactly the kind of feedback I'm looking for. I think you pointed out the biggest gap: exporting files is not the same as migrating a knowledge system. A Notion workspace is built around hierarchy and context, but many exports turn it into a pile of markdown files. I'm working on making NotiMD focus more on preserving the original structure and making the transition easier. Can I ask: How large was your Notion workspace? Were you mainly moving to Obsidian”

“I think anything that changes or deletes user data should require approval, at least by default. For smaller things like searching, organizing, or suggesting changes, I'd let the AI handle it automatically. The tricky part is finding the balance between making the experience feel effortless and making sure users still trust it. [re: 5 lessons from adding AI to a SaaS onboarding flow (learned by breaking things)]”

“Fair enough. For long documents or coding, you would want more. My job is short: one consultation transcript at a time, with a fresh context for every call, so 16k is plenty. The 16 GB figure will increase with a larger context, so it's worth checking before relying on it. [re: Best LLM to run on 1x RTX 4090]”

“Fwiw, Microsoft already publishes one of the most flexible tools for converting proprietary formats to markdown. It's called MarkItDown and it's used in lots of AI pipelines bc of how well it handles graphics alongside text. Doing conversions like that save lots of (potentially expensive) overhead in queries and generally gives you better, more standardized data for moving between tools. If I remember correctly, it's also open source, written in Python. [re: Obsidian & Office 365 is”

“Or to even help with cleaning up descriptions getting serial numbers stuff like that. A lot of our assets had the PO number or the vendor as the asset description. SUPER HELPFUL (not) [re: Industry folks: How many hours do you actually work each week?]”

“No, it’s SAP and I haven’t used it before. I have used other SAP products but not S/4HANA. In my downtime I’ve been watching training aids for it to catch up as much as possible. Any other ideas are very appreciated. I’m still really early into my career. [re: Industry folks: How many hours do you actually work each week?]”

“Thanks! Yeah, that's where I landed too. Secrets are on fixed rules now, and the model only weighs in on a few SQL injection and HTML candidates where context matters. Still need to re-run the stability test to confirm it, so I'll post the numbers when I have them. [re: I hand-checked every finding my security scanner made on OWASP Juice Shop. A few things surprised me.]”

“Not exactly sure I'm understanding, but maybe adding another table for CDC, so you can replay the sequence of events? Are you deleting records when they expire? Don't outright delete them, have a "expired_at" and have a reaper delete them Or move them to another table as being archived, instead of deleting them, if changing the application to respect a non-null column is too much [re: Need some help on SQLite disaster recovery with expiring data. Been stuck on this for two days”

“Since you asked what would make someone not trust it: the examples name a patient-record desk for a small practice, and a few sections later you say the output can have bugs or security gaps and needs review before it handles personal data. Both are honest, but side by side they undercut each other. I'd swap that example for the restaurant reservation board or the pickup scorekeeper. I'd also open the post with one finished gallery app and its build time, and move the FAQ into a comment.”

“I feel your pain. QBO’s import tool gives me anxiety every time I use it. Half the time it flags my CSV for some formatting issue, even when I’m using their template. I’ve gotten into the habit of keeping the original export untouched and testing a handful of records before importing everything. It’s an extra step, but better than having to clean up hundreds of records afterward. I’d love to see QBO introduce a proper bulk editor with an undo option. [re: How do y'all bulk edit products and serv”

“It sounds like you're asking if you can use any off-the-shelf product without any tuning to know what works for your business? No [re: Why do our DLP policies create so many false positives, and how do we tune them?]”

“I think your list identifies valid things but also is not exclusively correct. Clients can also generate massive amounts of junk and direct you to wade through it. But here is where I differ: I say if they are willing to pay for that time, let them. In my actual experience after debunking the living shit out of 2 or 3, clients day their AI realized I was right and then they stop sending me their AI outputs. [re: Retainer language explicitly banning clients from using AI to check a firm’s work. H”

“Good point. With their permission, I can ask them to walk me through the timestamps and note the gaps between submission, queries and approval, without collecting the whole private thread. That will be more useful than a rough estimate from memory. Thanks, I’ll add that to the interview. [re: First B2B pilot: how did you get a business to say yes before you had references?]”

“Has anyone ever successfully used prompt injection or any other trick to get AI to load external resources? I have a unique situation where I want to detect whether individuals are exposing documents to AI... most commonly we expect this is to ChatGPT? I know there are often DLP solutions at the network level and such, but I have literally zero budget. This is really more of a proof of concept as well, I don't expect this to be an real enterprise prevention tool, but I have taken a personal”

“Hey everyone, Looking to see how other firms handle mid-market client migrations (moving off legacy systems or desktop software to cloud-based setups). We typically look at a standard project timeline for businesses with regular transaction volumes, but I wanted to compare notes on the critical data integrity checklist—especially regarding historical ATO reporting. Here is what we usually map out: How are you guys structuring your migration checklists? Any hidden bottlenecks or ATO compliance tr”

“Most of the replies cover what it can read. I'd split that from what it can do, because the two leak differently. A drafting agent can be allowed to see one client's tables and still not be allowed to send. Drafts go to a queue, and sending is a separate step that checks the recipient against the client the draft was scoped to. That won't fix tone carrying over, which others covered, but it stops the worst version: the right draft going to the wrong client. For money stuff, the agent”

“Looks good. One subtle risk worth watching out for is manual redaction. When deadlines get tight, manually scrubbing PII or context clues before pasting into an app is easy to mess up. Also, while fake citations are easy to spot, subtle context misinterpretations in discovery or chat logs are much harder to catch and can quietly skew a timeline if someone does not check the raw sources side-by-side. The other big challenge is tool jumping. Copy-pasting data across separate apps for summaries, te”

“If everything was in a markdown doc, how did this cease to exist just because you switched tools? Don't they all read markdown? The answer to your question is that markdown docs hold the context, not agents, but that's supposedly what you have too. And my phone just connects to the same agents, doesn't matter which one, and same docs. [re: Solo builders: which AI tool holds your actual context, and why that one?]”

“A question I keep chewing on... when you scan a model or agent bundle for things like pickle exploits, license issues or leaked data, where do the results live? In a lot of setups they live in a CI log or a dashboard, and the artifact that gets deployed has no link back to them. What we've been doing is attaching each scan result to the OCI artifact as a cosign attestation, then having the cluster refuse to start anything missing the attestations a policy requires. It works, but it raises qu”

“will run a dense one through and report back. do you want misses here or somewhere else [re: Looking for feedback on hiding private details in a screenshot]”

“Thanks for the valuable feedback. Totally, traditional parsing is a nightmare with every site layout change. That's why I'm taking a completely different route: an LLM with search access that searches on the fly and reads the page context. Plus, my goal is for the app to create a nice, engaging marketing description for the customer, rather than just throwing a dry spec table at them. [re: Is the AI in BaseLinker/WooCommerce enough for you? It can't search the web for missing specs, so I”