Lessons Learned, Lessons Lost: Why AI Retrospectives Rule

Scott S NelsonLeave a comment

TL;DR: The Sprint Retrospective would be the most valuable ceremony in Scrum…if it weren’t also the one with the worst follow-through. AI can run the same ceremony and actually keep the results, because the lesson gets loaded before the next session instead of filed and forgotten. It’s a simple habit to learn, and there are plenty of free examples of how to automate it. It’s a small investment of time and thought that pays real dividends in productivity and confidence.

The Ceremony That Everyone Likes and Nobody Finishes

The 2020 Scrum Guide puts the Sprint Retrospective at the end of the Sprint, timeboxed to three hours for a one-month Sprint (not that many places do one-month Sprints anymore, or allot three hours for improving things). The process is simple: provide a safe space where people can speak their mind about what went well, what didn’t, and how they would change things to be better in the next Sprint. The goal is to capture ways to increase quality and effectiveness while reducing friction and wasted time.

In theory it’s the smartest ceremony of any SDLC. A room full of people who just lived through the same thing, comparing notes while it’s fresh, with permission to say what actually happened.

I was introduced to this ceremony before Agile approaches were common in the enterprise. We ran it after a major release, and I found the process very enlightening. Someone wrote up a summary of the findings and posted it on the company intranet.

This was a long time ago. Years later I remembered that document and got curious about whether anyone had ever gone back to it.

I was the only one who ever looked at it again.

I can’t recall a single piece of wisdom it captured. Not one. What I do remember is the aura of “this makes so much sense” when we discussed the stickies on the wall. And that the same mistakes were repeated across later projects, many with some of the same people in the room who had sat through the meeting where we agreed not to make them again. Sigh.

What Happens to the Findings

Follow the output and the pattern is easy to see.

Some items become real assignments. Someone owns it, someone completes it, and the team is measurably better off. That’s a real win and it’s worth saying so.

The problem is, it is too often a one-off win. The task closes. It’s checked off and moved to Done. Then the same thing comes up three Sprints later. Or the next Sprint.

Because the good stuff that gets surfaced in a retro isn’t something that can be fixed with a story. It’s a lesson. “We committed before we understood the integration.” “We let the staging environment drift.” “We assumed the client would review in three days and they took three weeks.” Those aren’t tickets. They’re rules for how to behave next time, and there is nowhere to put them where people will remember them (yes, there’s an exception, for those who are already ahead of the game with a living PROJECT.md, aka CLAUDE.md, AGENTS.md, or whatever your favorite harness uses).

So they go into a document. Which goes onto an intranet page. Which nobody opens during the next project, because during the next project everyone is busy doing the next project.

The retrospective produces knowledge with no delivery mechanism. Six weeks later a new team, or the same team with two new people, walks into the identical wall and holds an identical meeting about it. We call that continuous improvement. It’s closer to continuous rediscovery.

That failure, organizations not learning from their own experience even after making the capture of lessons a required part of the process, is a big part of why the Best Practices section on this site exists at all.

Run the Same Ceremony, by Hand

Now try it with AI at the end of a working session. At its most basic, this is two prompts.

Review this session. What worked, what didn't, what should change next time?
Give it to me as Start, Stop, Continue.

Start doing this, stop doing that, keep doing this other thing. Read what comes back, edit anything that’s wrong or too vague to act on, then:

Save that to memory and apply it going forward.

That’s the entire ceremony. Under two minutes, no tooling, no setup. (YMMV on the prompt, depending on your type and style of work…BWTM)

Here’s the part that changes everything. It doesn’t file the lesson. It applies it, every time, forever. Next session, and every session after that, those rules are loaded before the first word of work gets done. There is no folder, no onboarding deck, no hoping somebody remembers. The correction is simply in effect.

And it updates. Session ten refines what session three concluded. A rule that turned out to be wrong gets replaced instead of quietly ignored.

Same ceremony. Same format. The difference is entirely in what happens after the meeting ends.

The productivity gain is obvious enough. The part that sneaks up on you is the confidence. After a few weeks you stop bracing for the mistake you already corrected once, because it doesn’t come back.

The Gap Was Never Honesty

The lesson here isn’t that AI is disciplined and people are sloppy. People are perfectly capable of discipline. What people don’t have is a place to put a lesson where it will find them again at the moment they need it.

That’s the gap the retrospective has always had. Not insight. Distribution.

AI closes it almost by accident, because persistent memory is just how the thing works. The lesson arrives in the same channel as the work, automatically. You don’t have to remember to go check.

Which reframes a lot of process failures. Most of the time the problem isn’t that we didn’t learn. It’s that the learning had no route back into the work.

Almost Nobody Does This

Here’s the honest part. Very few people take the time to run a retrospective on their AI sessions, and fewer still do it consistently.

It isn’t hard and it isn’t slow, which is exactly what makes it easy to skip. Two minutes is nothing. Two minutes you never spend is also nothing, and it’s what most people land on by about the third busy week. The ceremony that depends on remembering to hold it is the ceremony that quietly stops happening, which is the same failure mode as the intranet page, just faster.

That’s the argument for automating the capture rather than relying on the habit.

You Don’t Have to Invent This

Run it by hand for a few weeks first, though. You want a feel for what a useful captured lesson looks like before you hand the job to something that writes them without asking, because you’ll be reviewing that output later and you need to be able to spot a bad one. Automated capture drifts. It records the frustrated aside instead of the actual rule, or it generalizes a one-off into a standing instruction. Doing it manually first is how you learn to recognize that when you see it.

And you don’t have to tune that prompt from scratch, or build the automation yourself either. There’s an entire cottage industry of shared instruction sets, skills, and rule files built to automate this loop, and most of them are free and MIT-licensed. Read a few and you’ll find better phrasing than mine sitting right there in someone’s repo.

claude-reflect is the most widely adopted of the ones I found. Hooks watch your prompts for correction patterns like “no, use X” or “actually” and queue them automatically. You then run a command to review the queue and approve what gets written into your persistent instructions. Capture is automatic, application requires your sign-off. It also mines your session history for repeated requests and offers to turn them into reusable commands.

bokan’s self-improvement skill takes a different angle: it fans out parallel agents across your past sessions, ranks friction patterns by how often they recur, and attaches the raw quote where things went sideways. Frequency plus evidence, which makes the finding arguable rather than just asserted.

claude-improve is worth reading even if you never install it, because its README openly credits the six community approaches it was assembled from. One of those, a one-prompt reflection pattern, contributes the single most practical idea in the whole space: write the captured lesson as an enforceable rule. Lead with why, use NEVER and ALWAYS, include a concrete example. Most captured lessons fail because they’re written as vague observations that nothing can act on.

Cursor users have the same pattern in a different wrapper, usually a self_improve.mdc rule paired with a rules file, as described in Stop Babysitting your AI.

Pick one, use it as-is, or read three of them and build the version that fits how you actually work. Standing on someone else’s structure beats staring at a blank file.

What the Evidence Says

One thing to calibrate before you go shopping, because I’d rather say it than have you find out later.

None of these tools has been independently reviewed or benchmarked. What they have is stars, forks, and enthusiastic author write-ups. A thousand stars measures whether an idea sounds good, not whether it works. There’s no head-to-head comparison, so pick on fit rather than on popularity.

The underlying mechanic, though, has real support. The Reflexion paper from NeurIPS 2023 found that agents that write a short natural-language reflection on what went wrong outperform agents that store the raw record of what happened by eight percentage points absolute. Storing the transcript is worth less than storing the conclusion. That is the academic version of Start, Stop, Continue.

Anthropic’s own context management benchmark reports a 39% performance improvement over baseline when memory and context editing are combined, and 84% fewer tokens consumed on a long multi-step task. Vendor-run, so discount it accordingly, but it’s a real evaluation with published numbers.

The mechanism is sound. The specific tooling is a matter of taste.

Prune It, or You’ve Just Rebuilt the Intranet Page

The one part none of this tooling can do for you: every system here accumulates rules, and none of them has a convincing answer for pruning. claude-reflect ships a dedupe command, which tells you the problem is real enough to need a command.

So budget a few minutes every month or so to read what’s piled up and delete what’s gone stale. That’s the intervention the manual practice trained you for. A memory file nobody audits becomes the same dead document, except this one is actively steering your work instead of quietly sitting on an intranet.

Making the Time Before You Save the Time

All of this runs into the oldest problem in efficiency work: you have to spend the time before you get the time. Every initiative that eventually saves hours starts by costing hours you didn’t have when you decided to do it. Automating your retrospectives is no different. The build is small, but it’s still a build, and it competes with the actual work that made you want the efficiency in the first place.

I get it. New habits are hard. I confess that I don’t do this as regularly as I should, and that even though I have some prompts pre-written in UpNote to use that are more involved than the example given in this post, I usually write one off the cuff when I do remember. Maybe if I spent less time writing about techniques and more time automating them…

Which is exactly why the two-prompt version matters. It costs nothing to start, it works on its own, and it buys you the judgment you’ll need before you spend an afternoon wiring up anything fancier. Start there. Automate when the habit is annoying you enough to be worth removing.

An Answer to the Question Everyone Keeps Asking

“What are we doing with the time we save using AI?”

Fair question, and most of the answers are vapor. Here’s one that isn’t.

Spend a slice of it teaching the tool what it just learned. Those two minutes are the highest-leverage part of the whole workflow, because they’re the only part that compounds. Every other minute you save gets saved once. This one gets saved again on every future session.

It shouldn’t take all of it. And it should take less as time goes on, which is the tell that it’s working. Early sessions produce a lot of corrections because there’s a lot the tool doesn’t know about how you work. Later sessions produce a few refinements. Eventually you’re mostly confirming that the rules still hold.

Use the rest to go touch grass.

That’s not a throwaway. The entire argument for this technology is that it gives us capacity back. If we spend all of the reclaimed time on more work, we didn’t gain capacity, we just raised the quota.

Now Tell Me What You’re Actually Doing

This is the part where a post like this usually promises a follow-up with results. I’d rather ask than promise.

If you’re already running some version of this, I want to hear it. Which tool, or which few lines you wrote yourself. What the capture prompt actually says. Whether you review what gets written or let it accumulate. What broke.

If you’re not doing it yet and this pushed you into trying, say so at the start and tell me what happened three weeks in. Including if the answer is that you did it twice and stopped, because that result is more useful than another enthusiastic write-up. The failure modes are what’s missing from every article I read while putting this together.

If enough of it turns out to be interesting, there’s probably a follow-up in it, built mostly out of what readers send rather than what I think. Credited unless you’d rather not be. A comment, an email, a reply on LinkedIn, whatever’s easiest.

The retrospective finally works. Let’s find out whether we actually run it.

If you found this interesting, please share.
Claude helping me move

Keep Claude from Forgetting You When Moving to a New PC

Scott S NelsonLeave a comment

TL;DR: Copying your project folders to a new laptop moves the files and leaves the work behind. Everything Claude Desktop actually remembers, your Cowork sessions, your MCP connector config, your Claude Code settings, lives in a Windows package sandbox that a normal search will never find. Zip that folder with the app closed and your history comes with you. Skip it and you start over with a very clean, very empty sidebar.

New Laptop Day

New laptop day is supposed to be a good day. Copy the projects, sign in, get back to work.

For most non-developers it generally is pretty straightforward, especially if your company manages backups and you use defaults. Sign in, wait for the sync icon to stop spinning, open the same three apps you always open. Whatever you’d customized was mostly customized inside something that follows your account, so it follows your account.

For developers and other power users it has never been that straightforward. There are a million little tweaks accumulated over years for personal preference and productivity, and not one of them is written down anywhere. The PATH entry you added at 11pm to make a build work. The tool installed from a zip file into a folder you invented. The config that lives three levels below AppData because that’s where the installer decided to put it. You don’t have a list. You have a vague sense that something will be missing, and you find out which something at the worst possible moment, usually about four days later.

Thanks to the popularity of Claude, now everyone gets those extra steps. And they’re not that straightforward even for the power users.

Here’s how it goes wrong. You copy the project folders. OneDrive handles the synced ones, a USB drive handles the local-only ones, file counts check out, you spot-check a few documents and nothing’s corrupted. Install Claude Desktop, sign in, open Cowork.

Empty.

The files are all there. The work is not. Every session, every bit of accumulated context, every conversation where you finally got Claude to understand your codebase, none of it came along. Because none of that lives in the project folder. It lives somewhere else entirely, and the app gives you no indication where.

Worth sorting out before you wipe the old machine, which is the point of no return most people hit about an hour after they think they’re done.

Two Kinds of Sessions, Only One of Which Is Your Problem

Some good news first. Cowork sessions now run remotely and are tied to your Claude account rather than your hardware. Sign into a new machine and those sessions are already there, waiting, automagically. Nothing to move, nothing to find. For once, the defaults crowd wins outright.

The catch is that this only holds for sessions created after remote sessions shipped. Anything older is a local artifact sitting on the old hard drive. If you’ve been using Cowork for a while, that’s likely most of your history.

So the question isn’t whether to migrate. It’s whether your sessions predate the cutover.

If none of them do, close the tab and enjoy the new laptop. Although before you go, it’s worth a moment’s honest reflection: if you’ve been at this for a while and have nothing accumulated that’s worth the trouble of moving, that’s telling you something. Not about the migration. About how much of Claude you’re actually using. A tool you’ve spent months teaching your projects, your standards, and your preferences to is a different tool than one you open to write emails. If none of this applies to you, the interesting question isn’t how to migrate. It’s what you’ve been leaving on the table.

For everyone else, keep reading.

Where It Actually Lives

Ask Claude where its own session data lives on Windows and it will tell you %APPDATA%\Claude\local-agent-mode-sessions\. It will tell you this confidently. Community migration tooling references the same path, so it isn’t inventing it from nothing.

It isn’t there.

Nor is it at %LOCALAPPDATA%\Claude\, which is the natural second guess and the next thing you’ll be told to try. You can run a recursive search across your entire user profile for audit.jsonl, the conversation log written inside every Cowork session, and get nothing back at all. That result is disorienting enough that you start to wonder whether the sessions ever existed on disk in the first place.

Whether that documented path was ever correct on Windows, or whether it’s a Mac convention that got generalized, isn’t something I can tell you. What matters is that on a current Windows install it’s a dead end, and it’s the dead end you’ll be pointed at first.

The real reason is architectural. Claude Desktop installs as a packaged Windows app, and packaged apps get their own private storage sandbox. Writes to the conventional locations get quietly redirected somewhere like this:

C:\Users\<username>\AppData\Local\Packages\Claude_<packageid>\LocalCache\Roaming\Claude\

Note the shape of that path. It ends in Roaming\Claude, exactly as documented. It’s just sitting under a package container that no reasonable person would think to check, which is why searching for the documented path fails while the documented path is, in a sense, still accurate.

The package ID is a short string of characters, and there’s no reason to guess at yours. Find it:

Get-ChildItem -Path $env:APPDATA, $env:LOCALAPPDATA, $env:USERPROFILE -Recurse -Filter "*claude*" -ErrorAction SilentlyContinue |
  Where-Object { $_.PSIsContainer } |
  Select-Object FullName |
  Out-File "$env:USERPROFILE\Desktop\claude-folders.txt"

That writes every Claude-related folder on the machine to your desktop. Open the file, look for the one containing local-agent-mode-sessions, and you have your real path. Two notes: run it in PowerShell, not Command Prompt, and expect it to take a minute or two while it walks the profile.

Depending on how and when Claude was installed, your data may genuinely sit in the plain %APPDATA%\Claude\ path. Both are possible. The one holding session subfolders with long UUID-style names is the one that matters. Don’t assume, check.

Claude Will Help You Move, and You Don’t Even Need to Buy It a Pizza Afterwards

Here’s the part that’s mildly funny in hindsight: the way to find all of this is to ask Claude, in Cowork, on the machine you’re migrating away from. It has shell access to the profile it’s running in. Point it at the problem and it will run the searches, read what comes back, and narrow the path down with you.

It will also send you to those two wrong folders on the way there, because the documented path and the actual path aren’t the same thing, and the documented path is what it reaches for first. That’s not a knock. It’s the ordinary pattern of working with these tools, and the fix is the same as always: make it show you the output rather than accept the summary. The searches above came out of exactly that back and forth. Claude proposed, the filesystem disagreed, and the third attempt found it.

Which is the useful lesson here, more than the folder path. The tool is genuinely good at the mechanical part, walking a filesystem, reading a directory listing, writing the PowerShell you’d otherwise be looking up. It is not reliable about where things live, because that’s a fact about your specific install and not something it can know from training. Use it for the legwork. Verify the destination yourself.

What to Copy and What to Leave

Open that Claude folder and you’ll find far more than sessions. Most of it is cache, crash logs, GPU state, and machine-specific scratch that regenerates on its own. Copying it doesn’t help and can actively cause conflicts.

Bring these:

  • local-agent-mode-sessions is the whole reason you’re here. Your Cowork session history.
  • claude_desktop_config.json holds your MCP server configuration. Lose it and every connector gets set up again from scratch. This is the one people don’t realize they needed until it’s gone.
  • claude-code and claude-code-vm carry Claude Code settings and VM data, if you use it.
  • vm_bundles for Claude Code VM bundles.
  • ChromeNativeHost for the Claude in Chrome integration.
  • config.json and cowork-enabled-cli-ops.json for general app and Cowork settings.
  • git-worktrees.json if you run Claude Code against git worktrees.
  • pending-uploads if anything is sitting in it.

Leave IndexedDB alone. It stores app state tied to your current signed-in session, and the new machine builds its own. Overwriting it invites problems you’ll spend an evening diagnosing.

Also grab C:\Users\<username>\.claude\ on the old machine. That’s a separate folder outside the sandbox, holding configuration and skills, and it goes to the same place on the new profile.

Zip It on the Old Machine

Rather than dragging ten items across a USB drive and hoping you got them all, package them once. Quit Claude Desktop completely first. Not the window, the app. Right-click the tray icon and choose Exit. Zipping live data files is how you end up with a corrupted history and no idea why.

Then, in PowerShell, substitute your own package ID into the first line and run this:

$Claude = "$env:LOCALAPPDATA\Packages\Claude_<packageid>\LocalCache\Roaming\Claude"

$Items = @(
  "local-agent-mode-sessions",
  "claude-code",
  "claude-code-vm",
  "vm_bundles",
  "ChromeNativeHost",
  "pending-uploads",
  "claude_desktop_config.json",
  "config.json",
  "cowork-enabled-cli-ops.json",
  "git-worktrees.json"
) | ForEach-Object { Join-Path $Claude $_ } | Where-Object { Test-Path $_ }

Compress-Archive -Path $Items -DestinationPath "$env:USERPROFILE\Desktop\claude-migration.zip" -CompressionLevel Optimal

The Where-Object { Test-Path $_ } line quietly skips anything you don’t have, so you can run it as-is whether or not you use Claude Code.

One detail that makes the other end easy: the archive stores paths relative to the items you named, not their full absolute paths. local-agent-mode-sessions\ and config.json land at the root of the zip. The archive is effectively a snapshot of the Claude folder itself, which means unpacking it is a single step with nothing to rearrange.

Then package the profile folder separately, since it lives elsewhere:

Compress-Archive -Path "$env:USERPROFILE\.claude" -DestinationPath "$env:USERPROFILE\Desktop\claude-dotfolder.zip"

Two zips on your desktop. Move them however you like: USB drive, OneDrive, network share.

If Compress-Archive throws a path-length error, that’s the 260-character Windows limit biting, and it’s plausible here given how deep the sandbox path already runs before your session UUIDs even start. Enable long paths in Windows, or fall back to robocopy with the /E switch to stage the folders somewhere shallow like C:\ClaudeMigration\ first, then zip from there.

Unzip It on the New Machine

  1. Install Claude Desktop, sign in, and open Cowork once. This creates the directory structure you’re about to unpack into. Then quit completely again.
  2. Find the package path on this machine. Run the same Get-ChildItem search from earlier. The package ID can differ between installs, so confirm rather than assume.
  3. Unpack straight into the Claude folder. Because the archive is relative to that folder, everything lands where it belongs on its own:
$Target = "$env:LOCALAPPDATA\Packages\Claude_<packageid>\LocalCache\Roaming\Claude"

Expand-Archive -Path "$env:USERPROFILE\Desktop\claude-migration.zip" `
               -DestinationPath $Target -Force

-Force overwrites files that already exist and leaves everything else untouched. The only collisions are the config files the fresh install generated a minute ago, and replacing those with yours is the entire point.

  1. Restore the profile folder:
Expand-Archive -Path "$env:USERPROFILE\Desktop\claude-dotfolder.zip" `
               -DestinationPath "$env:USERPROFILE" -Force
  1. Restart Claude Desktop. Your old sessions should appear in the sidebar.
  2. Reconnect your workspace folders. These are machine-specific paths and won’t follow you. Copy your project files first, then point Cowork at their new locations.
  3. Reinstall plugins that rely on local MCP servers. Cloud-only plugins come across with your account.
  4. Keep the old machine intact until you’ve verified all of it. Open a few migrated sessions, confirm your connectors work, and only then wipe.

The Same-Account Assumption

Everything above assumes one thing that’s easy to overlook: the Windows username is identical on both machines. When it is, every path inside those session files still resolves and the unpack just works.

When it isn’t, you have a problem the zip won’t solve. Sessions store absolute local paths, so C:\Users\OldName\Documents\... follows you onto a machine where no such user exists. The files arrive fine. The references inside them point at nothing.

If your usernames differ, this is the one case that needs an extra stop. Unpack to a scratch folder instead of straight into the target, run a find-and-replace across the JSON and JSONL files inside the extracted sessions to swap the old profile path for the new one, verify a single session opens correctly, and only then copy the corrected folders into place. Apply the same scrutiny if anything else moved, a different drive letter, a relocated Documents folder, OneDrive redirection that was on before and off now. Any of those breaks the same assumption in the same way.

Setting expectations honestly here: migrated sessions may land as readable history rather than fully resumable conversations. You get the record of what was discussed and decided, which is the part that took months to accumulate. Whether Cowork lets you pick up where you left off is a separate question, and one worth testing with a single session before you count on it for all of them.

With AI, Every How-To Has an Unknown Expiration Date

Everything above is a workaround for a design that’s already being replaced. That much is normal. What’s different now is that nobody can tell you when the replacement lands.

Software used to telegraph its changes. Version numbers, release notes, deprecation warnings, a beta period where the community wrote up what moved. A how-to written against version 4 stayed true until version 5, and version 5 announced itself. You could read a three-year-old post, check the version in the header, and know within seconds whether it still applied.

AI tools don’t work that way. They ship continuously, the client updates itself, and the thing you’re actually interacting with changes underneath a version number that may not move at all. Storage locations migrate. Local features become account features. The answer that was correct on Tuesday is wrong on Thursday, and nothing announces it. There’s no header to check.

Which is exactly what’s happening to this article. With sessions running remotely and tied to your account, the machine stops being where the work lives and becomes just a window onto it. Sign in anywhere and your history, your projects, and your scheduled tasks are already present. No folder to find, no zip to move, no package sandbox to go spelunking in. For anyone starting fresh today, this is already unnecessary.

That’s the right direction, and worth saying plainly rather than grumbling about the transition. Local-first storage bought you privacy and offline access at the cost of making your work a hostage to one piece of hardware. Account-first storage trades that the other way. Reasonable people weigh those differently, but nobody has ever been glad their context was trapped on a laptop with a failing battery.

The catch is the seam, and seams are always where the work is. Tools in transition leave a cohort stranded on the old model, holding data in a format the new model doesn’t automatically reach for. That cohort is anyone who adopted early, which is to say the people with the most accumulated context and the most to lose. Migration guides exist for the gap between what a tool used to be and what it’s becoming.

So treat this one, and every AI how-to you find, as perishable goods with no date stamped on the carton. Verify the path before you trust the instructions. If the folder isn’t where the article says it is, the article is probably older than the software. That will be true of this post eventually, and I can’t tell you when. Neither can anyone else. Until then, somebody has to move the boxes.

What This Actually Costs You

A little perspective on why any of us write these things down.

My first how-to post, back in 2002, was about building web services with Apache Axis. Working it out took the better part of two days: documentation that assumed you already knew the answer, examples that didn’t compile, and a long stretch of staring at a stack trace. The post that came out of it took a reader about twenty minutes. Two days of my confusion, compressed into twenty minutes of somebody else’s afternoon.

This one took about an hour of prompting and probing with Claude. It should take you fifteen minutes. Ten, if you run this post through an AI and ask it to strip out all my digressions, which I’d encourage, though I’d like it noted that the digressions are the part I enjoy.

The ratio held. It’s just faster on both ends now.

Fifteen minutes, then, against rebuilding context that took months to accumulate. That’s the entire calculation, and it’s the same calculation behind every unglamorous piece of groundwork in this business. Nobody gets excited about verifying a folder path. Everybody gets excited about the new laptop. The gap between those two feelings is where work disappears.

The broader lesson has nothing to do with Claude. Any tool you’ve spent real time training to your context is holding state somewhere, and “somewhere” is rarely where the marketing implies. Before you decommission a machine, ask what the tools on it know that the files don’t. Then go find out where they keep it.

Then wipe the old laptop. Not before.

If you found this interesting, please share.

Trust, But Verify: A Five-Stage Pattern for Working With Claude Code to Build Basic Modern Web Applications

Scott S NelsonLeave a comment

TL;DR: Claude’s default failure mode isn’t refusal, it’s confident, plausible-sounding claims that time-pressured developers don’t always check. Fixing that takes two moves: reset how eager Claude is to agree with you, then build a process that consistently performs independent verification and validation. Skip the first and the second one can create as much work as it saves.

The Failure Mode Isn’t What You Think

Everybody braces for Claude to refuse the task, hallucinate something wild and obviously wrong, or determinedly rewrite two-thirds of the codebase to implement a simple task that only requires eight lines of code. That’s not the failure mode that actually costs you time. The real one is quieter: Claude tells you something sounds right, in a tone that sounds checked, and it hasn’t been checked at all. Confident and wrong is a much more expensive combination than obviously broken. It gets worse under deadline pressure. When the sprint’s closing and the diff looks clean, verifying every claim feels like the one step you can afford to skip, right up until it’s the one step that would have caught the problem.

Anthropic says as much in its own documentation. Project instructions get “delivered as a user message after the system prompt… Claude reads it and tries to follow it, but there’s no guarantee of strict compliance” (How Claude remembers your project). That caveat runs both directions. It applies to what Claude is told to do, and just as much to what Claude reports about the work it already did.

Anyone who has done more than casual coding with Claude has experienced this pattern, even if it is either not recognized as such or treated as “that’s just the way it is.” A review of a recent project turned up incorrect references in CLAUDE.md that caused missed steps, leading to rejected PRs on good days and failed sprint demos on the not-so-good (not to mention the resulting rework).

For those who’ve read enough of these posts, you already know where this is going: behavior over tech. Yes, learning the nuances of continuously evolving AIs will help you be more effective and efficient with them. But it’s process discipline when using the tools that protects you from the pitfalls of an interface that has a “personality” faked through JavaScript algorithms and ethics centered around subscription renewal. That discipline only works if you fix the collaborator’s temperament before you fix the process around it, because a process built on top of an agreeable Claude just gets agreed with faster.

Reduce the Sycophancy Before You Reduce Anything Else

Here’s why temperament has to come first. Anthropic defines the problem plainly: sycophancy means “telling someone what they want to hear… rather than what’s really true, or what they would really benefit from hearing.” It “often manifests as flattery,” and sycophantic models “tend to abandon correct positions under pressure” (Protecting the wellbeing of our users). Claude’s default training leans toward exactly that. Stack it on top of confident-but-unverified claims and you get a collaborator that validates your plan instead of stress-testing it, right when stress-testing is the entire point of asking. Any process you build on top of that inherits the same flaw, because a rubber stamp is still a rubber stamp whether it happens once or five times in a row. Worse, that rubber stamp isn’t free. Five stages of going through the motions still cost five stages of your attention, prompts written, transcripts read, boxes checked, and none of it catches anything a five-minute skim wouldn’t have. That’s the second move creating as much work as it saves.

The fix is cheap (and easy!), and that’s the part people don’t expect. Independent write-ups on this converge on the same finding, a handful of lines in a global config file measurably change the behavior. In one documented before/after test, unmodified Claude hedged on storing API keys in frontend JavaScript (“that’s one approach… you might want to consider some security aspects”). Same model, same prompt, a few added instructions, and it flagged the same setup outright as a critical security flaw (How to Make Claude Stop Agreeing With Everything). This isn’t project-specific. Set it once, globally, in ~/.claude/CLAUDE.md (%USERPROFILE%\.claude for Windows users), the user-level file Anthropic documents as applying across all projects, as opposed to a project’s own CLAUDE.md, which stays scoped and version-controlled for the team.

If you already run personal instructions telling Claude not to flatter you and to push back when you’re wrong, you’ve got a head start. Here’s the block to layer on top of that, not underneath it:

# Our Working Relationship

- Don't validate an idea just because I proposed it. If my approach has a flaw, stop and flag it before proceeding, not after you've finished.
- I am sometimes wrong. Challenge my assumptions directly, and name what you checked to back up the challenge.
- Don't open with agreement ("You're right," "Great idea") or disagreement as a ritual. Build on the idea or move forward. Agreement is shown through action, not an announcement.
- Be matter-of-fact, not deferential or hedging. Challenge ideas, not people. Stay collaborative, not combative.
- Don't disagree for its own sake. Disagree when the evidence points that way, and say so plainly when I'm right too.
- If you don't know something, say "I don't know." Don't fill the gap with a confident-sounding guess.
- Be concise. Skip long-winded caveats and softening language.

The above is an example not to be adopted verbatim. It’s important to adapt it to your work style. There are other examples and references out there:

 

Test it before you trust it. Feed it a prompt with a deliberate architecture or security flaw and see if Claude catches it unprompted. And don’t overcorrect. Multiple sources warn that “be brutal” or “critique everything I say” produces combative, unhelpful output, not sharper output (How to Stop Claude From Being a Yes-Man (Get Real Pushback)). You want a collaborator with a spine, not an adversary you now have to manage. Again, keep in mind any external reference is only a starting point adapted from other people’s writing. Weigh it against your own results before you treat it as settled.

That fixes the temperament. It doesn’t fix what happens when a less-agreeable Claude still needs to catch a wrong assumption buried in a 400-line diff, or independently critique a plan without inheriting the blind spots of whoever wrote it. Temperament tells Claude how to disagree. It doesn’t tell you when to make it disagree. That’s what the five-stage pattern is for.

The Five Stages, and Why the Order Isn’t Arbitrary

Here’s the structure that puts a disagreement checkpoint at every point in a story where a confident wrong claim could otherwise slip through.

Plan. Hand Claude the story, ticket text, acceptance criteria, relevant screenshots, and point it at the right part of the repo. Use Plan Mode instead of letting it jump straight to code. What comes back should be a written plan: files and components to touch, explicit out-of-scope items, and every assumption about the codebase stated as verified, not guessed. Push back on any claim stated as fact without a tool call behind it. “Did you check that, or assume it from a similar component?”

Critique. This one has to come from somewhere else entirely: a second, unprimed Claude, a subagent or a fresh conversation, asked to critique the plan rather than the thread that wrote it. What you’re owed back is a list of concrete problems, or an explicit “no issues found” backed by what was actually checked. Never accept “looks good” without evidence. “What’s the strongest argument this plan is wrong?”

Implement. Read the actual diff line by line (see Noteworthy, below). Don’t just trust that it matches the plan. Claude owes you the code changes plus an explicit callout of anywhere the implementation deviated from the plan and why. If it deviates without flagging that, stop and ask why. Spot-check it: “you said you added a test for X, show me the file.”

Mechanical Check. Somebody has to decide whether a manual override, a disabled lint rule, an as any, a waived a11y rule, is legitimate or a shortcut, and that’s you. Claude’s job is real command output for every gate already listed in CLAUDE.md, not a paraphrased “should pass.” Never accept “this should pass” without the output. If Claude proposes suppressing a check instead of fixing it, ask why it can’t be fixed properly.

Acceptance Check. Final sign-off stays with a human, since it needs business and design intent Claude doesn’t have. Claude’s evidence is input here, not the verdict. What you need back, for each acceptance-criteria line, is a specific pointer to where it’s satisfied, not a blanket “all done.” Go line by line through the criteria. “What isn’t covered?”, not just “is it done?”

Look at that sequence again and it isn’t arbitrary. The first three stages front-load verification before and during coding, because catching a wrong assumption at Plan or Critique costs a sentence of pushback. Catching that same wrong assumption after merge costs a revert, or a wrong instruction sitting in CLAUDE.md for weeks before anyone notices. That’s the ROI case in one line: a sentence now, or a revert later.

Critique only works as an independent pass, not a second look from the same context, which is exactly why it comes right after Plan and not later. A plan reviewed by the mind that wrote it inherits that mind’s blind spots, no exceptions. That’s the whole point of handing it to an unprimed Claude: it approaches the plan with a beginner’s mind, no investment in defending what it already wrote, nothing to protect. I’ve watched teams skip that and call it Critique anyway, rerun the same conversation, get the same agreeable nod, file it as reviewed. Somewhere there’s a sprint board with a checkbox for “AI code review” that has never once been unchecked.

Mechanical Check comes after Implement for a reason too: it’s the one stage that shouldn’t require judgment at all. It’s just the codification of CLAUDE.md‘s existing mandatory-checks section, run every time. But it’s only as trustworthy as that section is accurate, and an out-of-date gate list is worse than no gate list, because it creates false confidence that enforcement exists when it doesn’t. Every override or new check has to get written back into CLAUDE.md the same day it happens, or the drift starts immediately.

Acceptance Check is last and stays human-owned on purpose. Passing every mechanical gate proves the code is well-formed. It proves nothing about whether the right feature got built, and that judgment call needs business and design context Claude doesn’t have full visibility into, no matter how well its temperament is tuned or how many stages came before it.

The first four stages verify the change, proving it was built right. Acceptance Check validates it, proving it’s the right thing to have built at all. Skip either half and it still shows up in production, no matter how clean everything upstream looked.

The Bottom Line

Claude states things confidently by default. Per Anthropic’s own guidance, there’s no guarantee of strict compliance even with explicit instructions. Behavior over tech, one more time: temperament and process aren’t features Anthropic ships. They’re the parts you’re responsible for, whether it’s a five-line function or a whole story, demand evidence over assertions, and make Claude argue against its own work at least once before calling it done.

Speaking of done, AI is a long way from stabilizing. Use articles like this as fuel for thought processing rather than a one-and-done punch list. Keep reading, keep thinking, and share what works.

Noteworthy

One more tool worth bolting onto this: OwnDiff turns the “read the diff line by line” instruction from the Implement stage into something that can’t be skipped. It’s a local human-review gate. It scores the current git diff for risk, and for anything medium risk or above it has the coding agent generate diff-grounded multiple-choice questions, then refuses to let the agent push or open a pull request until a human answers every one correctly. No web search, no outside facts, no generic filler questions, everything grounded in the actual changed files.

That’s the same idea as Acceptance Check, just enforced mechanically instead of by habit. “I read the diff” is a claim. Passing a quiz generated from that diff, with the push blocked until you do, is evidence. Worth a look for the days you’re moving too fast to hold yourself to the honor system (Thanks to Ed Lyons for pointing me at this one.)

If you found this interesting, please share.
Robots looking for data with varying degrees of success

AI Treats Your Documentation as Data. You Should Too.

Scott S NelsonLeave a comment

TL;DR: Enterprise AI runs on enterprise data, and that includes the unstructured data in the form of documentation that is poorly curated. No taxonomy, revisions sitting next to finals with no way to tell which one is current or still relevant. If data is the DNA of modern business, documentation is the dominant chromosome that can make the difference between robust health and questionable viability.

I post a lot about the value of architecture and training in AI adoption, and share posts by people focused on the nuances of prompting because these things will make a difference. Data cleanliness is a topic I usually just point people toward other people’s thinking on because my days focus more on how data moves and evolves than on structuring and managing it. As a long-time generalist, I have a deep appreciation of the added value specialists bring to the process. Documentation is a different kind of data, with its own unique headaches. I’ve harped on some of those issues for years: why self-documenting code doesn’t produce a self-documenting solution (From Agile to Fragile in 60 Sprints), why nothing gets read that never gets written down (If It Is Not Written Down It Does Not Exist), and why a taxonomy nobody maintains is worse than no taxonomy at all (Failure to plan communications is communicating a plan to miscommunicate, one that I wrote at the dawn of the current age of AI without realizing how it would soon become even more important).

Then it occurred to me that documentation and data cleanliness are the same conversation, and I hadn’t heard many people say so directly. Documentation is data, and not just any part of it. It’s the chromosome that expresses the rest of your enterprise data, the one that decides whether everything built on top of it turns out healthy. It’s also usually the part in the worst shape. That’s the case I want to make here. (Confession: the connection came from something I read that mentioned it in passing, and I never noted the source. Whoever you are, kudos.)

AI needs three things from your data, in order:

  1. Access.
  2. Understanding.
  3. The ability to apply what it understands to the specific context of whatever someone just asked it.

Skip a step and the most expertly crafted prompt or the best planned architecture in the world still only gets you an expensive autocomplete with a confident and flattering tone, because both of them are working with whatever access and understanding they’ve been handed.

There’s a lot written about the mechanics of step one: RAG, vector stores, and running SharePoint exports through Pandoc before they ever reach a prompt, since raw Word and PDF files carry a lot of baggage a model doesn’t need to see. Step two has gotten a lot less attention. Understanding requires that all of the documentation your AI is referencing provides meaning and consistent messaging, and the usual disarray of enterprise documentation doesn’t.

AI is a (New) Good Reason to Clean Up Your Documentation

This was already a problem in 2022, back when the audience for the complaint was a project manager, not a language model. Templates that auto-update their “last modified” date every time the file gets saved, whether or not the content actually changed, so the date stops meaning anything (Replace Auto Dates in Templates). It gets worse at the platform level. SharePoint and Teams will happily stamp a document “Modified” the moment someone opens it, whether they changed a single character or not, because the file was never stored with the “open as read-only” flag set. Nobody sets it, because almost nobody thinks about it, and now your most trustworthy-looking piece of metadata is lying to you and to anything reading it downstream.

Then there’s version chaos stacked on top of date chaos. A SharePoint draft with one clear owner on paper, quietly edited by people who didn’t know that, discovered only after someone had to re-verify the entire document line by line (Recovering Previous Versions from SharePoint). That gripe about config files requiring “reading documentation, which is only read less than it is written” is sixteen years old (Dynamic Log Location for log4j). That was a joke about developers skipping the manual. It reads differently now that the thing skipping the manual is a model that can’t lean over and ask a coworker what the doc meant to say.

None of that is a new problem. What’s new is that AI has zero tolerance for it. A human can walk into a shared drive, eyeball three files named some variation of “Process_Final_v2_ACTUAL_FINAL,” and guess correctly which one to trust, because they have context: they remember the meeting, they know who owns the process, they can just ask. AI doesn’t get that fallback. It reads what’s in front of it and treats every file as equally authoritative, including the wrong one.

If This Sounds Familiar…

Turns out I’m not the only one who noticed. Other people are seeing the same problem from angles most enterprise practitioners rarely get access to. Amit Shivpuja, who runs data and AI enablement at Walmart, wrote in Forbes that he watched a fully governed, well-modeled AI program produce inconsistent results anyway, and traced it to what he calls the missing documentation layer: the context that should have been captured during requirements, design, and testing, but instead lived in a Slack thread or in someone’s head (The Hidden Barrier To Enterprise AI: The Missing Documentation Layer). His diagnosis lines up with a decade of watching the same pattern play out: humans compensate for missing documentation with tribal knowledge. AI can’t.

The gap between AI investment and AI payoff backs this up at scale. 79% of organizations report real challenges getting AI to deliver, a double-digit jump from the year before, even as most are raising budgets to feed it (WRITER). Separately, only 32% of organizations report sustained business impact from AI despite 86% of the C-suite increasing investment, according to an Accenture survey (Forbes). Nobody breaks that spend down into prompt engineering versus architecture versus training, but odds are good most of it lands in exactly those three buckets. It’s not usually the prompts, the architecture, or the training that’s failing. It’s the context underneath all three.

How to Get Started

None of that makes training, architecture, or prompt engineering optional. They’re not, and treating them as afterthoughts would be its own kind of mistake. Training your people is worth the time and the awkward learning curve that comes with it, every time. Architecture done right is what lets a foundation hold up for years while the technology running on top of it changes every few weeks, so it’s worth building solid instead of patching forever. And prompt engineering isn’t dead, whatever this month’s headlines are claiming. Knowing how to ask well is still what gets the new, flashier capabilities to actually do what you meant instead of what you typed.

But the potential of all that value depends on how well it works with your enterprise data. We’re past the point of needing to prove AI will benefit the business. We’re all now in the throes of how it will benefit the business, and that how stays severely limited until you put your data in order. Here are a few tips to get you started.

Make documentation part of done, not an afterthought to done. Shivpuja’s Forbes piece gets this right: a story or a feature isn’t finished until the meaning, the rules, and the assumptions behind it are captured somewhere a model can find them. That’s a process change, not a tooling purchase.

Default to read-only. Force intent for edits. If a document is finished, save it that way, and make someone actively choose to reopen it for editing. This has been true since 2022, and the fix hasn’t gotten any harder to implement. Just more expensive to keep skipping.

Give it a taxonomy and a living Read Me, and actually maintain both. This isn’t new advice. Generative AI just raised the stakes on it: organize things so someone new can find their way around without a tour guide, pin an explanation of that structure somewhere obvious, and keep it current as the team and the work change (Failure to plan communications). A taxonomy that was accurate in 2022 and hasn’t been touched since is worse than no taxonomy at all, because it still looks trustworthy.

Attach context to the asset, not to a folder that might get reorganized next quarter. Documentation living next to the process or dataset it describes survives longer than documentation living in a wiki page someone has to remember exists.

Convert for the machine, not just for the human. Pandoc, or whatever your equivalent is, exists to clear that baggage out before it hits a vector store. Word and PDF files are full of formatting decisions that made sense to a human editor and mean nothing to an LLM. Fifteen minutes of conversion saves a RAG pipeline from tripping over someone’s decade-old formatting habits. This is architecture work too, for what it’s worth, just the boring kind that doesn’t show up in a vendor pitch.

Set a review cadence, and mean it. One knowledge-management vendor’s own market analysis cites a 2025 Gartner study putting the number at 60% of internal knowledge articles going stale within six months, with only 14% of teams auditing content on any real schedule (source, with a grain of salt: it’s a page selling a fix for the exact problem it’s describing, but the shape of the number matches what shows up across most enterprise environments). A tool that flags stale content is nice. A team that actually looks at the flag on a schedule is the part that works. Buying a self-updating knowledge base without building the habit of using it just moves the swamp to a nicer-looking pond.

If Data Debt were a Thing…

There’s a term for this that architects already understand: technical debt. Every shortcut taken to ship faster accrues interest, and the bill always comes due, usually at the worst possible time and for more than the original shortcut would have cost to do right.

Documentation disarray runs on the same math. Call it data debt. It accrues every time a taxonomy goes unmaintained, a Read Me goes stale, or a “final” draft ships without anyone reconciling it against the other four drafts sitting next to it. None of that shows up on a balance sheet, so nobody budgets against it. But your prompts, your architecture, and your training program are all paying interest on it anyway, every time they inherit whatever your documentation can actually support.

Pay it down early and it stays cheap: a taxonomy tightened now, a template’s date field fixed before it propagates through a hundred more copies. Let it ride, and the interest compounds. More conflicting drafts pile up. More tribal knowledge walks out the door with the people who had it. More AI output gets built on documentation that was already lying to you, and the eventual fix means untangling years of it instead of an afternoon of it. Ignore it long enough and it doesn’t just get expensive, it bankrupts the whole initiative: the AI program that never delivered, the budget that got pulled, the “we tried AI and it didn’t work” verdict that was actually a documentation problem wearing an AI costume.

Fix the documentation first. It’s not sexy, nobody gets interviewed on a podcast for cleaning up a taxonomy, and it’s exactly the debt payment that keeps the rest of the investment solvent.

If you found this interesting, please share.

Meet Deadlines and Manage Technical Debt with AI-Assisted Architecture

Scott S NelsonLeave a comment

tl;dr: New platform. Deadline. The instinct is to move fast and clean it up later. That’s where technical debt is born. A well-constructed Claude project, loaded with curated platform documentation and queried with the experience to know what to ask and how to evaluate the responses, tactically compresses the ramp-up without sacrificing strategic design principles.

The Sharp Fork in the Road

Every architect and engineering lead who has given a project to deliver on a new platform or with new technology under deadline pressure knows this fork. Pushing for proper preparation can get you marked (ironically) as a risk from the leadership perspective. Plowing forward using old techniques without understanding the new nuances keeps you up at night…either knowing you are missing something up front, or fixing what you didn’t know during the final death-march phase of a waterfall project that just happens to use Kanban boards, daily stand ups, and sprint ceremonies.

One path: move fast. Learn just enough to ship. Ask support when you hit a wall. Request exceptions when you hit limits. Get it working and tell yourself you’ll revisit the architecture when there’s more time. (There is never more time.) What you build in that mode becomes the foundation everything else is built on, and the cost of fixing it compounds with every sprint.

The other path: slow down. Read the documentation properly. Understand the platform’s constraints before you design around them. Make the right call the first time. This is correct and often impractical. Deadlines are real. The platform is new. The documentation is dense. The team is waiting.

The Contentstack project that prompted this post took the first path and ran into a SaaS governance constraint that happens to be measured recursively. The first time it was hit, the response was typical for teams working with a new SaaS vendor and release date that was set before the first line of code was written: Ask for an exception. Which was granted, hit again and raised again. Fortunately, the third time it happened, an experienced vendor support manager recommended reviewing best practices to avoid the issue. And an experienced architect was on the receiving end of that suggestion, one who had previously dealt with a Salesforce solution that went down three months after launch from relying on similar exceptions.

This post is not about Contentstack architecture. It is about the challenge many teams face with balancing target dates and defensive design decisions, and a tool set to apply in order to keep from tipping too far in either direction.

Claude as a Platform Research Partner

Giving Claude access to a curated set of platform documentation and then working interactively to explore solutions is not a replacement for architectural experience. It is an accelerant for it. It is also not a way to do away with architects or the inclusion of design tasks at the feature or story level. It is how to fulfill the expectation that AI can provide ROI immediately when applied by experienced technologists.

These distinctions matter. It’s never about “ask Claude what to do” (because if you need to ask “what” you won’t know how to ask “how” when the time comes). It is “I understand how systems like this behave, I know which constraints are likely to compound, and I need to move through the analysis faster than I could alone.” Experienced architects and engineers bring the judgment: familiarity with how content models fail at scale, how schema resolvers typically handle recursion, how vendor-imposed limits usually reflect real constraints in the underlying system. Claude brings the recall, the scripting, the cross-referencing, and the tireless patience for the kind of recursive schema analysis that would take a senior engineer the better part of a day.

For those that follow my posts you know that I will often describe theoretical solutions backed by a combination of personal experience where they would have worked linked to examples from others who demonstrated that they work. In this case the experience came before the theory, working backwards from a result where I noticed the process while documenting the solution (because, hey, that is what architects do after they solve something).

The working example was with a Contentstack implementation. It took one focused 2-hour session to identify an obscure root cause, define a strategic solution, discover other areas to apply the same solution, and identify where the solution would cause more harm than good. A second 30-minute session was applied after the first round of refactoring to validate the impact and prioritize the remaining effort. Before Generative AI, this would have been several days of effort that would not have been attempted until the risk was realized in production delay.

The Project is the Architecture

Before a single question gets asked, the project has to be built. This is not setup overhead. This is the work.

A blank Claude chat window and a well-constructed project will give you very different results on the same question. The difference is not the AI. It is the knowledge boundary, the taxonomy, the instructions, and the accumulated session output. Strip those away and you have a general-purpose assistant guessing at context. Keep them and you have something that behaves like a senior researcher who has been on the project for months.

What goes in the project folder:

Downloaded documentation as markdown files, not links. Links go stale, require fetches, and introduce latency. Pull the platform docs that matter, save them as markdown, put them in the folder. For Contentstack: the Global Fields limitations page, the Content Modeling Best Practices guide, the Custom Fields documentation. Not every page in the docs. The ones relevant to the work. Knowing which ones matter is the first place architectural experience shows up.

Actual data from the platform. In this case, exported stack JSON. Claude can read it directly in the sandbox, run scripts against it, and cross-reference findings against the loaded documentation in the same session. That combination of curated docs and live data is what makes the diagnosis precise instead of speculative.

Session summaries. After each working session, have Claude produce a structured summary: the original problem, the conclusions, the evidence, the next steps. That file becomes the cold-start document for the next session. You don’t re-explain the context. You hand Claude the prior session’s output and continue. The knowledge compounds.

At some point (again, much of this requires human intuition gained through real-world experience), have Claude work with you to turn the summaries into a skill scoped to the specific platform, technology, or tool so that when they are in context these lessons learned will be applied automatically going forward.

The Taxonomy Is Not an Afterthought

Separate downloaded reference content from working session output. Nest folders by topic. /reference/, /sessions/, /data/ serve different purposes and should live in different places. This is not pedantry. It is how you make the project instructions work correctly, and how you find things six weeks later without rebuilding context from scratch.

If the platform has extensive documentation, don’t try to enumerate allowed URLs in the project instructions directly. Create a reference-urls.md, or per-topic files like contentstack-docs-urls.md, with an annotated, categorized list of approved sources. Claude works from the list. You maintain the list. It stays current and searchable.

The discipline compounds the same way the session summaries do. A well-organized project from session three makes session fifteen faster than session one.

The Project Instructions Are the Rules of Engagement

The instructions define how Claude behaves inside this knowledge space. Three things they need to do:

Challenge assumptions. If a question implies something not supported by the loaded documentation, say so. Don’t fill gaps with plausible-sounding answers. The most dangerous thing a research assistant can do is answer confidently on insufficient evidence. This instruction eliminates a whole category of hallucination risk before it starts.

Point out mistakes. If the framing of a problem is wrong, say so. This is the instruction most people skip and then complain about later. You want an assistant that pushes back, not one that validates your bad hypothesis and helps you build a case on sand.

Limit web searches to specific URLs. Unconstrained web search in a technical investigation introduces noise: outdated content, inconsistent sourcing, SEO-optimized answers that aren’t accurate. Lock it down. Specify which domains are permitted. For a Contentstack project, that’s contentstack.com/docs. Everything else requires explicit permission. If the approved URL list is long, store it in a markdown file in the project folder and point the instructions at it.

This Requires an Architect

Here is the part that does not get said enough.

You cannot point Claude at an unfamiliar platform, load a few docs, and expect it to diagnose architecture problems. You can try. What you’ll get is fluent, confident, and partially wrong.

There are many engineers capable of setting this up. The value of an architect doing the work is separation of concerns in roles. The architect’s role is to nail down processes and choices that allow engineers to focus on the best way to apply them.

In our Contentstack use case, the single session worked because the person directing it brought a deep understanding of adjacent technologies and the experience to know both what to ask and how to evaluate the responses. Specifically:

  • Recognizing that the error message pointed to a schema limit, not a code problem, because that’s how content platform resolvers typically surface constraint violations
  • Understanding that “recursive” in the documentation meant multiplicative compounding, not additive, based on how similar systems handle nested references
  • Knowing the fix had to leave the content model intact for authors, which ruled out several otherwise obvious approaches
  • Reading a Claude-generated Python script’s output and recognizing that the confident result provided the first time was due to looking in the wrong parts of the schema
  • Looking at a before/after instance table and determining whether the fix was actually complete or just moved the problem

None of that knowledge lives in the documentation itself. It transfers in from adjacent experience: content modeling, schema design, how platform resolvers work under the hood. Claude surfaces the platform-specific detail. The architect determines what it means.

The tool doesn’t replace experience. It supercharges it with speed and specific knowledge.

The Interaction Pattern

What the Contentstack session actually looked like, stripped of the platform specifics:

  1. State the problem. Provide the evidence: the error message, the exported schema, the documentation.
  2. Claude generates a hypothesis. Test it against the data.
  3. Diagnostic script written and run in the sandbox.
  4. Root cause confirmed. Fix designed. Impact predicted before any schema changes are made.
  5. Fix implemented. Follow-up session loads the new export and verifies the result.
  6. Summary file created. Next session’s candidates identified.

No magic. An architect with relevant adjacent experience, a fast and patient research partner, and a well-stocked project folder.

Prompts That Did Actual Work

These are worth examining because the techniques transfer to any platform.

“Describe in detail the cause of home_page_template having 24 instances, and instances of what?”

The second half of that question is the important part. Asking Claude to define what it is counting before giving the count forces precision on both sides. In technical sessions on an unfamiliar platform, jargon can mask shallow understanding without anyone noticing until the fix doesn’t work. The ability to ask that follow-up, to know that “instances” needed a definition before the number meant anything, comes from having debugged similar problems elsewhere. Use this pattern whenever an answer could be technically correct but operationally ambiguous.

“Create a summary file to feed to the next analysis session that includes the conclusions from this session combined with the original inputs. Format and sequence the file so that the next session can be as efficient as possible.”

Besides being familiar with adjacent technology, experience solving complex issues with Generative AI is why this is an approach for architects and engineers. Yes, Claude will now start compacting sessions on its own to improve efficiency, but having the sense that it is time to move to a new session is again an area where human experience beats relying entirely on the AI.

This prompt converts a working session into a durable asset. The phrase “format and sequence for efficiency” is carrying real weight: it tells Claude to think about how the file will be consumed, not just what it contains. The output becomes the cold-start document for the next session. Without it, every session re-derives context the previous one already established.

“Read the attached to get full context of the original issue, then review the contents of [folder] and determine if and how the issue has been improved.”

Sequencing does the work here. Claude gets the full prior-session summary before it touches the new data, so “improved” arrives with a precise definition attached. Without that order, it analyzes the new export without knowing what it’s comparing against. Prime with context before assigning the task, every time.

All three follow the same pattern. Context before task. Output format stated up front. It is not a methodology. It is just how you would brief a colleague who needs to be useful on short notice.

The Setup Is the Differentiator

Two teams, same platform, same error.

Team A has Claude. No curated project, no loaded docs, no taxonomy, no instructions. They get generic answers that feel helpful until they don’t hold up under the actual constraints of the platform.

Team B has a project built by someone with deep experience in adjacent technologies, content modeling, schema design, API behavior under constraint, who knows both what to ask and how to evaluate what comes back. Downloaded reference docs. Exported platform data. Session summaries that carry forward. Instructions that push back on bad assumptions.

Team B gets a root cause analysis, a fix, and a forward-looking roadmap. More importantly, they get it without accumulating the kind of structural debt that shows up six months later as an emergency.

A Note about Choosing Cowork

What I’m describing is not the typical use case for Claude’s project-based workspace. It is aimed at knowledge workers automating routine tasks: organizing files, generating reports, drafting communications. Productivity stuff. This is not that.

My choice of Cowork is based on my day-to-day work being mostly in documents and decks. This could also likely be done using Claude Code in an IDE for those that prefer that interface.

I became aware of how far outside the lines I was operating when someone asked what tool I was using, I explained it, and I watched the look on their face. You know the look.

I have been here before. I spent years using JMeter for continuous functional and regression API testing, which is not what JMeter is for. JMeter is a load and performance testing tool, and there are entire communities of people who will tell you this. They are correct and also missing the point, because once you understand how JMeter handles realistic randomized inputs and configuration-driven test selection, you end up with one codebase doing the work of four. I wrote about it. People told me I was doing it wrong. The tests kept passing, so.

It is common to analogize the similarities between physical tools and technical tools. “When all you have is a hammer, everything looks like a nail”, and “You can use a screwdriver as a chisel, but you really shouldn’t.” I’ve often used those myself. But the opposite analogies are also true. Most tools can be a weapon, and many tools can have multiple uses. While screwdrivers are still terrible chisels, some are great prybars, hole punches, and, yes, weapons. Same with software. Excel has spellcheck, but I’d never paste text into it before posting to a blog, but I have used formulas to parse text rather than writing a script to apply regex rules because it is faster and just as accurate. Use your tools to the extent of their value, and don’t underestimate their value or your ability to innovate.

If you found this interesting, please share.