Support Agents Can Increase Deflection…and Defection

TL;DR: Large Language Models (LLMs) are excellent at retrieval but fail at self-awareness. Without a structured hand-off architecture, they become expensive loop-engines that drive users to competitors.

The Evolution of Deflection

Support deflection is a three-act play where the customer usually loses.

It started with static FAQs. Digital filing cabinets where users did the manual labor of hunting for answers.

Then came semantic support search. This used natural language processing to map a query to a relevant article.

Eventually, we got chat bots that acted as conversational proxies for that search. Each step was pitched as a win-win: faster answers for the customer and massive overhead savings for the business.

The problem is the “savings” side of the equation. Businesses have prioritized it so aggressively that automation is now being forced beyond its logical limits. We have moved from helping the user find an answer to preventing the user from reaching a human at all costs. (This is usually where the ROI calculation ignores churn.)

A recent example of this friction is an interaction with the Substack support agent.

The Context Leak

In the first interaction, the user uploads a PDF detailing a failure. The bot acknowledges the delivery failure but immediately drops the core issue (the fact that this was a Substack-recommended configuration that failed).

BOT: The screenshot you uploaded shows that an email you sent to an author at their @substack.com address was not delivered because it was flagged by Substack's automated security protections. To connect with Substack writers, please use the comments, chat, or direct message features by creating a Substack profile or using the Substack app. Let me know if you need help with any of these options or have other questions about contacting authors.

This is a classic context window failure. The bot is reading the “now” but has forgotten the “why.” (Pro tip: Support bots require a persistent Problem Summary in the system prompt to prevent the initial state from being pushed out by chat history.)

The “Loop of Death”

The next phase is the retrieval loop. The bot recommends settings the user has already confirmed are active.

CUSTOMER: I set up Stripe and want to change the email address to my substack address as recommended but email sent to that address are being refused. Content attached and below (in case you block attachments). BOT:Your Substack alias (scottsnelson1@substack.com) is recommended for use as the Stripe support email, but direct emails sent to this address may be blocked by Substack's automated security protections. To ensure you can receive replies forwarded from your Substack alias, set

The bot suggests setting “Who can reply” to “Everyone.” The user confirms this was done before the test. The bot’s response is a generic “thanks for confirming,” followed by an admission that its own security protections might still block the mail.

CUSTOMER: It is already set to Everyone and was before the test. BOT: Thank you for confirming that your

This is the architectural equivalent of a car that won’t start because it’s out of gas, while the manual keeps explaining how to turn the key.


Logic vs. Retrieval

LLMs typically prioritize the most relevant documentation. The “gotcha” is that documentation often contains circular logic or edge cases the model isn’t weighted to resolve.

When the user explicitly asks how to use the recommended address if Substack is going to block it, the bot resets to a template. It is no longer “solving.” It is just “responding.”

CUSTOMER: How can I use the substack address for Stripe if substack is going to block the emails sent to that address? BOT: Your Substack alias (scottsnelson1@substack.com) is intended to forward replies from your newsletter readers, not to serve as a general-purpose support inbox for all external emails. While we recommend using your Substack alias as the Stripe support email for privacy, Substack’s automated security protections may block emails sent directly from services like Stripe or other automated systems, meaning you might not receive those messages. If you need to ensure you receive all communications from Stripe, it’s best to use your main account email or another address you control as your Stripe support email. Would you like guidance on how to update your support email in Stripe, or do you have other questions about managing your publication’s contact details?


Tactical Breakdown

The agent failed to pivot. It was out of its depth by the second prompt, but it lacked the internal logic to admit it. Instead of acknowledging the mismatch between the “recommended setup” and the “blocking error,” it reverted to a script.

The ultimate UX insult happened at the end. After a prolonged exchange full of specific data and a technical PDF, the bot asked the customer to “describe the issue” to conclude the session. It essentially hit ‘Refresh’ on the customer’s frustration.

CUSTOMER: I would prefer you fix your invalid instructions so other people don't have to be aggravated like this in the future. BOT: Thank you for sharing your feedback about the instructions for using your Substack alias as the Stripe support email. To help us address this issue, could you describe the specific bug or confusion you encountered with the Substack product? For example, let us know exactly what happened when you tried to use your Substack alias for Stripe support, and what you expected to happen instead. Your detailed description will help us better understand the problem and improve the experience for other users.

Even when the user mentioned leaving for a competitor, the bot stayed in its polite, useless loop. It missed every red flag in the conversation.


Architecture: The Exit Ramp

This is a design failure, not a model failure. You cannot expect an LLM to manage its own boundaries. You have to build them. (Think of it as adding a “break glass” protocol to your chat logic.)

The Resilient Support Stack

  1. Sentiment-Based Escalation: Use a secondary, lightweight model to monitor the chat for frustration signals. If terms like “frustrated,” “waste of time,” or competitor names appear, kill the bot and ping a human.

  2. Repetition Kill-Switch: Track the number of times a specific documentation link or advice block is served. If the bot repeats itself twice, it has failed. Trigger an automatic hand-off.

  3. Persistent State Management: Store the Original Problem Description outside the chat context window. Force the bot to reference this core state in every turn so it doesn’t forget why the user is there.

  4. The “Depth” Check: If a user’s prompt complexity (like an uploaded log file) exceeds a certain threshold, the bot should immediately preface its response with: “I’m looking at your logs. If I can’t solve this in two steps, I’ll get a human involved.”

Architecture is the differentiator between a tool and a liability in AI adoption. The model is just a component. The success of the implementation depends on the plumbing that surrounds it.

If your current AI adoption feels more like a liability than a multiplier, or is stalled by FUD, Contact Me to discuss options.

If you found this interesting, please share.

© Scott S. Nelson

Learning AI and Going Broad or Deep First

It depends.

Yeah, I hate that answer, too, but it’s because we all prefer simple answers to the real ones. We also want to believe in overnight success, one size fits all, and that the plug and play option is all we need. And don’t get me started on long term weather predictions!

But it helps to know what it depends on, which is, in this case, where you are in your AI journey. The same approach really applies to any learning journey where there are multiple aspects, so we’re going to start with looking at it simply from the perspective of learning.

Why and How and When to Start Deep

If this is your first foray into a new realm of knowledge, start by going deep on one aspect.

Pick the area that you are most interested in. Intrinsic motivation is a better driver for learning than any reason that includes “have to”. Once you have picked that topic, dig in and follow your curiosity until you feel you can converse freely on the topic. This is how you build a baseline mental model.

Going deep in one specific corner makes other adjacent areas easier to absorb later because you actually have a frame of reference to hang new information on. When you encounter a new tool, you filter it through existing mental models to facilitate integration of new knowledge. This cognitive filtering means you aren’t starting from scratch every time a model updates. You are simply updating a specific branch of an existing tree. (See The Memory Paradox: Why Our Brains Need Knowledge in an Age of AI)

The Pivot to Breadth: Mapping the Landscape

Once that baseline mental model exists, going broad is more valuable.

This first accumulation of breadth is to understand what’s possible, or available. You aren’t trying to master everything. You’re mapping the space so you know where the boundaries are. This aligns with the “T-shaped professional” model, defined by having deep expertise in a specific area while also possessing broad knowledge across various disciplines. This structure ensures you have enough technical depth to contribute high-value work immediately. It also gives you enough breadth to collaborate with experts in adjacent fields without needing a translator.

Going broad makes it easier to know exactly when and where to go deep next. Knowing what exists and what is possible makes it easier to say “I have an idea of how that can be done” with conviction.

The Trap of Constant Depth

The problem with going deep on one thing at a time, after the initial deep dive, is that when you need the knowledge or skill in a practical situation, there may be something adjacent that will make it easier or is better suited.

If you’re buried in a single silo, you won’t see it. This is why pure specialists struggle when their niche technology shifts. Markets move faster than individual mastery, which is why modern engineering organizations must embed specialists into existing teams. Breadth prevents you from becoming a legacy asset the moment your specific tech is disrupted. It provides the foundation for transferring implicit knowledge, which is the exact kind of knowledge needed to generate creative ways of tackling business problems. Innovation happens at the intersection of two unrelated fields.

Managing the Hierarchy of Ideas

To move between breadth and depth effectively, you have to understand how to categorize information. A  practical framework to understand how to conceptualize those categories in a given realm of knowledge is The Hierarchy of Ideas. This concept allows you to mentally zoom in and out of a topic. It ensures you are always operating at the exact level of detail the current problem requires.

Think of a hierarchy using transportation as the frame. At the top, you have the abstract concept of “Transportation”, which includes planes, boats, trains, cars, skateboards, and ox carts. Moving down a level, you find “wheeled vehicles”, which is still broad enough to encompass trains and scooters. Further down, “Cars” will include internal combustion, electric, and peddle powered. As a mechanic, you will be more interested in learning the distinctions between a Ferrari and Hyundai, or between the Sonata and Kona. The higher you go, the more general and broad the idea becomes. The lower you go, the more specific and detailed it gets.

Navigating this hierarchy is done through “chunking.” When you chunk up, you move from the specific Tesla to the broader category of “Transportation” to understand the big picture. When you chunk down, you move from the general concept of “Cars” into the specific components like the “Battery Management System” to find depth. You can also chunk laterally, moving from “Cars” over to “Trains.” This allows you to find alternative solutions that exist at the exact same level of utility.

The AI Sandbox: Applying Levels and Chunks

AI is like transportation in the way that zoology is like geology. They aren’t one giant subject. There’s a hierarchy of distinct concepts, applications, audiences, and values that you have to navigate intentionally.

Start by chunking down into a specific primary aspect. Dive into development. If you choose Software Development, don’t just use a generic chatbot. Master how tools integrate directly into the developer’s workflow. Modern development is shifting toward a model where the AI handles low-level syntax. The human engineers and architects manages high-level logic and security.

If you choose Marketing, dive into tools capable of predicting future trends. These platforms move you from general demographic targeting to individual-level behavioral forecasting in real-time. This creates your first deep anchor.

Once you feel steady, chunk up. Skim through news and articles about the overall space so you get a sense of the capabilities. Map the broader landscape—from vector databases to multimodal generation. Staying informed at this high level prevents you from getting blindsided by architectural shifts.

As you build that breadth, you chunk laterally. Then, when something comes up that would benefit greatly from an aspect other than your first specialty, you will recognize that your current focus isn’t the right one in that context. If you are deep in software development but hit a bottleneck in data quality, your broad map will point laterally toward data architecture. You will have a good idea what aspect is better suited. Then you can partner with someone that has that expertise, or learn it deeply yourself, or both.

Effective mastery requires building a foundation deep enough to create your mental anchor, while maintaining a wide enough perimeter to spot the right tools for the job. You cannot specialize into obsolescence and expect to stay relevant in a field that moves this fast. Whether you are ready for your first technical deep dive or you are currently gathering seeds for future growth, the only wrong move is standing still. Pick a starting point and get to work.

For your convenience (plus, I hate to throw away interesting artifacts that AI outputs when researching my articles), below are some areas (i.e., non-exhaustive) to consider when conceptualizing the Hierarchy of Ideas around AI.

AI Deep Dive Reference Table

Specialization Primary Focus Practical Application
Software Development AI-assisted coding and autonomous agents. Tools handle boilerplate code and test generation. This shifts engineering cycles away from syntax and toward system architecture.
Marketing Campaigns Predictive analytics and forecasting. Systems are built for predicting future trends. Marketers deploy these models to adjust budgets preemptively rather than reacting to yesterday’s reports.
Prompt Engineering Advanced linguistics and logic structures. Mastery involves navigating how mental models assist in problem-solving. This discipline structures language strictly enough to force an LLM into predictable reasoning patterns.
Data Architecture AI-ready data pipelines and vector databases. Success requires establishing a comprehensive inventory of everything in your ecosystem. AI models hallucinate when fed fragmented data; clean pipelines act as the essential guardrail against garbage outputs.
Content Creation Generative text, image, and video workflows. AI enables executing multidisciplinary mental models for solving complex problems. Scaling content now relies on curating outputs against a strict brand voice, not writing from a blank page.
Business Intelligence Pattern detection and anomaly resolution. BI teams use AI to deploy real-time anomaly detection. This replaces static dashboards with active alerting systems that diagnose the drop in metrics before leadership even asks.
If you found this interesting, please share.

© Scott S. Nelson

Developers are now Agent Managers: Enter the New Matrix

TL;DR: The use of AI in development has shifted from a coding assistant to a team of agents doing the heavy lifting. This requires developers to skill up in management and forces a fundamental shift in how software roles collaborate.

The Managerial Migration

For those not watching closely (which is most people, and perhaps only a few reading this), the world of software delivery is in the midst of a tectonic shift. The use of AI is evolving from a simple coding assistant to a team of agents, or experts, performing the bulk of the work. This moves the developer role from a traditional software engineer to an agent manager.
The change in role definition and skills is another aspect of the paradigm shift the Age of AI is bringing. If this sounds new to you, it is because you may have missed the transition that the proliferation of the “WWW” subdomain brought to IT in the 90s. We are all going to come out better, but it is going to be a long haul as we re-learn lessons from last time and write new ones for this era.

The Expertise Gap in Management

One common misconception is that this automation means developers can be replaced by Product, Project, or Program managers. This is mostly the “Wall Street Rumor Mill,” which is only just now being revised from “replacing people with AI” to “shifting investment from employees to AI vendors”. At least that is more honest.
The “average” manager often lacks the technical depth to write a precise specification or review the complex output of an agentic workflow. Managing a digital workforce requires the same technical understanding and focus as writing source code. If you cannot perform a rigorous technical review of what AI agents produce, you should not put it into production (unless you suffer from a terminal case of technical hubris).

The Developer Drift

While many managers lack the depth to take over, developers are not guaranteed success in this new role without learning to view technical problems from different angles. Many developers tend to drift from the business context without a reminder, which is why lifecycle ceremonies exist to gather feedback from users and owners. For some, this is a forest-versus-trees effect, while for others, it is the temptation of a “cool” approach over a practical solution.
The speed of AI can take a minor gap in understanding and expand it into a costly chasm. When an agent can produce a week’s worth of logic in seconds, the cost of moving in the wrong direction scales exponentially. The team must find a way to collaborate where agents are a factor beyond just a tool choice.

Grit over Gift

This proficiency is not a magic gift: it is a byproduct of learning, practice, and pushing boundaries. There is a persistent myth that “prompt engineering” is an inherent talent or a shortcut for the lazy. It is actually the opposite. Real proficiency comes from hundreds of hours spent in invisible iteration. You have to break the agents to understand how to fix the workflow. These skills are then applied to context engineering, where the developer becomes the manager and the back-and-forth transitions to a human-in-the-loop system.
Deep experience can sometimes trigger intellectual rigor mortis, where you stop looking for a better way because you already know the “right” way. To succeed now, you need the grit to unlearn habits that are no longer efficient. High ROI in the age of AI belongs to the person who pushes boundaries through practice, not the one waiting for the “perfect” model to arrive.

The Practical Pivot

As we navigate this “.ai moment,” leadership, managers, and developers need a new way to interact. It is no longer about a ticket hand-off: it is about real-time orchestration.
  • Developers: Start treating your AI tools as interns, not calculators. An intern needs guidance, a clear spec, and a rigorous peer review. If they produce garbage, it is a reflection of your management. Mentor your agents by providing better context and documentation.
  • Managers: Help leadership understand that the “silver bullet” still requires expert aim. AI is a force multiplier, but it requires a human who knows where to point the barrel. Use these tools to bridge the communication gap, not to eliminate the experts.
  • Everyone: Support each other in cross-training. Incorporate big-picture product thinking with low-level solutioning. Document the new workflows immediately, as your team now includes transient sessions that lack long-term memory.
Incorporating this new layer requires new connections, shifts in responsibility, and overlaps that act as double-checks from different perspectives rather than simple redundancies.
If you found this interesting, please share.

© Scott S. Nelson

The Beginners Mind Stands on a Foundation

TL;DR: Experience is a liability when it kills curiosity. AI proficiency is a byproduct of hundreds of hours spent in invisible iteration.

The Expertise Trap

Deep experience often triggers intellectual rigor mortis. You have seen the “right” way to do things for a decade, so you stop looking for the better way. A beginner mindset is not about being a blank slate (which is just another word for useless). It requires enough of a foundation to know when you are headed in the right direction, or perhaps a parallel path with new perspectives, before getting back on track.

If you have twenty years of experience but no curiosity, you are just a legacy system waiting for a decommission date. The ROI on a “beginner” attitude is higher because it allows for rapid pivoting. You need the basics to provide a compass (to avoid spending days debugging a syntax error), but you need the mindset to explore the paths that lead to breakthroughs.

The AI Sweat Equity

There is a myth that prompting is a low-skill activity. It is not. Most people really good at prompting have iterated and learned. The developers currently running multiple agents and building software 4 to 10 times faster than they did last year have been in months of practice to get there.

This is iteration 0 work. It is messy and mostly undocumented (because the tech moves faster than the README files). What makes it daunting for those first starting is that the people who are now good at it did it without formal training. There is a tendency to forget how much effort went into the initial struggle.

Building the Mental Infrastructure

Learning new technical skills requires toggling between different cognitive states. Barbara Oakley (author of Learn Like a Pro: Science-Based Tools to Become Better at Anything and Learning How to Learn, among other great books) describes this as the tension between focused and diffuse modes. Focused mode is for the granular syntax: the structure of a prompt or a script. Diffuse mode is where the beginner’s mindset lives. It is the relaxed, curious state that allows your brain to make the non-linear connections required to solve a problem that does not have a documentation entry yet.

She emphasizes chunking: breaking complex concepts into small, functional units until they become second nature. This prevents cognitive overload when the system throws an error you have never seen before. Curiosity is a tool that keeps you in the diffuse mode long enough to see the “big picture” before diving back into the details. I took her class on Coursera at the start of my AI journey, and I recommend everyone do the same, even if your interests are in other areas. It applies to learning anything, and you will thank yourself for doing so.

They say “oh, it’s easy, you just do this”, which looks like magic to the beginner. It is not entirely different from visiting a new area and asking a local for directions. Every time they start with “Oh, that’s easy”, there is a good chance you are going to get lost following their directions.

Locals navigate by landmarks that either do not stand out to an outsider or have disappeared from all but the local’s memories. They tell you to turn where the oak tree used to be or past the shop that changed names five years ago. They have internalized the route so deeply they forget the friction of finding it the first time.

And sometimes, that makes the trip more fun. Stop looking for the “perfect” prompt or the “right” workflow. Spend more time being “lost” in the tool. The goal is not to avoid the detour: the goal is to have a strong enough foundation to know how to get back to the main road once the detour stops being productive.


A Simple Roadmap

If you haven’t begun your journey with Generative AI, or feel a bit lost, here’s a simple roadmap to help you along:

  1. Pick one model and stay there: Stop comparing benchmarks and just use one tool (Claude, GPT, or an LLM via API) for a week straight to understand its specific “personality.”
  2. Iterate on a single prompt 50 times: Don’t just accept the first output. Change one variable at a time until you understand exactly what triggers a hallucination vs. a logic block.
  3. Read the system prompt documentation: Most users treat AI like a search engine. Read the actual technical guides on “system roles” and “temperature” to understand the controls.
  4. Practice manual orchestration: Before you try to automate a multi-agent system, act as the agent yourself. Copy the output of one model into another and manually fix the “gotchas” in between.
  5. Fail on purpose: Try to make the model break. If you don’t know the edges of the tool, you won’t know when you are standing on a cliff.
If you found this interesting, please share.

© Scott S. Nelson