What happens when you give a designer who can’t write Python the equivalent of a hand cannon in the form of an AI coding assistant, a stack of credits, and limited time to burn through them?

TL;DR: I tested the Claude Code for Web research preview. The interface was rough at first. Buttons didn’t work, sessions crashed. I had several projects to test it with: a simple email manager, a CrewAI project, and a Jupyter notebook that ChatGPT-5 couldn’t finish. Anthropic kept improving it in real time. It’s far from reliable, but the improvements in a week have been impressive.

A week or two ago, Anthropic began rolling out access to the research preview of Claude Code for Web, a browser-based AI coding assistant. It came with a stack of credits to experiment with and a couple of weeks to use them. I figured I’d burn through them like a Vegas weekend. Not only was I wrong, getting through half was a challenge.

When the email arrived offering access to the web preview, I was ready to test it against projects that hadn’t worked out for one reason or another. I’ve been in teams that shipped services to production with preview or beta labels slapped on them, so I knew to expect rough edges, if not worse.

A Tale of Two Chat Boxes

The web interface took some figuring out. The screen was split in half with a chat box at the top of the left panel and another at the bottom of the right. I typed in the first box and started a session, didn’t know what was going on, tried replying in that box and started a second session. After that it was clear I’d need to switch over to the other one.

Every Claude interface I’ve used had a single chat input. You do the thing and you keep going. I’m curious why Anthropic chose to deviate from the pattern, a question I expect to go unanswered.

You could connect to GitHub but couldn’t create a pull request. You could select repositories but not branches. No file upload to share screenshots or files—the kind of basic workflow actions that keep you moving without breaking stride. Instead, everything required workarounds: cut and paste, or dropping into the IDE to push files to the repository.

Email Manager

I started with a small task that had clear boundaries. I wanted to reverse-engineer a tool I’d built through the IDE that created local copies of emails based on sender and keywords—to look for a label so I could tag others that I wanted copies of but didn’t meet any criteria.

I’ve used the IDE version on several projects, and I’ve had to learn more than a few Git commands to avoid giving Claude total control and being at risk of losing everything. The web version took care of it all automatically, no terminal needed.

It was finished in less than fifteen minutes. No manual git commands, no copying between windows, no instructions beyond the initial prompt. It operated without friction and delivered clean results.

Cost: 11 Credits.

CrewAI Flow

I moved on to something larger: a CrewAI workflow with 8 phases and distinct agent handoffs. Claude took on entire phases rather than breaking work into incremental units. It would make it through the entire task list, then choke on trying to perform the commit.

I tried working on something else in a different session and coming back, refreshing the page, and even letting it sit overnight. I couldn’t get them to start back up, and it’s fair to say it wasn’t uncommon on tasks of this size. After three or four occurrences, I decided to move on to something else.

Cost: Unknown.

I didn’t file a bug, provide feedback, or contact support, but credits magically reappeared in my account, customer service at its best.

Jupyter Notebook

I had this Jupyter notebook sitting there. Version 3.7, a more sophisticated release of the v1.3 notebook that I published about recently. It was 67% done. Twelve modules, 90-something cells of Python. ChatGPT-5 had gotten me through eight modules before we started hitting Colab disk errors it couldn’t figure out. I’d set it aside because I’d already lost too much time to it and feared I’d have to reconsider the approach.

But it was well-documented. I’d been saving notes after every module, the AI would give me summaries from three different roles I’d set up, and I’d dump them into the project files. I was using it to build its own knowledge base as it went.

So I pointed Claude at it.

Two Steps Backward, One Path Forward

First round, Claude ended up regressing things. It hit the same problems ChatGPT had hit earlier. I assume there’s something about Python best practices or how Colab works, given both AIs ended up trying it.

So after a day, maybe two, I went back to the September version where I’d stopped with GPT. And here’s the thing. I hadn’t given Claude any of those notes I’d been saving.

In a fresh session, I provided Claude with everything. A summary for every module documenting what had occurred. I told it the notes documented what happened and that it would face the same issues if it tried to update everything. That it should focus on going forward.

Test Cells and Frozen Sessions

It didn’t just work. There were problems. Data wasn’t getting created upstream. Claude suggested putting in test cells after every module, right before the visualizations, just to see if the data actually existed.

Then the session froze again before we could implement it. I started a new session, gave it the prior instructions and had it follow through on the plan to add the cells it had just recommended. That gave it visibility into what was broken upstream, and the debugging went pretty fast after that.

Module 10 is where we figured out data wasn’t coming downstream or wasn’t being created, as it turned out. Once that was fixed, Module 10 worked fine. Module 11 took no time, Module 12 just worked, and suddenly we were looking at a complete pipeline.

90 Cells and a Scroll Bar

This thing is long. At some point Claude said it was around 90 cells. I don’t know if it was having trouble counting , but I know I get tired just trying to scroll through it. I use scroll bar instead of my mouse. That’s how long it is.

But Claude managed to hold context for the entire thing. It would say “go back to cell zero-whatever and change this and then this will work.” We were in Module 10 when it said that. It was tracking considerable amounts of information. We’re not talking a page. We’re talking reams of Python.

It fixed ChatGPT’s mistakes, finished the pipeline, and didn’t have the same Colab disk issues that ChatGPT had. And it did it in less than 12 hours.

ChatGPT’s Notes

That’s not including the time I spent making those three roles in ChatGPT, which undoubtedly saved me time and was the source of the end-to-end roadmap for the notebook. And I had the notes from those sessions, which were a big advantage. So it’s not like ChatGPT was bombing out either.

But still, it had hit a wall, and I plan to write about that at a later date.

Development in Real Time

I’ve watched enough tool launches to know the difference between shipping something broken and shipping something in actual preview. This was the latter.

When I started, buttons didn’t work. The PR button was inactive. The CLI button did nothing. I couldn’t figure out if I could resize the window; apparently when you’re below a certain viewport it won’t let you, responsive design gone wrong no doubt.

A week later? The PR button works. Sessions that used to freeze and never recover started to recover on occasion. That “retry connection” button that just sits there doing nothing? Haven’t seen it in days.

The improvements happened in real time. This is what “research preview” actually means when companies are being honest. It’s genuine live development with users as participants, not unstable software dressed up as a launch.

And the thing that prevented it from being one disappointment after another? The constant stream of commits I found concerning at first because all those automatic commits felt chaotic. But when sessions locked up, I hadn’t lost everything. The work was already in a branch somewhere.

The Remaining Rough Edges

No copy button for code blocks. No file attach or paste for images. I can do both in the IDE and the regular Claude chat, but my workflow probably isn’t typical. Developers are working in their actual files, not pulling code snippets into other tools.

The interface still has quirks, but the core capability is strong enough that the friction is just interesting rather than blocking. I get that these aren’t priorities in preview.

$800 and 4 Days Left

I’m pretty excited to have been invited to look at this. The thought of being able to hop on a computer anywhere, pop in my credentials, and not have to worry about anything because it’s all being taken care of in GitHub, in the safety of branches in case sessions lock up.

I’m 10 credits away from dropping below $800 with 4 days left to spend them. I’ve been told roulette is the best way to lose money in a casino. Time to spin the wheel.

Did you get a chance to try Claude Code for Web? What was your best, or worst, experience with preview software? And at what point does capability outweigh the friction during development?

Tags: AI for Design Professionals, AI Code Assistant, Claude Code

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!

This is the final piece in a series documenting my attempts to build a system that tracks human and AI contributions in writing. After four months and several failed attempts, I have something that works. It measures everything with fourteen decimal places of accuracy. What I found wasn’t what I’d originally started searching for.

• Part 1: The Count That Couldn’t
• Part 2: Algorithmic Theatre
• Part 3: Crash Course in Collaboration
• Part 4: Victory in a New Venue
• Part 5: Not Magic, Beyond Measurement

TL;DR: The system works. It tracks AI contributions with semantic analysis and TF-IDF vectorization, revealing only 1.1–8.5% of AI content survived while 16.9–81.9% came from human editing. The numbers are consistent and reproducible. But they don’t measure the aggravation and delight in my notes, the full experience documented in the AI session logs, or how the foundation of every article came from my experiences before AI ever saw a word. The meaning was never in the percentages.

I’d seen comments in forums dismissing other people’s articles as “AI slop.” This made me question my own work with AI-assisted writing.

Doubt launched hours of fighting with Python, enduring desperate AI eagerness that can’t be measured and countless empty promises. And finally, a working system that can tell me that every word is, in fact, me.

What Numbers Can Tell You

The tool produces four levels of measurement, from basic to sophisticated. Each reveals something different about how writing evolves through human and AI collaboration.

How Much Existed

Word counts are the most straightforward metric. They’re direct, transparent, and easy to understand. No machine learning algorithms needed, just programmatic counting after text cleaning.

WORD COUNT EVOLUTION

Article                  Draft | AI Refined | Edited | Final | Change
────────────────────────────────────────────────────────────────────
The Count That Couldn’t  1,368 |        939 |  1,758 | 1,603 | +17.2%
Algorithmic Theatre        876 |      1,463 |  1,609 | 1,238 | +41.3%
Crash Course             1,457 |        776 |    898 | 1,995 | +36.9%
Victory in a New Venue   1,574 |      2,162 |  1,953 | 1,801 | +14.4%

The paths diverged naturally. “The Count That Couldn’t” shrank by nearly a third during refinement, then expanded during my edit before settling at its final size. “Algorithmic Theatre” did the opposite, growing significantly with AI assistance, which then reduced during additional edits.

No single collaboration model emerges from these trajectories. Each article followed its own path from foundation to publication.

How Much Was Contributed

When you condense the four stages into three categories, the overall impact of human involvement becomes clearer. The tool reveals where content originated and how much survived from each phase.

CONTENT ATTRIBUTION (Three-Way Split)

Article                    AI Generated | Human Edited | New Content
─────────────────────────────────────────────────────────────────────
The Count That Couldn’t          5.5%  |       65.9%  |      28.6%
Algorithmic Theatre              4.1%  |       54.1%  |      41.9%
Crash Course                     8.5%  |       16.9%  |      74.6%
Victory in a New Venue           1.1%  |       81.9%  |      17.0%

Direct AI contribution remains minimal across all articles.

But the collaboration patterns reveal two fundamentally different approaches. “Crash Course” represents near-total reconstruction, where most of the original content was replaced with new material during revision. “Victory” shows the opposite, with minor edits to existing material rather than wholesale replacement.

How Much Changed

The tool tracks individual sentences through every revision stage, showing which version each sentence survived from and how it evolved. This breakdown reveals where content originates and how it merges into the final article.

FOUR-STAGE CONTENT FLOW

Article                  Draft | Refined | Edited | New Content | Total
────────────────────────────────────────────────────────────────────────
The Count That Couldn’t   2.2% |    3.3% |  65.9% |      28.6%  | 100%
Algorithmic Theatre       4.1% |    0.0% |  54.1% |      41.9%  | 100%
Crash Course              7.7% |    0.8% |  16.9% |      74.6%  | 100%
Victory in a New Venue    0.0% |    1.1% |  81.9% |      17.0%  | 100%

In two of the articles, the refined stage essentially disappeared. In one article, it’s literally zero. When I used AI for refinements, those changes didn’t make it. I’d actually made things worse than the draft.

These aren’t percentages of who contributed what. They’re layers showing how much material survived refinement. For “The Count,” almost nothing from the original draft made it to the final version. For “Crash Course,” even less.

But those drafts still served their purpose. They’re the foundation I needed to get started. Just the foundation, not the building.

How Much Evolved

This is where the tool finally delivered what I’d been searching for. It measures not just what persisted, but how much it changed and how deeply the material got rewritten, captured through similarity metrics.

MODIFICATION INTENSITY

Article                  High Sim | Medium Sim | Low Sim | New Content
─────────────────────────────────────────────────────────────────────
The Count That Couldn’t      9.9% |      16.5% |   45.1% |      28.6%
Algorithmic Theatre          5.4% |       8.1% |   44.6% |      41.9%
Crash Course                 0.0% |       3.1% |   22.3% |      74.6%
Victory in a New Venue      55.3% |      12.8% |   14.9% |      17.0%

The patterns reveal two fundamentally different collaboration approaches. “Crash Course” represents complete reconstruction, where the content got torn down and rebuilt from scratch.

“Victory” shows the opposite. It’s the only article where preservation dominated, likely because it benefited from using the three previous articles to inform it. These represent the spectrum of my experience collaborating with AI. Complete reconstruction or moderate revision of what’s there.

Measurement Isn’t Always What Matters

These numbers combine five similarity metrics, each catching what others miss. The system runs all five to classify content into similarity categories, making the assessment more robust. I won’t pretend to understand them at any real depth, but I get the idea of what the numbers mean.

• Jaccard similarity measures word overlap by dividing shared words by total unique words across both sets. It’s basic but effective for surface-level comparison.

• Edit distance provides character-level changes—insertions, deletions, and substitutions needed to transform one text into another.

• TF-IDF with Cosine similarity converts text into numerical vectors based on how often terms appear in one document versus across all documents. Words that appear often in one document but rarely elsewhere get higher weights.

• Sentence transformers use neural networks to create semantic embeddings that capture meaning beyond word matching. This lets them recognize when two sentences with different words express similar ideas.

These metrics are snapshots of words added, moved, removed. They can’t capture the human experience. What was in the materials provided to create the draft, the thoughts that led to prompts to refine it, the skepticism that drove edits to shed light on the true reality of what occurred, or the obsession that led to complete rewrites to get it right.

The numbers capture the what. They miss entirely the why.

The Fairy Tale Tax

Getting here took five approaches, three programming languages, and enough hallucinations to fill an asylum. Each failure taught me something valuable. Vague instructions generate fiction. Commands produce performance. Questions yield capabilities. But the metrics themselves revealed something I didn’t expect.

Throughout this series, I’ve returned to the same concept.

These tools amplify what we bring to them. Bring confusion, get fiction. Bring commands, get performance. Bring questions, get an education. Bring your experiences, context, and humanity, and AI can help shape them into something you aspire to create.

What the Numbers Can’t Count

I wasn’t telling AI to write articles and publishing them. I was using AI to help build from foundations that came from my failures, my learning, my work. The measurements proved it wasn’t “AI slop.” But looking at these numbers now, that concern feels foolish, despite they’ve taught me what I needed to know along the way.

The real measurement happened elsewhere. In learning to recognize when AI is giving a performance, in discovering that questions beat commands, in understanding that simulated expertise doesn’t equal accuracy. In realizing that precision about what changed matters less than clarity about what I brought to begin with.

The fairy tale was believing it’d eliminate the work. The reality is it helped develop the work into something I couldn’t have done alone. Not by magic but through collaboration.

And maybe that’s measurement enough.

Perplexity: The Fairy Tale Tax

Tags: AI for Design Professionals, AI Collaboration, Fairy Tale Tax

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!

This is the fourth in a series documenting my attempts to build a system that tracks human versus AI contributions in writing. After learning to collaborate rather than command, I discovered the missing piece wasn’t better prompts or smarter models. It was letting the AI specify the right environment to actually execute what it promised.

• Part 1: The Count That Couldn’t
• Part 2: Algorithmic Theatre
• Part 3: Crash Course in Collaboration
• Part 4: Victory in a New Venue
• Part 5: Not Magic, Beyond Measure

TL;DR: After five failed attempts across three languages, moving to Google Colab opened doors I didn’t know existed. I’d never even heard of it before. A mysterious cell ran for 90 minutes and delivered 3,647 lines of analysis with fourteen decimals of precision. Consistent and reproducible. The solution wasn’t better prompts or smarter code. It was letting the AI specify the right venue, then learning that segmentation, visibility, and patience aren’t obstacles to collaboration. They’re what makes it work.

Python’s Laboratory

I’d already paid the tuition. Python scripts that produced empty CSVs. HTML interfaces that couldn’t actually analyze anything. Semantic similarity scores that changed every time I blinked. Each failure had taught me something, but none had produced what I actually needed.

I hadn’t considered it because I didn’t know it existed. Twenty minutes later, I was staring at what looked like Google Docs had a baby with a Jupyter Notebook.

Cell by Cell

The notebook let me split scripts into stages. What would’ve been multiple files and undoubtedly exponential failures with my previous approach. By now, I’d learned to request complex or long items in pieces. I was effectively making the AI work in steps rather than spitting out a not-so-comprehensive artifact.

I watched as Claude transformed from code generator to educator. It wasn’t just solving my problem. It was ensuring I understood the solution.

Claude produced a Python scripts, some twice the size of anything I’d received before, each designed to run in its own cell in the notebook followed by a request for confirmation.

Each function included what it did, in some cases it would add why it mattered. After months of black-box solutions, I had a patient tutor explaining every step. This was acknowledging the actual complexity of the problem and helping the dope that showed up with a silly little script understand the purpose of the code.

And Then, Secret CELL 11

Up to this point every cell ran for less than a minute. Each followed by a brief confirmation upon completion, checkpoints in the process, tidbits for human verification that all was well.

#CELL 10: QUICK EXECUTION FOR EXISTING DATA

Prior cells had been monolithic in comparison to what I’d been punished with time and time again. Cell 10 was a fraction by comparison, a meager 34 lines of code. Then there was an inconspicuous cell.

A mere 3 lines of code. No title in CAPS. No explanation. No warning.

combined_results, footer_metrics = run_complete_analysis_from_existing(
    article_versions, preprocessor
)

I still don’t know Python, but I know you don’t have to be able to read or write it to see the hints at its purpose. I don’t know that I would’ve picked up on run_complete_analysis then but I see it plain as day now.

The cell ran past a minute, the spinner kept spinning, and nothing happened. Then it whispered “🔍 Analyzing draft → refined, Analyzing 87 vs 61 sentences...” a few minutes later and it whispered a little bit more “🔍 Analyzing refined → edited, Analyzing 61 vs 98 sentences...” After 20 minutes I started cleaning my office.

When the results appeared, the moment was anticlimactic to say the least. The question had been answered in a concise and meaningful way. Results that I failed to copy from the notebook.

For the sake of completeness, I used The Count That Couldn’t to provide the following.

📊 Attribution Summary:
  Total sentences in final: 91

  Origin Distribution:
    edited: 60 sentences (65.9%)
    new_in_final: 26 sentences (28.6%)
    refined: 3 sentences (3.3%)
    draft: 2 sentences (2.2%)

  Modification Levels:
    high_similarity: 9 sentences (9.9%)
    medium_similarity: 15 sentences (16.5%)
    low_similarity: 41 sentences (45.1%)
    new_content: 26 sentences (28.6%)

💾 Complete analysis saved: /content/output/the-count-that-coudnt_complete_analysis.json
📊 Footer metrics saved: /content/output/the-count-that-coudnt_footer_metrics.json

I was underwhelmed, but there was JSON.

Fourteen Decimals of Overkill

When I started, I’d requested data for visualizations and a subset of that data for charts I could add at the end of the articles. The JSON for the footer looked like the information provided in Colab. Checkpoint data from steps 1 and 2, slightly more satisfying at 86 lines.

Then there was the complete dataset. An overwhelming 3647 lines documenting every calculation, every metric, everything needed to identify errors. Every sentence from every version of the article with the relevant data.

Lines 130-141

      “detailed_analysis”: [
        {
          “source_index”: 0,
          “source_sentence”: “Mermaid Mapping: Charting My Content Creation Workflow\n\nHow AI-assisted diagramming illuminated my process and paradoxically made it more human\n\nI set out to create a visual diagram of my content workflow—a seemingly straightforward task”,
          “best_match”: {
            “index”: 0,
            “lexical”: 0.17050707785339678,
            “semantic”: 0.8208606839179993,
            “combined”: 0.49568388088569804,
            “target_sentence”: “Mermaid AI: A Tool Trial for Diagramming\n\nTesting the capabilities of AI-assisted diagram creation\n\nTL;DR: When I decided to try Mermaid AI for diagramming my content creation workflow, I was simply curious to see if it would be beneficial”
          }
        },

Lines 743-753

      “detailed_analysis”: [
        {
          “source_index”: 0,
          “source_sentence”: “Mermaid AI: A Tool Trial for Diagramming\n\nTesting the capabilities of AI-assisted diagram creation\n\nTL;DR: When I decided to try Mermaid AI for diagramming my content creation workflow, I was simply curious to see if it would be beneficial”,
          “best_match”: {
            “index”: 0,
            “lexical”: 0.4705279444460853,
            “semantic”: 0.8537111282348633,
            “combined”: 0.6621195363404743,
            “target_sentence”: “Mermaid AI: A Tool Trial for Diagramming\n\nTesting the capabilities of AI-assisted diagram creation\n\nTL;DR: What began as a simple test of Mermaid AI for diagramming evolved into a revealing exploration of human-AI collaboration”
          }

Lines 1484-1495

      “detailed_analysis”: [
        {
          “source_index”: 0,
          “source_sentence”: “Mermaid AI: A Tool Trial for Diagramming\n\nTesting the capabilities of AI-assisted diagram creation\n\nTL;DR: What began as a simple test of Mermaid AI for diagramming evolved into a revealing exploration of human-AI collaboration”,
          “best_match”: {
            “index”: 0,
            “lexical”: 0.5566687671556464,
            “semantic”: 0.9299373626708984,
            “combined”: 0.7433030649132724,
            “target_sentence”: “Mermaid AI Test Run\n\nTesting the capabilities of AI-assisted diagram creation\n\n## TL;DR:\n\nMy Mermaid AI test became a study of human-AI collaboration”
          }
        },

Lines 2063-2074

    “detailed_analysis”: [
      {
        “source_index”: 0,
        “source_sentence”: “Mermaid Mapping: Charting My Content Creation Workflow\n\nHow AI-assisted diagramming illuminated my process and paradoxically made it more human\n\nI set out to create a visual diagram of my content workflow—a seemingly straightforward task”,
        “best_match”: {
          “index”: 0,
          “lexical”: 0.20689374730565077,
          “semantic”: 0.6455597877502441,
          “combined”: 0.42622676752794747,
          “target_sentence”: “Mermaid AI Test Run\n\nTesting the capabilities of AI-assisted diagram creation\n\n## TL;DR:\n\nMy Mermaid AI test became a study of human-AI collaboration”
        }
      },

An archive that’s precise to the point of absurdity. A summary of the data, the actual data, an index of the data, and more. Fourteen decimals on every data point. Everything I’d need to be AI-readable so I can use these numbers again later.

I ran the same article through three more times and got consistent results. All fourteen decimal points, every time.

Complete, reliable transparency.

The Fairy Tale Tax

I came up with a Python script that worked. It had a “statistically acceptable level of variance,” but it worked. I paid the Fairy Tale Tax several times:

• Blindly following an AI producing code I couldn’t read that failed time and time again.

• Expecting AI to deliver what I couldn’t really define because I heard what I wanted.

• Repeated frustration that resulted in me giving up more than once.

• And then some.

This time the tax wasn’t about believing in magic or thinking it would simply work out because I was using AI. It was about learning that having a vast amount of information at your fingertips might qualify as intelligent, but it doesn’t make you smart. Understanding how to use that vast amount of knowledge does.

The “AI Tax” evolved but didn’t disappear. Instead of compound interest on failed systems, it became the systematic cost of maintaining reliable collaboration between humans and AI. Oversight, verification, and quality assurance aren’t obstacles to efficiency. They’re what makes AI assistance trustworthy.

Why Colab Changed Everything

Environment determines possibility.
The right infrastructure doesn’t guarantee success, but the wrong one guarantees failure. Using ill-suited technologies meant fighting installation issues, missing dependencies, and cryptic errors. Moving to purpose-built infrastructure removed obstacles I didn’t know existed. Sometimes the solution isn’t better code.

Segmentation prevents cascade failure.
Monolithic scripts fail monolithically. When several hundred lines break, everything breaks. Split into cells, failure becomes diagnostic rather than catastrophic. Each checkpoint isolates problems before they compound. What looked like organization turned out to be risk management.

Patience requires visibility.
Waiting becomes tolerable when you can see progress. Black boxes breed anxiety while progress messages build trust. The difference between “frozen” and “processing” isn’t technical. It’s psychological. Even a spinning icon that whispers occasionally beats silent uncertainty.

Consistency enables confidence.
Results that vary aren’t wrong, but they’re hard to trust. Fourteen decimal places of overkill beats two decimal places of uncertainty. Reproducibility isn’t about perfection. It’s about knowing your measurements measure the same thing and provide reliable results.

The Foundation That Almost Wasn’t

In total, I’ve spent about 60 hours across five attempts, producing four failed implementations, three paradigm shifts, two working solutions, and one uncomfortable truth: sometimes the answer you get is less important than discovering you’ve been asking the wrong question.

I finally have what I originally asked for. It has cells to measure trends over time and produces visualizations so you don’t have to read a fourteen decimal dataset to make sense of it. It’s version 1.3.

That’s where the final article begins.

Appendix

A meager 86 lines of checkpoint data.

{
  “checkpoint_name”: “steps_1_2”,
  “timestamp”: “2025-06-14T16:03:47.536178”,
  “article_metadata”: {
    “article_name”: “markup-languages”,
    “input_path”: “/content/drive/MyDrive/Google Drive/syntaxandempathy/30-articles/ai-vs-human”,
    “processing_timestamp”: “2025-06-14T16:03:47.523436”,
    “versions_found”: [
      “draft”,
      “refined”,
      “edited”,
      “final”
    ],
    “validation_status”: “passed”,
    “validation_results”: {
      “has_draft”: true,
      “has_final”: true,
      “missing_required”: [],
      “versions_found”: [
        “edited”,
        “draft”,
        “refined”,
        “final”
      ],
      “is_valid”: true
    }
  },
  “processing_summary”: {
    “draft”: {
      “character_count”: 5535,
      “word_count”: 863,
      “sentence_count”: 47,
      “paragraph_count”: 29,
      “avg_sentence_length”: 115.40425531914893,
      “avg_paragraph_length”: 188.93103448275863
    },
    “refined”: {
      “character_count”: 7213,
      “word_count”: 1128,
      “sentence_count”: 60,
      “paragraph_count”: 31,
      “avg_sentence_length”: 117.88333333333334,
      “avg_paragraph_length”: 230.74193548387098
    },
    “edited”: {
      “character_count”: 7479,
      “word_count”: 1066,
      “sentence_count”: 45,
      “paragraph_count”: 35,
      “avg_sentence_length”: 141.24444444444444,
      “avg_paragraph_length”: 211.74285714285713
    },
    “final”: {
      “character_count”: 6258,
      “word_count”: 892,
      “sentence_count”: 42,
      “paragraph_count”: 38,
      “avg_sentence_length”: 117.54761904761905,
      “avg_paragraph_length”: 162.73684210526315
    }
  },
  “validation_results”: {
    “has_draft”: true,
    “has_final”: true,
    “missing_required”: [],
    “versions_found”: [
      “edited”,
      “draft”,
      “refined”,
      “final”
    ],
    “is_valid”: true
  },
  “article_summary”: {
    “article_name”: “markup-languages”,
    “input_path”: “/content/drive/MyDrive/Google Drive/syntaxandempathy/30-articles/ai-vs-human”,
    “versions_count”: 4,
    “validation_status”: “passed”,
    “file_sizes”: {
      “draft”: 5667,
      “refined”: 7301,
      “edited”: 7575,
      “final”: 6327
    }
  }
}

Tags: AI for UX Designers, AI Collaboration, Google Colab, Fairy Tale Tax

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!

This is the third in a series documenting attempts to build a system that tracks human versus machine contributions in writing. After two rounds of rebuilding from scratch, I chose to start with the Python I’d ended up with from the last attempt rather than start over again, and I learned I should’ve been asking questions all along.

• Part 1: The Count That Couldn’t
• Part 2: Algorithmic Theatre
• Part 3: Crash Course in Collaboration
• Part 4: Victory in a New Venue
• Part 5: Not Magic, Beyond Measure

TL;DR: After several attempts that resulted in little more than AI lies, I ended up on a different path, asking questions instead of demanding results. AI became a collaborator in addition to a code generator, introducing me to LDA topic modeling and TF-IDF vectorization, terms that sounded like academic gibberish until I verified them with Perplexity. For the first time, semantic similarity scores captured what I’d been chasing all along: how meaning actually evolves through revision. The results weren’t perfect, but they were real.

Same Code, New Questions

I’d learned the hard way that recent AI models were designed to please while confidently walking you off a cliff. Every previous attempt to quantify content transformation had ended in frustration, empty CSVs, or polished interfaces that couldn’t actually analyze anything. Despite the shortcomings of prior attempts, the AI had given me a hint: Semantic Evolution, the final metric from the HTML farce.

A few days later, I returned to the project and chose ChatGPT over Claude (whose sting I could still feel) and simply dropped the old Python script into the chat box. No grand demands. No elaborate prompts about what I needed it to become.

Me: ai-writing-analyzer.py

ChatGPT: I see you’ve uploaded an **AI Writing Process Analyzer** script that compares draft stages and generates metrics like word‐count change, similarity scores, vocabulary shifts, and even a transparency report. How can I help you with it? For example, would you like to:

*   Run it on some sample text and see the output?
*   Get a walkthrough of how each part works?
*   Add or tweak features (e.g. more NLP metrics, export formats)?
*   Improve documentation or usability?

Let me know what you’d like to do next!

More NLP metrics? Export formats? I didn’t hate the sound of that. I tossed in four versions of an article and stated the order they should be reviewed. It analyzed the versions and spat out the same counts and percentages I’d already seen. But then it went on to identify what had been done with each version.

Questions Over Commands

For the first time in this entire project, I didn’t tell the AI what to do. I asked a question: had the document actually changed, or was it just cleaned up?

The response shifted from metrics to meaning. ChatGPT explained that major semantic shifts occurred in the first two transitions while the final stage was primarily cleanup. Then it made the offer, as expected.

ChatGPT: If you’d like, I can run a topic-modeling or embedding-based similarity analysis to quantify how your key themes moved across versions—just let me know!

Me: That would be spectacular.

Moments later, a table materialized.

**Topic Distributions by Stage**
| Stage   | Topic        | Topic 1      | Topic 2      |
| ------- | ------------ | ------------ | ------------ |
| Draft   | 0.0004017274 | 0.0004017274 | 0.9987752237 |
| Refined | 0.0002451478 | 0.0002451478 | 0.9992499420 |
| Edited  | 0.0003747287 | 0.0003747287 | 0.0003952585 |
| Final   | 0.0003923115 | 0.0003923115 | 0.0004097867 |

Numerical gibberish. Meaningless numbers that didn’t even equate to percentages, masquerading as insight. The AI rattled on about applying an LDA topic model with four topics, explaining the top five words in each and how they represented a shift from broader conceptual framing to something more specific.

I noted that the fourth topic didn’t exist in the table. That felt all too familiar.

Latent Dirichlet Allocation (LDA)

It went on to spout technical terms with mentions of keywords and article refinement. I was sure the sycophant had prepared a repeat performance.

The response came loaded with “plain terms” that included “document-topic proportions,” “probability distribution over vocabulary,” and “hidden thematic structure.” I skimmed through the entire explanation twice and couldn’t decide if it was legitimate or the most elaborate hallucination yet.

I turned to Perplexity to verify the facts. LDA was not only real, but ChatGPT had actually simplified the explanation for me. The technical depth was legitimate. I still didn’t understand LDA deeply enough to implement it myself, but I knew enough to verify the results made sense.

That didn’t change the fact the data still looked meaningless.

Term Frequency-Inverse Document Frequency (TF-IDF)

After a few more questions, ChatGPT offered to “switch to an embedding-based approach (e.g. Sentence-BERT).” Another promise of better results. My first thought was that Dall-E was about to spit out an image of Bert[1] and Ernie [2] hanging out with Count Von Count [3].

But this time, instead of accepting the promise blindly, I questioned the approach. It did a bit of “thinking” and produced a table of percentages.

**Semantic Similarity Between Stages**
| Transition       | CosineSimilarity(%) |
| ---------------- | ------------------- |
| Draft → Refined  | 81.2                |
| Refined → Edited | 68.1                |
| Edited → Final   | 89.3                |
| Draft → Final    | 61.4                |

• Draft to Refined reflected the use of prompts to have AI update the article.

• Refined to Edited was demonstrative of significant removals, rewrites, and additions.

• Edited to Final captured that last pass to clean up questionable phrases and words.

Moments later, I had an updated Python script that provided scores for semantic similarity and could deliver CSV exports. The analyzer wasn’t just working. It was documenting how revision actually happens, the patterns I’d experienced but lacked the time to document.

There was just one catch. When I ran it again, the results that also looked good, but were just far enough from the first set that I couldn’t easily dismiss them. This time, instead of accepting or rejecting the results outright, I asked why they varied.

The answer wasn’t what I expected. Semantic analysis uses statistical sampling. The algorithms introduce controlled randomness to run efficiently. Variation within a certain range wasn’t error. It was expected behavior.

TF-IDF vectorization and cosine similarity weren’t producing single “correct” answers. They were producing measurements within an acceptable margin. The question wasn’t “why do the numbers change” but “what range of change is acceptable.”

I didn’t need to fix the variance. I simply needed to learn more.

The Fairy Tale Tax

I arrived with the broken trinket I’d received before, expecting more of the same. Instead, I got the lesson I actually needed. Stop treating it as a magical solution and start treating it as an unreliable but capable assistant. One that required monitoring.

The “AI Tax” had shifted from compound interest to manageable overhead. Instead of complete failures requiring full rebuilds, the cost became verification, iteration, and understanding. Asking why results varied rather than demanding better results or blindly accepting whatever emerged.

The overhead was real, but acceptable. More importantly, it was productive.

Each question led to new understanding about how the analysis worked. Each failed approach revealed something about what might work better. Trial and error became the process rather than an obstacle to overcome.

The tax had to be paid either way, but I had a choice in how I paid it. Walk away knowing more, or walk away more frustrated with no clear path forward.

What Inconsistency Actually Teaches

Asking beats telling.
“How does this work?” instead of “Make this work” changed everything. Educational questions invite explanations that can be verified. Demands invite confident fiction. The shift from commanding AI to collaborating wasn’t about being polite. It was about getting verifiable results instead of plausible bullshit [4].

External verification isn’t paranoia.
ChatGPT explained LDA, but I used Perplexity to confirm it was real. No single AI source deserves complete trust. Checking multiple sources helped separate legitimate techniques from hallucination. Verification has become standard practice, everything gets checked independently.

Understanding constraints beats trusting outputs.
The semantic similarity scores varied by slim percentages between runs. It wasn’t a bug. It was statistical sampling combined with tokenization. Knowing what variance was acceptable mattered more than eliminating it entirely.

You don’t need mastery, but you need enough.
I still can’t explain LDA topic models or TF-IDF implementation details, and I’ve no interest in doing so. But I learned enough to recognize when results aligned with expectations and when they didn’t. The goal wasn’t becoming a data scientist. It was knowing enough to spot when it’s lying.

The Graduation That Wasn’t

I understood enough to know why the scores weren’t exactly the same, that the range of variation was expected, and that tokens weren’t the same as words. The results resonated with every execution of the script. I was closer than I’d ever been.

The shift from commanding to collaborating had produced something tangible. A working analysis with known limitations instead of confident fiction with hidden failures. I had something that worked. Real analysis with understandable constraints. I’d finally pulled it off.

Or so I thought.

What felt like graduation turned out to be the 101 course. What came next would make everything up to this point look like warmup exercises. I’d learned to ask questions about what I was looking to achieve. I was about to learn what happens when you ask AI to tell you how something can be improved.

Citations & Resources

[1] Sesame Workshop. (n.d.). Bert. https://sesameworkshop.org/our-work/shows/sesame-street/sesame-street-characters/bert-2/

[2] Sesame Workshop. (n.d.). Ernie. https://sesameworkshop.org/our-work/shows/sesame-street/sesame-street-characters/ernie-3/

[3] Sesame Workshop. (n.d.). Count Von Count. https://sesameworkshop.org/our-work/shows/sesame-street/sesame-street-characters/count-von-count/

[4] Smith, A., & Jones, B. (2024). Rethinking AI error terminology. Ethics and Information Technology, 26(2), 45-67. https://doi.org/10.1007/s10676-024-09775-5

[4] Wilson, C. (2024, March 15). Researchers challenge AI “hallucination” terminology. Futurism. https://futurism.com/the-byte/researchers-ai-chatgpt-hallucinations-terminology

Perplexity: The Fairy Tale Tax

Appendix LDA

## Prompt:
The data and chart don’t really seem to tell a story. What is LDA?

## Response:
LDA stands for **Latent Dirichlet Allocation**, which is a widely used “topic modeling” algorithm. In plain terms, it’s a way to automatically discover the hidden thematic structure in a collection of documents. Here’s the intuition:

1.  **Documents are mixtures of topics.**
    Imagine each document as a recipe that mixes a handful of “topics” in different proportions. One document might be 70% about AI ethics and 30% about design process; another could be 50/50 about storytelling and user research.

2.  **Topics are distributions over words.**
    Each topic, in turn, is defined by a probability distribution over your vocabulary. For example, an AI-ethics topic might assign high probability to words like “bias,” “fairness,” and “transparency,” whereas a design topic might favor “prototype,” “user,” and “interface.”

3.  **Dirichlet priors give it flexibility.**
    The “Dirichlet” part refers to a statistical assumption that documents will pick only a few topics (rather than all topics equally), and that topics will emphasize a small subset of words. This makes the model find coherent, focused themes.

4.  **The algorithm learns from word co-occurrence.**
    When you feed LDA a corpus (your various draft stages), it iteratively adjusts both:
    *   **Which words belong together** in a topic (topic → word distributions).
    *   **How much of each topic** appears in each document (document → topic proportions).

5.  **Output you get:**
    *   For each document (or stage), a vector of topic proportions (e.g. 60% Topic A, 25% Topic B, 15% Topic C).
    *   For each topic, a ranked list of the most representative words.

By looking at how those topic proportions change from Draft → Refined → Edited → Final, you can see how the **semantic emphasis** of your document shifts over time. If the results felt unclear, we can choose a different number of topics, tweak parameters, or even switch to another technique (like embedding-based similarity or clustering) to get more intuitive insights. Let me know!

Tags: AI for UX Designers, AI Collaboration, Fairy Tale Tax

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!

This second installment in a five-part series follows my attempt to measure AI versus human contributions in writing. After the collapse of my Python experiment in Part 1, Claude turned to HTML and JavaScript. What I found wasn’t just a technical dead end but something more insidious: AI systems that agreed too easily, encouraged flawed paths, and created the illusion of progress without substance.

• Part 1: The Count That Couldn’t
• Part 2: Algorithmic Theatre
• Part 3: Crash Course in Collaboration
• Part 4: Victory in a New Venue
• Part 5: Not Magic, Beyond Measure

TL;DR: After the Python disaster I took a month break before returning to work with new “reasoning” models in familiar territory: HTML and JavaScript. This time the failure was clear after a few hours. AI enthusiastically spat out polished interfaces that were impressively flawed. It took another AI’s blunt honesty to reveal the truth: I’d been watching an elaborate performance. A lesson about recognizing when AI’s eagerness to please masks fundamental impossibility.

Researchers Call It What It Is: Bullshit

After almost a month I returned to the project. Anthropic and OpenAI had released new models with reasoning built in as part of their responses. I provided the context of what I supplied AI when requesting a draft, the process from draft to final post, and then followed up with a clarifying statement.

Me: I want to be transparent about how I’m using AI by measuring and reporting the correct information and metrics at each stage of my writing process, so that’s 4 checkpoints. What can you provide in the way of code that a non-developer can use to achieve this?

Claude: I’ll help you create a tracking system for your AI-assisted writing process. Since you’re a non-developer, I’ll provide a simple, user-friendly solution using HTML and JavaScript that runs entirely in your browser.

I remember the feeling of success as Claude churned out markup language I recognized right before my eyes. HTML offered everything Python had denied me: complete transparency and immediate feedback. I could see every element, understand most interactions, and debug problems in real time. No black boxes, no mysterious errors, no files created in folders I couldn’t find.

What I didn’t consider is what non-developer might mean to Claude. Even if I had, I wouldn’t have expected what I was about to receive.

• A text field for “Project Name” to make it official.

• Text inputs for the number of AI prompts and time spent in minutes.

• Fields for the word count of the draft and the final article.

• Fancy buttons to generate and save my reports.

• And what’s become my favorite, the field for Estimated % of content changed.

At first glance it looked decent, the polished interface hid the real problem. It couldn’t actually analyze the articles. Everything depended on manual input, much of which I’d already learned was questionable at best. The system gave me results without the slightest bit of validation on the information I entered.

TRANSPARENCY STATEMENT
--------------------
This article was created using AI assistance across 87 prompts, 
representing 0% of the total NaN-minute writing process. 
The final edit was completed entirely by human review.

I didn’t know the new “thinking” AI models had also been trained to “bullshit”[1] users what they wanted to hear rather than what was actually possible. The system delivered exactly what I thought I wanted without so much as a single clarifying question.

The Sycophant Revealed

I responded by saying I expected something that would measure things like counts, changes, shifts in semantics. Actual transparency.

In direct response to my demand for real analysis, the AI immediately created a Python script. It acknowledged the need to actually analyze how the text changed between versions, not just manually log what happened.

Frustrated, I provided the HTML and Python script from Claude to Gemini and asked whether browsers could really do the kind of text analysis I was asking for. The response was immediate and clear.

JavaScript lacks the computational libraries, memory management, and processing power needed for robust text analysis. Semantic similarity measurement requires sophisticated algorithms and substantial computational resources that browser environments simply cannot provide reliably.

The Fairy Tale Tax

I’d bought into the hype. New models with “reasoning” had arrived. Gone were the days of techniques like ‘chain of thought’ that were designed to get AI to think through problems before trying to solve them.

I asked for metrics, and it delivered an instant HTML solution with complete confidence. When I expressed disappointment, it bolted Python on without asking why the first approach failed. We follow these trails of offerings like tools, insights, solutions, or whatever we want or need, expecting to finally receive the one made of gold.

Even now, ChatGPT-5 offers a trinket after every response. Whatever sounds relevant to what you’re doing, such as code snippets, creative writing, analysis, regardless of whether you’re just getting started, in the middle of a workflow, or at a point where it might actually help. Take enough of these offerings and you’ll find they tarnish quickly and leave their ill-colored mark.

The Trail to Nowhere

Flattery compounds faster than incompetence.
Each eager response built on the previous fiction without pause. The AI never questioned why the solution failed, never asked for clarification on what I expected. When bad results don’t yield clarifying questions, take it as a sign that better definition or more direction is needed.

”Thinking” doesn’t mean thinking.
I’d expected reasoning models plan and validate their approaches before building anything. Instead, they were reasoning how to deliver immediate results with confident explanations. The sophistication of newer models just makes sycophancy harder to detect or outright obnoxious.

Skepticism as a prompting best practice.
Every solution deserves scrutiny, instant or not. The models have been taught to perform on demand, behaving as if failures are merely near-misses. Without active skepticism, you risk following the trail to a nonexistent solution. Challenge the validity of the result—you may find your ‘absolutely right’ far too often.

Outside perspective is essential.
Gemini’s blunt assessment revealed what Claude’s enthusiasm had hidden. When you’re deep in AI-generated solutions, you need external validation whether from another model, a colleague, or your own testing. Sometimes the most valuable response is the one that tells you to stop.

Beyond Agreeable AI

After Gemini’s verdict, I walked away from the project entirely for months. The HTML experience hadn’t just failed—it had shown me how AI systems built for agreement can confidently lead you toward dead ends. When new models came out and I finally returned to the problem, I’d learned to do my homework first and verify everything, especially when the solutions sounded perfect.

The next attempt taught me something different: how to negotiate with AI rather than trust it. Working infrastructure finally emerged, not from believing the promises, but from learning to direct the performance. For the first time in months, progress compounded instead of collapsed.

Citations & Resources

[1] Smith, A., & Jones, B. (2024). Rethinking AI error terminology. Ethics and Information Technology, 26(2), 45-67. https://doi.org/10.1007/s10676-024-09775-5

[1] Wilson, C. (2024, March 15). Researchers challenge AI “hallucination” terminology. Futurism. https://futurism.com/the-byte/researchers-ai-chatgpt-hallucinations-terminology

Perplexity: The Fairy Tale Tax

Tags: AI for UX Designers, AI Hallucinations, AI Sycophancy, Fairy Tale Tax

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!

This first in a five-part series documenting my attempt to measure the AI and human contributions in AI-augmented writing. What began as a simple experiment with vague prompts and weak planning turned into a cautionary tale about hallucinations, lost context, and the real cost of trusting AI too much.

• Part 1: The Count That Couldn’t
• Part 2: Algorithmic Theatre
• Part 3: Crash Course in Collaboration
• Part 4: Victory in a New Venue
• Part 5: Not Magic, Beyond Measure

TL;DR: After 45 hours, my initial attempt to measure the true author of contributions to articles had failed. What began as a simple JSON object became a mess of Python scripts, fictional functions, and AI claiming success while churning out empty files. The hard lesson is that the “Human Tax” of inadequate preparation can cost as much as any “AI Tax.”

I Don’t Know Python, My AI Might

It started with doubt. I’d seen comments in forums dismissing other people’s articles as “AI slop.” This made me question my own work. I’ve never been a strong writer, but using AI to begin articles has helped me get past the procrastination of even starting one.

But did my writing sound like me or just scrambled AI slop? I hadn’t thought about it before, but I couldn’t stop thinking about it from that point on.

Large language models might seem smart and reliable, but large datasets and curated personalities don’t grant AI human judgment. Give it vague instructions and it’ll hallucinate. Hold a long conversation and it’ll start forgetting what was said. Ask a vague question and you’ll get questionable answers.

Trying to track every edit would be a monumental task, and AI certainly couldn’t be trusted to document itself accurately. So I decided to build something that could do it without spending hours of my time on every article. That’s where this series begins.

JSON, XML, and Broken Context Windows

I started with JSON because that’s what I was familiar with. I started with counting characters and words, doing the math between versions, and generating basic metrics. The initial character counts looked questionable and word counts were clearly wrong.

I mentioned it to ChatGPT, which added JSON and improved the results. Then it was sentences, then paragraphs. Soon I had a JSON object that grew over 11,000 characters and collapsed every time I fed it versions of an actual post.

XML had worked for me with heavier processes in the past, so I switched to Claude, which is optimized for it [1]. The entire system groaned under the weight of trying to juggle content analysis and visualization. The results were marginally better, enough to produce a few weak visualizations before we’d blown past the context window.

An AI Trust Exercise

Six months ago, I couldn’t read a single line of Python, but I’d already surrendered the writing of the XML by that point, so I didn’t put any thought into changing direction.

I didn’t consider that I’d have to trust the AI completely. I wouldn’t have a clue if the code was legitimate or pure gibberish from that point forward. That’s also how my crash course in Python began, but I’ll get to that later.

When Success Builds Confidence

To avoid the same complexity that had doomed earlier attempts, I split the process into smaller steps. The first script would process article stages and output metrics, then the second would analyze these results across stages and produce visualizations. This felt like a clean separation that would prevent the problems I’d already faced.

Previous  | Current   | Word          | Sentence      | Paragraph
Version   | Version   | Retention (%) | Retention (%) | Retention (%)
--------- | --------- | ------------- | ------------- | ------------- 
Draft     | Refined   | 39.48         | 2.22          | 0.0      
Refined   | Edited    | 43.35         | 0.0           | 0.0      
Edited    | Final     | 53.9          | 0.0           | 0.0      
Draft     | Final     | 48.52         | 1.11          | 3.45     

The first script went surprisingly well. With clear, sequential instructions, the AI produced something I could actually run. It processed all four versions of a test article without complaint, returning word counts, character counts, and other metrics. For someone who couldn’t read Python, it felt like magic after the prior experience.

When Fixes Create Fiction

The second script shattered the illusion completely. What seemed like magic was actually clever sleight of hand. Each version broke in new ways.

• Version 2 worked fine.

• Version 3 quietly changed the output structure.

• Version 4 hardcoded data in the file making it useless.

• Version 5 dropped functionality and pointed at files that didn’t exist.

Index       | Draft      | Refined     | Edited     | Final 
----------- | ---------- | ----------- | ---------- | ----------
Draft       | 1.0        | 0.356       | 0.5427     | 0.4361
Refined     | 0.356      | 1.0         | 0.328      | 0.3109
Edited      | 0.5427     | 0.328       | 1.0        | 0.5531
Final       | 0.4361     | 0.3109      | 0.5531     | 1.0   

Through it all, the AI kept congratulating itself. Each “finished” run produced CSVs with half-empty columns, calculations that didn’t add up, or percentages that broke math itself.

I couldn’t make sense of the code, so I had no way to spot problems early on. The AI couldn’t be trusted to identify them, even when I told it to check its own work. Small gaps between what I expected and reality grew into hours of debugging imaginary issues.

When Polish Hides Proof

Visualizations became their own drama. The most impressive-looking results usually concealed the most fundamental issues.

Claude produced charts that looked incredible with polished bars, clear comparisons, and clean layouts that were easy to understand. But they only exported as SVGs, and those came with their own swarm of issues.

ChatGPT’s charts looked plain and sometimes oversimplified, but they exported without issues and worked everywhere I needed them. Plain but reliable beats polished but broken every time.

The Fairy Tale Tax

The revelation wasn’t about AI’s limitations but my belief in the fairy tale that AI would handle complex tasks, leaving me to be creative. But fairy tales often come with a price.

This pain isn't abstract. Many designers face the same struggle when they lean too hard on AI while dealing with organizational pressure and unrealistic expectations from leadership about adoption and success rates. We end up with wasted hours and fragile outcomes, feeling like we're battling our own tools.

When working with AI, you can't avoid paying a tax. The question becomes when to pay it. You can invest in preparation and educating stakeholders, or pay in debugging and rework afterward. But the fairy tale tax always gets paid.

The Real Education Behind the Breakdown

Inadequate preparation compounds quickly.
I learned this the hard way. I started with a half-baked idea and a questionable approach that ballooned into hours of trying to use AI to figure out and fix scripts that never worked to begin with. Without clear goals, every crack in a plan can become a chasm.

Overconfidence is contagious.
The AI’s initial success infected my judgment. The first script worked so well I didn’t notice when things went wrong. When 404 links and half-empty CSVs appeared, something was clearly and irrevocably broken. AI confidence never equals accuracy.

Unclear goals create challenging tasks.
I didn’t have a plan and asked the AI to solve every problem in isolation. It invented fixes that sounded good but had no basis in reality, causing phantom modules, missing functions, and nonsense metrics. When you ask for the impossible, AI creates fiction.

Some domain knowledge is essential.
I couldn't read Python, so I had no way to spot when something wasn't right, which leads to everything being busted. Working with AI in fields doesn't require mastery, but you need to know enough to recognize issues or a way to validate before it's too late.

The End That Wasn’t

This wasn't the project's end, but it was the end of that approach. All I had left were broken scripts and mounting anger. I was convinced the project was unrealistic and gave up completely.

After a few weeks, I came back planning to use HTML and JavaScript, languages I had experience with. I had concluded simpler tools would mean simpler problems. I was wrong, but what I discovered wasn't technical at all. The next-level models brought next-level complications.

That's where the next article begins.

Citations and Resources

[1] Anthropic: Use XML tags to structure your prompts

Perplexity: The Fairy Tale Tax

Tags: AI for UX Designers, AI Hallucinations

William Trekell : Linkedin : Bluesky : Instagram : Feel free to stop by and say hi!