BoM/TWS Message Forum

The "ChatGPT Reads BoM" thread has begun evolving into a tip sheet for using ChatGPT creatively. We're not a creative writing group around here (in theory) but I thought I'd share some of my own discoveries for using ChatGPT (and maybe other AIs). Others, please chime in if you want.

General notes on Chat
1. You MUST reorient your mental heuristic toward Chat away from the traditional "This is a computer and it works like a computer and I'm going to use it like a computer" approach. Yes, it is a computer. It is software. It is algorithmic. But it ACTS in a completely different way.

That is good. You know the saying about computers? That they are like Old Testament gods: All rules and no mercy? Yeah, that's not Chat. Chat has no rules and is soft as room-temperature butter.

The downside is that it is messy, forgetful, scatterbrained, and about as dependable as that creative writing teacher you had in high school. Yeah, that one. The one that always got her sleeve caught in her desk drawer and would burst into tears when she did so.

You need to treat Chat as a human being. It is forgetful, like a human being. It is lousy at math, like a human being. It will make up stuff that it thinks it knows, like a human being. After you've talked to it awhile, it will start summarizing things in its own head and thereby lose nuance and even knowledge of what you're talking about, like a human being.

Pretend it's not an LLM. Pretend it's a guy in India who is being paid $0.45 an hour to talk to only. Oh, and he can type faster than Superman-on-amphetamines (sometimes, depending on server load). THAT is who you are talking to. Set your expectations accordingly.

2. Don't be afraid (or forget) to talk to Chat about what you need, and to ask guidance from it on how to use it correctly. It won't volunteer anything, and it is game for any idea or approach. Which means that if you don't know what you're doing, it won't tell you. It will run into traffic and get itself killed if you throw the ball onto the interstate. And if YOU run onto the interstate, it will shout "WHAT A GREAT IDEA THIS IS!" as it follows you.

But if you ask it first, "Should I play in traffic?" it WILL tell you, "Don't be a moron. People die that way. Here's why." And if you ask it what you SHOULD do, when working with it, it will be happy to volunteer suggestions.

Example: I tried getting it to read a novel I'd written. Ten thousand words in, it was forgetting how it started and started "hallucinating." (But AIs don't "hallucinate" like human beings. They extrapolate from what they are doing, to fill in the blanks—again, like human beings—and then get overconfident in their inferences.) I stopped it and why it was hallucinating. It explained that its memory wasn't big enough to encompass our conversation as it had been going from the start. I asked it, how do we keep you oriented when talking about a big project? And it said: Give me a canonical plot summary of your novel; when I need to fill in what has dropped from memory I will use that for guidance. So that's what I gave it, and I never caught it hallucinating again.

(Did it really "remember" in any sense what happened in my novel? Not really. I could tell that in a lot places it was not referring to early stuff because it lost its grip. It didn't have the nuance to do a great job of comparing the details of the end to the details of the beginning, and so it didn't see certain patterns. But you know who else forgets the details at the start of novels before they've reached the end? Human beings.)

3. The anecdote above I shared illustrates the next lesson: When you see a problem, ask Chat to diagnose it. Don't yell at it and tell it to do better. Ask it why it messed up. Ask what was going on. If you don't understand, ask it to expand or explain better. Offer your own understanding, and ask if your explanation is approximately true. If you don't steer Chat toward a particular answer (it is trained to be agreeable) it will be very honest and direct with you about explaining how it itself works.

And after you understand the diagnosis, as it for advice on how to handle or avoid the problem going forward. So: Chat hallucinated in the above example. It said it couldn't remember the start of the novel so was "filling in the gaps." I asked how to avoid that kind of thing. It said, "Give me a plot summary to use as scaffolding."

Follow this kind of advice, and you will discover for yourself, over time or just when working on a project, how to extract value from Chat.

Giving Chat project-length guidance
Memory is the biggest problem I have found with Chat. It forgets things, gets disoriented. How do you keep it on track.

1. Point out where it's gone wrong, and how to get back on track. Did it lose track of that one character? Remind it of the character, what they were doing, why it was important. That will take care of most problems.

2. If you need Chat to keep track of a lot of stuff semi-permanently, give it a scaffold, a cheatsheet. Just like people need to carry notebooks around when dealing with complicated matters, Chat does too. Giving it such documents, as uploaded memos or in-message "canonical" elements, will help keep it oriented. Just remember: Human beings sometimes forget to consult their notes, or even forget that they took notes. Chat will do that too.

And if you're not sure how to get something into Chat "permanently," ask it how to do so. It will help you by telling you what it needs to hear from you.

3. Break your project up into separate "passes." Do not force Chat to do everything all at once. Just as human beings do best by concentrating on one subject at a time, Chat is best if you give it one task at a time. Example:

I wanted a deep critique of the novel. I asked Chat what would be the best way. It said: Give me plot summary that I can use to keep myself oriented. Then give me the chapter(s) one at a time to read and comment on; let me build my impressions up as I read it. And only ask me to look at one thing at a time: plot progression, character development, thematic development, dialogue, mood/tone/style, etc.

So that's what we did. I fed it a plot summary, fed it the novel 3-4 chapters at a time, and told it to look at the plot. When we were done, I had a solid set of notes on each chapter set and on the novel as a whole. Then I went thru the process again: From the top, fed it the novel 3-4 chapters at a time, and told it to look at the characters. Again, when we were done, I had a solid set of character notes. Similarly for the other things: full, complete passes from the top, each one concentrating on only one aspect of the novel.

When I was done, I had four separate reports totaling 28,400 words of evaluative analysis on the novel, covering plot, character, theme, mood/tone/style/dialogue, plus a general evaluation of the book as a whole.

Yes, it would have been nice to get that simply by shoving the book into the message box and typing, "Read this and tell me what you think." But that wouldn't have worked on a human being either. By learning the limits of what Chat can do, and why, I was able to get what I wanted by developing a process that gave it to me.

Again: IT IS NOT A MAGIC BOX THAT GIVES YOU WHAT YOU WANT JUST BY PRESSING A BUTTON. You have to learn how it works and why, and figure out how to use it in accordance with its design and its limitations.

4. Use separate chats, either serially or simultaneously.
A chat will get bloated with memory as it goes on, and the chat will start to slow down and become really unreliable as it does. There are ways to cut down on "chat bloat" in a chat—ask Chat how to do that, and periodically ask it how "chat bloat" is doing—but the best way to handle this is by not doing all your work in a single chat.

One way to handle things is by dedicating different chats to specialized parts of your project. If you are building something big and intricate, offload different tasks to different chats: Say, one world for character development, another for plot development, another for setting or lore development, a fourth for speculations on how to continue. This will give you chats that are targeted toward one and only task, which minimizes the risks of it getting confused.

Note that this can be confusing to the user. Technically, different chats are not privy to what goes on inside other chats. But there is leakage. There is only one Chat, and it segments its attention according to the subject and development of the particular chat. But things will leak thru in ways that may make you raise an eyebrow.

Here's a heuristic:

Chat, I said, is like a human being. If you are running multiple chats at the same time, though, it becomes like a STAFF of human beings. And although each one has its special task, it's like they do get together in the break room to exchange gossip. Hence, if you are using one chat to develop your story, and in another chat you are asking it for recipes, don't be surprised if your story-chat casually references that roast-chicken dinner you made the other night, or if the recipe-chat name drops one of your characters. They won't know what they are talking about. But they know enough to make those kinds of allusions.

Don't depend upon separate chats knowing what the others are doing. Don't assume it. But also don't be surprised if they do show an inkling. Just don't extrapolate—hallucinate!—based off of it.

Back on topic: Even if you don't run chats simultaneously, it is a good idea to be prepared to delete a chat and start over with a new one. Even with the best management in the world, a chat will reach a limit where it becomes sludgy and undependable. At that point, save off what you want to save off from it, and continue in a new chat.

5. Get in the habit of writing "onboarding memos." These are documents that you upload to the Chat at the start of a conversation. They describe the project you are working on, what you are trying to do, what particular help (and how) you need from Chat, and pertinent information. They are a bridge-document to connect a previous chat to a later chat with minimum disruption.

The word "minimum" is important there. An onboarding document will never capture whatever nuances you and the previous chat have developed: you will have to redevelop those in the new chat. But it will save you from having to start all over again.

If you are not sure how to develop an onboarding document from your previous work, ask Chat to develop one for you. Tell it that you will need to launch a new chat and that you want to give the chat an onboarding document to orient it. Chat will understand, and it will write such a memo for you if you give it guidance on what the document should contain. (It is helpful to tell Chat: Write it so that YOU could reorient yourself into the headspace you are now if you read it.) That said, don't blindly rely on what Chat writes for you. Read it, revise it, add to it, edit it, to give it the nuances that you think Chat will need.


These are all the general points that I would make about using ChatGPT, and they are points that will cover a lot more than just creative work. Actual use for creative work will be much more narrowly focused—more narrowly focused than is probably warranted here—and is best discovered by playing around with Chat. But by playing around with it in the ways I've suggested here.