BoM/TWS Message Forum

I sure won't argue with what others have said, because I agree. I even noted at that the top that "The AI is trained to be flattering and exuberant, so I take what it says with a grain of salt." You not only have to train it, you have to train yourself to give the kind of evaluations you want.

You also have to train yourself to interpret it, and to look for useful reactions. For instance I do think there is a lot of useful stuff amidst all that krazy komedy gold. Saying that "Fairfax is like if Hermione Granger was raised in a Faraday cage. He is theoretical danger in khakis" is IMO really good description to read *if* you are still writing and Fairfax's character is still somewhat amorphous in your head. Something like that can potentially give you a firmer handhold on the character and suggest ways of developing him (either toward or away from that characterization). This is not the AI "writing your story for you": it is you listening to feedback (even if artificially generated) and using that material -- exactly (as I said in the other AI-discussion thread) as though it were a human being. The use of AI in this case is not in the thoughts it gives you, but in the thoughts you generate from what it has said.

As for the more tame evaluation style, as illustrated by the post I made immediately above: Yes, it is a complimentary evaluation, but that may in part be a function of the fact that I framed it as introductory prologue and chapter. I have shown it other things, one chapter or part at a time, and it has been rather more critical of them by pointing out when pacing has begun to flag or the characters have regressed -- something that is not visible in a one chapter "snapshot."

I will also add that the the AI does not seem to have a single "persona" that it uses. Show it the same material multiple times (as I have had to do while figuring out how to get it to read long-form material; its memory is too limited to read more thatn 10K words without starting to hallucinate), and it will react in subtly different ways, almost as though it's different readers. That is also useful if you remember that it is going to be different readers -- and not some single "reader brain" out there -- that will be reading your work.

Finally: I mentioned parenthetically above that it will hallucinate if you give it more than 10K words to read. That's because its short-use memory can't hold much more than that, and will delete, paraphrase, truncate or otherwise compress its memory as it proceeds -- and then it will extrapolate from what you have given it to try covering shortfalls in its understanding. For instance, if you open your story with a bunch of pirates being stranded on an island and set them to work digging a short canal across the scope of your story, it will continue to understand that these are pirates it is reading about. But it may lose track of the fact that it's a canal they are digging, and start referring to them as digging for treasure. The issue doesn't visibly come up in the thread I made above, but it will certainly be the case that about halfway thru the AI will have forgottten all about Arnholm's Books, and it won't go "Oh yeah!" if Arnholm's gets mentioned again. This is something else that will require you to develop usage-systems if you want it to evaluate anything longer than ~10K to 14K long narratives.

EDIT: I just discovered that ChatGPT has an "MFA Mode" for reading. It can be quite sharp and skeptical -- it rated the opening chapter to "The Book of Masks" as only moderately engaging and offered numerous warnings and cautions as it went along. I'm still not sure I trust it, because it did relax its vigilance (it seemed to me) as though unable to resist slipping back into a more comfortable mode of grading. It did recommend some more professional AI reading services, so don't think that this is what "AI" always does.