“Ghiblification” and quick thoughts on distinguishing generation from creativity

My job has put me in the position of explaining creativity to technologists for a long time now. This function became about a thousand times more fraught as AI-generated images transitioned from “nightmare fuel” to what we see today.

Wanting a chance to think about this with people I respected led me to the “Balancing Creativity and Mastery” group at Polaris last year. The collaborative process pulled a kind of prototype framework out of my head that I’ve been meaning to articulate for awhile. I won’t be doing that in full here, but briefly:

It’s useful to think about creativity in three dimensions or tiers:

• Craft: the ability to execute a style with tightly defined rules;
• Synthesis: the ability to remix crafted styles into combinatorial variations;
• Invention: the ability to define new style rules. 

There aren’t a lot of reasons to create frameworks like this, practically speaking, but in my particular communication work I’ve found that this helps people generally outside creative communities to think about creativity. And it’s especially relevant for AI.

AI thus far has demonstrated high abilities in craft, moderate abilities in synthesis, and no ability whatsoever in invention. Moreover, I personally find it fascinating that invention may be an impossible wall for today’s AI architectures -- but that's another topic.

One of the things that I appreciate most about AI is the way that it pushes us to define things that we previously took for granted. Fluency is the most recent example; prior to LLM technologies, I think we confused fluency for intelligence more than we do today. Glibness or wittiness has been a hack around human skepticism for as long as we’ve had language, and it turns out a machine can do it much more readily than it can count the number of r’s in “strawberry”.

In this way, I think what we are seeing from generative image models is that there is a strong distinction between generation and creativity, even though it’s easy to confuse one for another.

While I was writing this, Cory Doctorow posted about AI art in a cogent way that I think will resonate greatly with creative people. He emphasizes the hundreds of “communicative decisions” that take place in the human creation of art, and how these are essentially absent in the creation of AI art. I like this take, although I suspect, given the way the internet works today, the people who might benefit most from the message will never receive it.

Like Cory’s highlighting of communicative decisions in the practice of creating art, creativity is actually optional in the generative process. So is that communicative decision-making – or rather, that communication happens entirely internally and we don’t really have a way of knowing whether it’s going on inside a person or not. Decisions are being made, and maybe you can argue that those communicative decisions happen on a subconscious level even if the maker is unaware of it, but I would argue it is possible to make what others would recognize as “art” without participating in that decision-making process. However, the result is likely to be one of those things that causes a lot of argument about what “art” is.

The great dark ghiblification highlights for me the importance of the distinction between generation and creativity, and the relationship between these two things is at the heart of why these technologies are eliciting genuine hurt in the way they currently operate.

“Generation” is part of the creative process. It is a necessary component of creativity, and it's quite hard, so there's an uncanny impressiveness to a machine's generative capability. A machine has no hesitation, because it has no inner self with which to hesitate. And I should say, specifically, that today’s foundation models have very little hesitation – they may sometimes arrive on refusal, but this is still an unhesitating navigation to a place in their latent space based on tuning by the AI’s creator.

“Writer’s block” is perhaps the most famous barrier to generation, and it is importantly a barrier to that particular aspect of creativity more than anything else. The other aspects of creativity – soul searching, analysis of other creative works, voice formation – those can occur separately but without generation they are invisible.

Cory describes art as the translation of “a big, numinous, irreducible feeling that fills the artist's mind”, and in this he touches on one of the things that distinguishes creativity from generation. The confrontation with one’s inner big, numinous, irreducible feelings is a process that only happens inside a richly complex inner self, a system of patterns altered by every moment of its existence. This is what makes art hard; when we look inside ourselves to confront those feelings, we can be terrified.

Creativity requires bravery. And bravery is only possible in the presence of fear. A machine has no fear and therefore has no hesitation, but neither does it have bravery. The creative process involves opening the box of ghosts inside your soul and searching for the one that can connect you to other humans across time and space.

If this ghostly feeling has been expressed before, you are in the territory of craft and synthesis. If it has not been expressed before, you are outside the book – you are inventing.

Where this becomes hurtful for people who have engaged with the creative process and its various demons is when this invention, which is the emergent culmination of so much inner work, so much practice and insight, is then peeled off and applied like a sticker to content that did not go through the creative process. It is a subversion of that thousandfold chain of communicative decisions. And it doesn’t even really matter what the end result looks like – because the process is the point.

One of the reasons for the disconnect between the pro-AI-art crowd and the anti is also a confusion between commercial and noncommercial art. A great deal of art on the internet is commercial art – art created with an intention to sell products. But much of the art that lives in training datasets was not commercial to begin with; it was practice art or art created by artists building portfolios and calling cards. It might look beautiful, it might even look like commercial art, but communicative decisions are being made in the nascent artist in terms of what to imitate and practice. Because the distinction happens on a process level, the line between commercial and noncommercial art is often blurry, and this is why we can get strange language in lawsuits arguing over whether artists should receive overtime pay, for instance; they hinge on whether the work is “creative”. (In the cases where artists have won overtime lawsuits in the US, it has been on the basis that the work they were doing was not creative.)

The intersections between art and commerce are way too complicated for a “quick thought” blog post, but another missed aspect of the AI art debate concerns the type of work that is used in training data. (This is beyond the foundational consent issue which stands on its own.) To use something someone created for a generative purpose feels like a violation on a basic level to many people who have experienced the creative process. There's a falsity to it, a fundamental misunderstanding of what the process entails. The reaction is instinctive, visceral-- it comes from an understanding that the craft is an articulation of the creativity, and the creativity, being this deeply intimate act of facing one’s internal demons, is sacred.

Our rimban at Seattle Betsuin recently quoted Nagasaki atomic bomb survivor Katsuji Yoshida, who gave many presentations on his experience, and closed all of them with the words: “the basis of peace is for people to understand the pain of others.”

I don’t think that the pain concerning generative AI is simple. I think there is pain in the hearts of people who, as children, want to be “creative” but think they aren’t talented enough to do so. I wonder if many of them go into technical fields. And I don’t doubt that they get a glimmer of joy from generative AI and a feeling of having created something – the same way that Guitar Hero gives you a feeling of performing music.

I think it is possible to both recognize this pain and also recognize the enormous anxiety and hurt of the artists who feel injured by generative AI. I don’t think that we can move forward with these technologies without resolving this – or rather, I hope that we don’t. To bury this injury is to plant the seeds for it to flourish in harmful action later, inevitably. And we can never have peace if we look away from the pain.