But I think you’re missing a pretty basic factor: book sales. The production of any work of fiction is most immediately explained by market forces. There would be no science fiction if writers couldn’t sell their novels successfully.
Exactly the opposite is true. Majority of science fiction creators struggled for majority of their lives. Only extremely few found commercial success and then they very often propagated ideas found in others' work.
The success of a entertainment product is not the same as the success of a creative idea - better described in this context as "meme".
"Star Wars" original trilogy was immensely successful despite being plain, boring and repetitive in all creative terms except for visual effects which is the main reason why it succeeded. "A New Hope" had two revolutionary elements - the iconic spaceship scenes and the combination of eastern spiritual and mythological elements at its very fundamental level. Star Wars is a breakout hit because it's a kung fu movie in space (which triggers emotional response "I can relate to this") dressed up in medieval chivalry and filled with fun spaceship designs recreating WW2 aerial and naval battles (which triggers cognitive recognition: "I know this").
But "Star Wars" was created on the backs of countless stories that had incomparably smaller commercial effect or in many instances were commercial failures.
You likely wouldn't tell it if you saw me face to face but I'm a proper gigantic (50+) nerd. 99% of my or my fellow gigantic nerd's engagement with creative content was engagement with self-made or commercially "barely viable" content. The people who are "nerds" today are ... even less than posers. They're brainless idiots who can't tell feces apart from pearls. The "whales" who fund the businesses were always the minority in our population. Being a nerd was about what you do, rather than what you have.
"Sci-fi" is not a handful of successful products. It's the entire culture associated with the mindset and ideas and - most importantly - the very people who cultivate it. This is why Star Wars now is a failure. It first ignores and then insults the very people that made it a success. These people made Star Wars big not because of the Star Wars movies but because of the countless recreations of the story or its derivatives in their own minds.
Matrix didn't become a hit because of the movie but because it could be seemingly endlessly applied to everyday culture.
Harry Potter didn't become a hit because Rowling is an amazing author but because the books were a type of "core handbook" for roleplaying game (including an important live action component) that the children reading the books could engage in with endless gadgets sold at a significant markup because of franchising rights.
In other words you seek the forest and miss it because in your mind a forest requires a solid wall of trees not trees scattered all around an area.
What made western sci-fi art a success was a culture recceptive to the ideas it carried. What makes Chinese sci-fi a failure is the lack of this very culture. How that culture emerges - I explained it in my previous post. But you think that it's all about mainstream products that become commercial success.
When the mainstream product becomes a commercial success the creative success is already complete.
Star Wars didn't explode ex nihilo. It was the culmination of half a century of American sci-fi at the time when it was mostly overtly disrespected and ignored by the mainstream. And American sf wouldn't have existed without European sf that preceded and inspired it. It had to accrue critical mass to explode the way it did and all that was necessary was a trigger that was "A New Hope".
Similarly Marvel movies didn't explode ex nihilo. They were a culmination of half a century of American superhero culture at the time when it was mostly disrespected and ignored by the mainstream. It had to accrue critical mass and wait for the trigger that was "Iron Man". Marvel movies succeeded because they were made by Marvel fans for Marvel fans. Previous superhero movies failed because they weren't. That was the critical mass.
Same with Lord of the Rings type of fantasy. Same with BDSM Vampire Romance. Same with every other "sudden unexpected success story".
The mainstream is dumb. It has only the feeding orifice and the defecating orifice. It must be fed something good to defecate something acceptable because as the feeding continues the quality will degrade over time.
Or to use a more scientific explanation - dissemination and quality of ideas is like movement through spacetime and mass/energy. Except that instead of spacetime you have noosphere - people and their imaginations. Movement is dissemination of memes. Mass/energy is the attractive potential of the meme - unknown and attractive or repetitive but recognizable. And from there go with the same logic as genetic inheritance since memes and genes are one and the same thing just in different medium.
If you have high energy you move through spacetime. If you have high mass the spacetime moves through you. High energy is individual attractive memes (niche populations of creative individuals). High mass is large numbers of repetitive memes (mainstream population of "normie" individuals).
When high energy dominates as space it is filled with high creative (energy) potential but it's like plasma or magma. It doesn't appeal to large mass (mainstream). Mass (recognition) wants more mass. Energy (originality) wants more energy.
When large mass begins to dominate a space that is a part of the multi-verse of cultural noosphere it begins to bend "spacetime" around itself and the high energy original memes begin to be drawn into the large mass that outnumbers and swamps it. The intertia of large mass makes it impossible to change and since time slows down around large masses there is no inside the mass. It's creatively stuck in time. (As are the high energy ideas - ironically). Only the highest energy ideas survive.
But unless you are one of these high-energy receptors you won't notice them until you're drawn by high mass. You don't know that you're in a forest until you trip and fall into a tree.
That's literally how the universe works. It's organised around mass rather than energy.
And if we add that art is mostly a response to trauma and high energies affect matter in the way that resembles trauma (gamma rays have highest energy and are absolutely destructive...) you may start to think how much of our existence is just a fractal of the basic physics of the universe.
There’s only so many new ideas considering, as you said, we are no longer in the middle of a period as disruptive to society as the Industrial Revolution was.
Ironically we are absolutely in one such period. Just that the disruption happens around our understanding of mind, identity and self, sexuality and reproduction and evolution understood both narrowly and broadly. We're just entering the creative phase.
The future is weird and exciting but it will be extremely difficult to tell apart the insane and destructive from the sane and creative. Just like it was every time in the past during such changes. Machines. Societal change and revolution. Lack of god and religious morals. Robots. Nuclear power. Germ theory. Sexual revolution. Astrophysics. Relativity. AI. That's why we have the fairytales and their monsters.
Ironically the "woke" garbage that dominates art may be the infection that will trigger adaptive response, the very factor that causes the evolutionary shift into another stage of punctuated equilibrium. We may yet get interesting art. In many ways we are already beginning to get it. And who knows, perhaps the monsters are out first and the monster-hunters are only beginning to find their courage. Who knows?
You just have to keep the eyes open. The mind's eyes. And if we are to end... even death can be a road to awe.
----
And since we're here I think it's time to return to my original avatar. While still displeased at least it represents what I'd like to represent - if you know what it truly means. Have a good week.
Last edited: