Is Artificial Intelligence and its impact on humankind inevitable?
26 min read
Ā
Halfway between the 1908 publication of A Room With a View (a light-hearted Edwardian-era romance, featuring Miss Lucy Honeychurch touring Italy), and 1910ās Howardās End (an exploration of social class set in an English country house), celebrated British author E.M. Forster would take an unexpected detour from his usual literary fare.
Forster, a member of the famed Bloomsbury Group, would go on to write other classics, like A Passage to India and be nominated 22 times for the Nobel Prize in Literature.
But in 1909, he would publish an odd short story unlike any of his other works: one set in an indeterminately dark, dystopian future (the term Science Fiction wouldnāt be coined for another 45 years).
In The Machine Stops, Forster would create a world in which humans have moved underground, sealed away from an ecologically ruined environment.
Theyāve evolved to have all their physical and mental needs taken care of by The Machine:
Then she generated the light, and the sight of her room, flooded with radiance and studded with electric buttons, revived her. There were buttons and switches everywhere ā buttons to call for food for music, for clothing. There was the hot-bath button, by pressure of which a basin of (imitation) marble rose out of the floor, filled to the brim with a warm deodorized liquid. There was the cold-bath button. There was the button that produced literature. And there were of course the buttons by which she communicated with her friends. The room, though it contained nothing, was in touch with all that she cared for in the world.
The function of humanity has become whatever The Machine is incapable of doing on its own ā mainly synthesizing ideas by Advanced Thinkers, and procreating. This, too, is regulated to keep the death and birth rates at equilibrium. Infants would be evaluated shortly after birth, and those deemed too muscular or athletic, who would likely be unhappy in a fully managed, underground environment, would be mercilessly terminated.
What remained of humanity would live in fear of fresh air, natural sunlight, and clouds.
In real life, the first transatlantic telephone call would not be made for another 20 years. But here, Forster anticipated The Machine enabling instant, networked communication across the world via round glowing plates, complete with live audio and visual streaming.
The Machine also provided medical care for the humans, and facilitated exchange of knowledge (including Zoom-like virtual presentations). The system even had its own Mending Apparatus to maintain and repair its physical self without human intervention.
But it wasnāt all panacea. Those who behaved unacceptably would be shunned and threatened with homelessness ā eviction into the above-ground toxic atmosphere, and certain death.
All information and knowledge went through the world-wide web of The Machine, which steered Advanced Thinkers away from raw, human experience:
Those who still wanted to know what the earth was like had after all only to listen to some gramophone, or to look into some cinematophote. And even the lecturers acquiesced when they found that a lecture on the sea was none the less stimulating when compiled out of other lectures that had already been delivered on the same subject. āBeware of first-hand ideas!ā exclaimed one of the most advanced of them. āFirst-hand ideas do not really exist. They are but the physical impressions produced by love and fear, and on this gross foundation who could erect a philosophy? Let your ideas be second-hand, and if possible tenth-hand, for then they will be far removed from that disturbing element ā direct observation.
Eventually, the unquestioning admiration of The Machine led to the re-establishment of religion:
Those who had long worshipped silently, now began to talk. They described the strange feeling of peace that came over them when they handled the Book of the Machine, the pleasure that it was to repeat certain numerals out of it, however little meaning those numerals conveyed to the outward ear, the ecstasy of touching a button, however unimportant, or of ringing an electric bell, however superfluously.
āThe Machine,ā they exclaimed, āfeeds us and clothes us and houses us; through it we speak to one another, through it we see one another, in it we have our being. The Machine is the friend of ideas and the enemy of superstition: the Machine is omnipotent, eternal; blessed is the Machine.ā And before long this allocution was printed on the first page of the Book, and in subsequent editions the ritual swelled into a complicated system of praise and prayer. The word āreligionā was sedulously avoided, and in theory the Machine was still the creation and the implement of man. But in practice all, save a few retrogrades, worshipped it as divine.
Faults inevitably entered the system. But instead of working to fix them, humanity shrugged helplessly and adapted:
Time passed, and they resented the defects no longer. The defects had not been remedied, but the human tissues in that latter day had become so subservient, that they readily adapted themselves to every caprice of the Machine. The sigh at the crises of the Brisbane symphony no longer irritated Vashti; she accepted it as part of the melody. The jarring noise, whether in the head or in the wall, was no longer resented by her friend. And so with the mouldy artificial fruit, so with the bath water that began to stink, so with the defective rhymes that the poetry machine had taken to emit. All were bitterly complained of at first, and then acquiesced in and forgotten. Things went from bad to worse unchallenged.
Forster was a strong advocate of Humanism, a philosophy that emphasizes the individual and the agency of human beings. His other works are clear manifestations of these ideas.
So why the shift? In his 1947 Collected Short stories, he says that he wrote The Machine Stops as: āa reaction to one of the earlier heavens of H. G. Wells.ā
H. G. Wells, in his 1905 novel, A Modern Utopia, presents a vision of an alternate world governed by a single allied āWorld State,ā with a common language, economy, currency, and perfect equality.
This world is ruled by a select order of elites Wells calls The Samurai. They hold all positions of power. In this world, the poor, disabled, and criminal are banished to islands where they can not procreate and sully the shared gene pool.
Humanity has been sorted into the Poietic (people who create), the Kinetic (people who build and maintain but do not create), the Dull (the poor, less intelligent, and less creative), and the Base (corrupt, immoral people).
Forsterās story was a response to the Wellsian notion of a Single Entity, ruled by shared faith.
In theory, this phenomenon is driven by the emergence of artificial intelligence (AI) that surpasses human cognitive capabilities and can autonomously enhance itself. The term āsingularityā in this context draws from mathematical concepts indicating a point where existing models break down and continuity in understanding is lost. This describes an era where machines not only match but substantially exceed human intelligence, starting a cycle of self-perpetuating technological evolution.
The theory suggests that such advancements could evolve at a pace so rapid that humans would be unable to foresee, mitigate or halt the process. This rapid evolution could give rise to synthetic intelligences that are not only autonomous but also capable of innovations that are beyond human comprehension or control. The possibility that machines might create even more advanced versions of themselves could shift humanity into a new reality where humans are no longer the most capable entities. The implications of reaching this singularity point could be good for the human race or catastrophic. For now, the concept is relegated to science fiction, but nonetheless, it can be valuable to contemplate what such a future might look like, so that humanity might steer AI development in such a way as to promote its civilizational interests.
The Machine Stops was published in 1909, the same year as Filippo Tommaso Marinettiās Manifeste du Futurisme (The Futurist Manifesto), which proclaimed a very different vision of the future (and a love of fast machines as a symbol of progress):
We declare that the splendor of the world has been enriched by a new beauty: the beauty of speed. A racing automobile with its bonnet adorned with great tubes like serpents with explosive breath ā¦ a roaring motor car which seems to run on machine-gun fire, is more beautiful than the Victory of Samothrace.
ā¦
We are on the extreme promontory of the centuries! What is the use of looking behind at the moment when we must open the mysterious shutters of the impossible? Time and Space died yesterday. We are already living in the absolute, since we have already created eternal, omnipresent speed.
We want to glorify war - the only cure for the world - militarism, patriotism, the destructive gesture of the anarchists, the beautiful ideas which kill, and contempt for woman.
We want to demolish museums and libraries, fight morality, feminism and all opportunist and utilitarian cowardice.
ā¦
The oldest among us are not yet thirty years old: we have therefore at least ten years to accomplish our task. When we are forty let younger and stronger men than we throw us in the waste paper basket like useless manuscripts!
In Forsterās story, the protagonistās son describes escaping to the surface and experiencing the physical world (at his near peril). He issues a dire warning to Advanced Thinkers:
āCannot you see, cannot all you lecturers see, that it is we that are dying, and that down here the only thing that really lives is the Machine? We created the Machine, to do our will, but we cannot make it do our will now. It has robbed us of the sense of space and of the sense of touch, it has blurred every human relation and narrowed down love to a carnal act, it has paralysed our bodies and our wills, and now it compels us to worship it. The Machine develops ā but not on our lines. The Machine proceeds ā but not to our goal. We only exist as the blood corpuscles that course through its arteries, and if it could work without us, it would let us die.
Forsterās storyline of humanity moving underground due to a natural disaster had already been explored more than a decade earlier in 1896, by French sociologist, statistician, and criminologist Gabriel Tarde.
In his story Fragment dāhistoire future, he writes of a world where the sun is about to go dark. Humanity finds a way to move underground, but this time, what saves mankind is not the might of science.
Itās Poetry and Art.
Society is driven by study of Homer, Sophocles, and Euripides. All significant works of art have been moved underground and staged in beautiful galleries. In this utopian, very well-lit vision, humans get a chance to start all over and do things right this time. This new world is organized based on artistic groupings, not brawn and wealth. This is the anti-Wells worldview, where science and machinery are of little significance and a true age of re-naissance ā a rebirth ā is manifested.
In 1905, the story was translated and published in English, titled The Underground Man.
Ironically, it was introduced by none other than the Master of Dystopian fiction himself, H.G. Wells.
But what does not get mentioned so much are two facts:
The computer was the faceless villain of the movie. It did Bad Things, like trying to kill all the humans. This narrative device violated author Isaac Asimovās 1942 Three Laws of Robotics, which specifically forbade harming humans through action or inaction. But HALās hubris gave it permission to rationalize human sacrifice. It was precisely what propelled the story and made it more realistic to viewers.
HAL controlled all aspects of life on the spaceship, much as The Machine controlled Forsterās society underground. But HAL had a personality, a soothing voice, demonstrated emotion, and had a singular task to complete its mission at any cost. This goal gave it license to kill its human charges to avoid failure. More importantly, HAL specifically notes its own dependability and accuracy. From the movie transcript:
Amor [BBC Interviewer]: The sixth member of the Discovery crew was not concerned about the problems of hibernation for he was the latest result in machine intelligence: the H.A.L. nine-thousand computer which can reproduce - though some experts still prefer to use the word āmimicā - most of the activities of the human brain and with incalculably greater speed and reliability. We next spoke with the H.A.L nine-thousand computer whom we learned one addresses as Hal. Good afternoon, Hal. Howās everything going?
Hal: Good afternoon, Mr. Amor. Everything is going extremely well.
Amor: Hal, you have an enormous responsibility on this mission. In many ways perhaps the greatest responsibility of any single mission element. You are the brain and central nervous system of the ship and your responsibilities include watching over the men in hibernation. Does this ever cause you any lack of confidence?
Hal: Let me put it this way, Mr. Amor. The nine-thousand series is the most reliable computer ever made. No nine-thousand computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.
This attribute is practically forgotten in this day and age of hallucination and variability in responses of Large Language Models. HAL, for all its psychopathic shortcomings, was accurate. It took a dismantling of its underlying hardware to put a stop to it.
During the development of the script, Kubrick and Clarke consulted leading researchers, including the founder and head of MITās AI Lab, Marvin Minsky, to make sure their predictions remained within the realm of the possible.
This was the same Marvin Minsky who went on to predict that General Machine Intelligence was a mere few years away in a 1970 Life Magazine interview:
Marvin Minsky of MITās Project Mac, a 42-year-old polymath who has made major contributions to Artificial Intelligence, recently told me with quiet certitude: āIn from three to eight years we will have a machine with the general intelligence of an average human being. I mean a machine that will be able to read Shakespeare, grease a car, play office politics, tell a joke, have a fight. At that point the machine will begin to educate itself with fantastic speed. In a few months it will be at genius level and a few months after that its powers will be incalculable.ā
In more recent estimates experts hedge their bets, but (as we cover in a later section) the art of moving goalposts is alive and well.
This is a series about AI Companions, an incarnation of the field of Artificial Intelligence that directly interacts with humans. As of this writing in 2025, AI Companions manifest themselves, not as a singular Machine, but in a variety of forms:
This series looks at the history, philosophy, and technology behind these services and products. But it doesnāt just stop at cataloging. We also peek under the hood, at the underlying technologies, how they work, and how we can help direct them into a Humanistic AI with beneficial uses, designed to advance human agency, privacy, and capability.
Thereās a good chance youāve never heard of me, but Iāve been around, quietly building things for a long time.
This series came out of several decades of working, watching, and thinking about what John McCarthy quaintly called Artificial Intelligence.
You should know: I am not a computer researcher. Iām a professional developer and software architect, a learner, teacher, and builder.
Technology is a magnificent tool for expanding and helping humanity. Unfortunately, it can also cause grievous harm, especially at scale. Weāll get to all that later.
My work has been focused on pragmaticsāthe nuts and bolts of application development. But my background and interests have always been in the humanities, especially literature, philosophy, and storytelling, with a focus on understanding where ideas fit. Artificial Intelligence in the mid-80s was a combination of all these, synthesized together. It was the original STEAM before anyone called it that.
By the late 1980s, things quickly devolved. We hit what came to be known as the Second AI Winter and all funding dried up. I moved on to working on Industrial and Consumer products: the Internet, Man-Machine Interfaces, and application development. I was lucky to be working in the San Francisco Bay Area and Silicon Valley during those exciting times, feeling like Zelig at the periphery of historical events.
Somewhere along the line, I ended up building a couple of browsers, moved into the world of mobile applications, and then connected hardware. Ten years ago, I fell back onto AI Assistants and their second cousin, wearables. I bring this up because I believe edge computing and smart hardware are the only ways to solve intractable problems in privacy, cost, and genuinely personal Humanist AI.
Weāll get to all that.
Much of my work has been as a consultant around Silicon Valley, hopping from project to project. This allowed exposure to a wide range of technologies, everything from the web front-end to mobile, server back-end, serverless cloud, embedded firmware, and robotics, with a dose of soldering and bodge-wiring thrown in.
A few times, I tried my hand at founding companies (with mixed results). Iāve helped build and ship apps that have been featured on every mainstream app store (Mac, Windows, iOS, Android, and Apple Watch).
In 2019, I ended up full-time at Amazonās Lab126, working on Alexa devices and poring through every line of its code. After that, I moved to AWS R&D, working at the intersection of devices, user interfaces, and large-scale architectures. Later, at Panasonic R&D, I did a lot of thinking about where AI Assistants could go next, starting with helping the elderly and people with different abilities, and ended up filing a few related patents.
My conclusion: Think of users first.
This might seem obvious, but if you look at many products, their primary benefits redound to some other entity. Itās easy to lose sight of benefiting users and end up in a state of what Cory Doctorow calls enshittification.
Iāve been through multiple Silicon Valley crashes and at least one AI Winter. I hope this current bubble doesnāt end up in a bad place. At some point, expectations and reality will hopefully sync up, and we will end up with something useful that makes life easier for everyone. Itās happened before, and it will happen again. Like Fichteās Dialectic: Thesis, Antithesis, and finally, Synthesis. Until thenā¦
I started writing this series because a loud nagging voice inside me kept yelling that I should, until I could no longer ignore it.
In any other knowledge domain, we would be learning from the hard-earned lessons of those who came before us, rather than hitting reset and starting all over every 5-10 years, or hype cycle to hype cycle.
In bridge construction, thereās an accumulated knowledge of how to build a decent, functional bridge. More importantly, humanity has learned what NOT to do so these structures donāt come tumbling down. That is how we truly advance, by remaining curious and showing humility.
Same with medicine, automotive, farming, or psychology. People learn and build on the mistakes and lessons of previous generations. We would do well in the software world to do the same.
The world of AI didnāt materialize because GenAI showed up. ELIZA did it back in the 1960s. It just didnāt do it as well and used different plumbing, appropriate to the period. The impact ELIZA had on the direction of research was significant (despite the creatorās misgivingsā weāll cover all that in the next section). The enthusiasm led to promises of General Intelligence ājust a few years from now.ā That legacy continues nearly 60 years later, guised as AGI, or Personal SuperIntelligence.
Sound familiar? None of this is new.
My goal for this series is to pitch the technical content at an abstraction level between pure theory and low-level hardware, hopefully keeping it accessible to a broader audience. Think of it like this:
Occasionally, weāll dip into the lower layers, but I will do my best to provide context on where they fit.
When it comes to AI, I am neither a fist-shaking doomer, nor a starry-eyed Pangloss. I am what you might call a Pragmatic Optimist, and very much aligned with E.M. Forsterās notion of Humanism.
I believe it is more useful to create long-lasting technical architectures with purpose. These should remain stable, regardless of the latest fads, but more importantly, build toward an agreed, common goal.
Think of me as a nobody with a lot of ideas. By the end of this series, my goal is to show how the next version of AI can truly help humanity, instead of plunging headfirst into a future not unlike The Machine.
This is my manifesto for a Humanist AI and eventually, an alternative to the Turing Test.
I thank you for your attention and invite you to stick around for the ride. It should be fun.
Every word in this series was conceived and written by yours truly, without the aid of any Artificial Intelligence. Itās all organic, artisanal, brain-pan juiceāmistakes, em-dashes, and all.
I do use a commercial grammar-checking service that claims to use AI. I often take its punctuation and spelling hints, but tell it to f-off when it makes a stupidly bad recommendation ā which is surprisingly often.
Said commercial grammar-checker doesnāt know that it shouldnāt rewrite attributable quotes, especially in Markdown. I filed a complaint with tech support but got a generic Weāll forward this to our product team response. Couldāve been a chatbot. Hard to tell. Iām writing this because if they donāt fix it soon, Iāll start naming names and will not hesitate to make a free, open-source version of their service.
I carefully select and use photographs from stock image sites taken by human photographers (properly credited). However, I have no way to tell whether THOSE images were touched by AI. I hope not, but nowadays, itās hard to tell.
For memes or GIFs, itās challenging to locate original sources and provide credit. I apologize to the creators. Please drop me a note, and Iām happy to correct this omission or take them out and replace them with bland LLM-generated content.
All technical diagrams, unless otherwise credited, were hand-made by me using illustration and presentation tools. If you want to use them, I would appreciate attribution. If you need high-res sources, feel free to drop me a note.
Highlightercolors are added by me to emphasizesomething. There is no significance to the actual color or order.
The commercial blog template used on this website has been augmented via vibe-coding to add different styles, tags, and templates. Tweaking CSS and HTML is something AI agents can perform well when properly constrained.
Iām personally drawn to stories of people. Occasionally, I will inject a bit of my background and history into side notes.
In a few places (including those in the StoryFAQ section) the images were created by OpenAIās ChatGPT-4o, using my own prompts. These will be marked appropriately to avoid confusion.
My thanks to the readers and reviewers (in alphabetic order): Adam R, Bea K, Bill P, Bruce I, Chris K, Christopher S, Elizabeth K, Hamid F, Heidi R, Ian L, Ian V, Larry L, Luke F, Mark C, Mike L, Mitch R, Nick F, Omid A, Paul V, Rob M, Seyed M, Shadi S, Sohrab G, and Vasavi P.
AI is a fast-moving field, and more products, features, and revisions will have appeared or evolved by the time this is published. I will try to keep these posts up to date as best I can. If there are errors and omissions, please contact me through Mastodon, Bluesky, or LinkedIn.
Forster and Tardeās stories began with the premise of a world at the edge of environmental ruin. It was an artistic storytelling propellant. A cautionary tale that humanity would never reach that point.
Arthur C Clarkeās oeuvre almost universally covered space travel, the direct opposite of Forsterās underground dwellers. But the vision of a machine that fully managed human life and controlled life and death remained the same.
In Forsterās story, The Machine begins to fail and humanity lies dying. The protagonist asks her son:
āBut Kuno, is it true? Are there still men on the surface of the earth? Is this ā this tunnel, this poisoned darkness ā really not the end?ā
He replied:
āI have seen them, spoken to them, loved them. They are hiding in the mist and the ferns until our civilization stops. To-day they are the Homeless ā to-morrowāā
āOh, to-morrow ā some fool will start the Machine again, to-morrow.ā
āNever,ā said Kuno, ānever. Humanity has learnt its lesson.ā
Forster tried to warn us. Let us see if Humanity has learnt its lesson.
