GOFAI for Game Dev

Artificial Intelligence & AI & Machine Learning - 30212411048


Neural Networks, Machine Learning (ML), Deep Learning, … – that’s modern AI. These methods have produced spectacular advances in vision, natural language, pattern recognition, and many other areas. Entire academic, corporate, and national efforts have sprung up to join the race to avoid any ‘ML Gap’. In fact, the terms ML and AI have now commonly become synonymous.

A related hardware trend has taken the computing industry by storm: Massively Parallel Processing (MPP). ML, specifically the training of neural networks, greatly benefits from such hardware acceleration. This wave began with the ubiquitous Graphics Processing Unit (GPU), then expanding into devices such as the Associative Processing Unit (APU), Tensor Processing Unit (TPU), Vision Processing Unit (VPU), and even Field Programmable Gate Array (FPGA). Even tiny chips that perform dedicated monitoring and automation tasks (edge computing) are now integrating such MPP technology.

Of course, game designers have been scrambling to enhance and augment their offerings by incorporating such methods. Mainly in the realm of AI players, to better reflect human strategic thinking. Chess, Go, general knowledge, and many other specific machine players have now achieved supremacy over human players. Playing some modern games against one or more AI players is almost a ‘social’ experience.

It wasn’t always thus.

Back in the single-threaded days, those halcyon days of early PC gaming, AI had a much wider horizon. Symbolic themes such as logic, optimization, semantics, rules, expert systems, goals, and graphical knowledge representation dominated. Steps and decisions were the basis of design, and often some sort of ‘theory of mind’ existed, even if very simplistic. Logic and knowledge representation tools sprouted, such as Lisp, Prolog, and several Production Systems. Navigating decision trees, usually via some form of context awareness and backtracking capability was a process that was reassessed at each step. This was great for automating simple, explicit thought processes and playing simple games (tic-tac-toe, checkers, adventure, cards, etc). However, it fell short when confronted with the much harder areas of vision and common sense. Good Old Fashioned AI (GOFAI) went the way of the buggy whip. The lull that persisted from the late 1980s through the early 1990s is an example of an ‘AI Winter’.

Aletschgletscher-Eggishorn



But perhaps that total abandonment was a mistake. It’s easy to rush right past available treasures when blinded by shiny new objects.

Modern games are certainly beautiful to look at, with amazing realism and depth, but are they more meaningful? more playable? more fun?? Gameplay is about much more than realistic perception and rendering. This is especially true for Game Based Learning (GBL). In GBL, the firing of neurons in the player’s brain is the goal, much more than the passive presentation of eye candy.

The Human Computer Interface (HCI) has actually progressed little over the decades. Instead of the game living mostly in the player’s mind, as it did in the primitive arcade days, it is now presented almost as a movie, with entire scenes played out for them. Much of the overall experience can be obtained just by watching game demos. The most compelling feature of modern games is their community… of human players!

What if gameplay itself was the focus? Exploiting and augmenting the innate power of human intelligence. HCI enhancements could once again move the game into the player’s mind, where it belongs. Instead of pre-fab, frustrating, stultifying scenarios, the player engages with a snappy, assistive, powerful, and ‘syntonic’ game. Simple rules could be explicitly available or even player-created. The great fun of Minecraft is largely due to the process of learning and mastering such rules, which are presented in a progressive, bootstrapping way. Instead of hiding the underlying mechanisms in order to present slick, canned movies, the player could be aware of how and why things actually work. The best combat games are those that enable the player to construct their own custom devices. Sadly, this is always achieved with a rigid menu system instead of a pencil-and-paper approach. Again, the player’s own creativity is stifled or ignored. Fairly complex worlds, even player-modifiable ones, can be created in Prolog. Explicit rules and facts are both computer and human readable. Transparency and openness are the keys to truly immersive games – pulling back the Oz curtain is a good thing.


Automation is another way to greatly enhance the playing experience. Sometimes the player is forced to repeat a series of steps even though they only wish to make one or two minor changes from their last go through. Wasting time is not fun. Giving the player access to such a series to make minor tweaks would be a great time saver, which would allow the player to focus on strategy, not mundane tactics or plain drudgery. And a tiny sprinkling of assistive AI to avoid stupid mistakes could enhance gameplay. Players don’t mind AIs ‘cheating’ so much as they mind pointless glitches and ‘gotchas’. Most decent chess AIs have had this feature for decades. It’s surprising that many modern games don’t even have a simple ‘undo stack’, instead relying on frequent game saves. The best games also have an ‘advisor’ system that helps the player with decision-making in the current context, with the option to squelch advice and tips as the player gains experience. The focus should be not only on rich content, but also rich context.

There is another area where automation is valuable: testing. By providing for massive machine-play error checking and discovery of gameplay glitches and bottlenecks, Automated Testing can greatly speed and enhance development. This can include an array of techniques, from short scripts to sets of rules and goals to full-blown simulations of human players.


But here’s the biggest nugget for GOFAI in games: Expert Systems. Capturing well-structure semantic knowledge is a perfect fit for strategy games. Many simple ‘How-To’ experts that encapsulate human knowledge can be easily created, rapidly consulted, and efficiently stored without any massive frameworks or ML tools. The skill of AIs can be enhanced merely by ‘learning’ from human players. Note that this is far, far simpler than actual Machine Learning. Most rule knowledge bases take up kilobytes of RAM, not gigabytes of code and storage. A packaging system would allow players to share (or even purchase) expert systems from others. Both gameplay and AI skill could be upgraded and personalized. This brings in a robust educational component, again, like Minecraft does. Human readability is key. We all have a basic comprehension of facts, rules, and inference. There’s no daunting learning curve required. Why not incorporate this innate comprehension into games?

In summary, gameplay has become too enamored of shiny bobbles and photo-realism. The current wave of Machine Learning is leading to evermore complex and power-hungry requirements, a burden on both developers and players. GOFAI could mitigate things rapidly and easily, and shift the game experience from the eyes to the mind’s eye. Game dev expands beyond the studio, into the laboratory. Let the fun begin.

Minecraft and AI: Project Malmo

In a previous post, I complimented Microsoft on their purchase of Minecraft. I ruminated on the potential for STEM and experiential learning it opens up, particularly with the addition of HoloLens and augmented reality. Recently, Microsoft announced the public availability of their Project Malmo platform that uses Minecraft to provide an interactive environment for development and testing of intelligent agents. This further illuminates Microsoft’s long-term plans for Minecraft.

In contrast to highly specific, applied AI, Project Malmo harnesses the almost unlimited exploration and experimentation possibilities of Minecraft as a research tool for general artificial intelligence. Agents can learn, comprehend, communicate, and do tasks in the Minecraft world. A mod for Minecraft provides an API for programming, and uses XML and JSON to hold representations of the world in computer memory. Agents can explore their surroundings, see and touch Minecraft blocks, take actions, and receive feedback from the environment. This enables reinforcement learning. Instead of just applying deductive, symbolic reasoning, agents can benefit from inductive (experiential) learning.

The potential benefits are compelling.

Sandbox development allows strategies and algorithms developed in the Minecraft world to be later moved to a much larger simulation environment, possibly on distributed systems and/or supercomputers. Computational agents do not require sensory augmentation or biological interfaces, since they connect ‘directly’ to the simulated world. The ability to ‘overclock’ the simulation frees agents from the limits of our time scale (they can ‘live’ many days in an hour of real time).

For remote sensing applications, such as planetary probes & rovers, the benefits are huge. Autonomous machines must be developed and tested before they are deployed. The time delay incurred by vast distance precludes ground-based control. By the time people on Earth receive video from Mars and send the command, “don’t drive off that cliff”, it’s far too late. The ‘smarts’ to navigate and make decisions locally must exist in the robot. Hazardous locations, even here on Earth, also require considerable autonomous learning and decision-making.

Collaboration and comparison between people and teams is possible with a common testing ground. Minecraft can be played as an individual, but the real power is that a Minecraft world can be hosted on a server, enabling many people, wherever they are, to experience and simultaneously participate in that Minecraft world. Players, agents, landscapes & artifacts, or even a combination can all exist and interact. This is much more like the natural world.

Project Malmo encourages public participation and involvement. It’s an open architecture that invites experimentation by all. This is somewhat similar to the citizen science movement, which not only provides the benefits of ‘crowdsourcing’ to scientific research, but also enhances public understanding of scientific methods. In the modern world, a pervasive, basic understanding of artificial intelligence would be of tremendous benefit.

The ability to go far beyond the development of a single agent also opens up the possibility for social simulations of almost limitless scale. Agent-based models have long shown their usefulness in the study of social interaction, from ant colonies to human nations and cultures. One example is the simulation of ancient civilizations. This could lead to an entirely new approach to deciphering Linear A, for example. I considered a much more vast model in Future Psychohistory. In 2011, when I wrote that article, I had a standard neural network model in mind, but if I were to write it today, I’d definitely go with intelligent agents.

The concept of agency is central in many areas of life science. Relational databases are far too static for life science – Cassandra for Life Science. The proper representation of life is not tabular, but associative. Dynamic systems interact, adapt, and even learn. This is the basis of evolution. Machinery as tiny as enzymes and ribosomes, and possibly even medical nano-robots in the near future, all require much more dynamic models to study.

Project Malmo offers major benefits for teaching computer programming. Different paradigms such as procedural, functional, object oriented, declarative, and concatenative are all suitable and helpful in the construction of agents. And of course, there’s one of my favourite hobby horses – constructionism. This only adds to the already expansive use of Minecraft in education.

And for game play itself, Project Malmo can be of great usefulness. Dangerous and tedious tasks can be handed off to agents, amplifying a human player’s efforts. Tutoring and nurturing AI agents is a good use of human intelligence — and it’s a lot of fun.


AI research and Minecraft now have a powerful and resourceful champion – nice. I look forward to seeing the fruits of Project Malmo.

Minecraft and STEM

Microsoft recently bought Mojang AB, the Swedish company that makes Minecraft, for a reported $2.5 billion. It was not Microsoft’s first attempt to woo Mojang, and many thought they would never be able to close the deal.

In 2009, Markus Persson (‘Notch’), an unknown independent Swedish game developer, released the first version of his creation, Minecraft. It was not a normal ‘game’. It had no defined sequence, no milestones or levels of achievement, and most curiously, no end goal. What it did have was a charming, simplistic world in which a player could ‘mine’ and ‘craft’ for hours (or days) (or weeks) according to their own desires. In fact, the game could be played without any other mobile agents (‘mobs’) at all, leaving an entire world to explore and terraform with no limits. In 2010, maps became truly infinite, growing automatically as the player’s avatar (‘Steve’) approached an edge. The fact that no one else ever has or ever will live in, or even see, your privately created world gives one a compelling, almost haunting feeling.

It reminds me of the 1967 Star Trek episode “Arena”. Captain Kirk is locked in mortal combat with a frightening alien creature on a deserted and barren world. He must find or craft weapons from whatever materials he can find. He finally succeeds by assembling the raw materials for a crude cannon. We have all imagined ourselves cast back to a more primitive environment, where our own imagination and gumption are nearly all we have at our disposal. Interestingly, the goal of many advanced Minecraft players is to populate their world with coffee shops, restaurants, and villages.

Over time, Minecraft has expanded to include many characters, tools, and landscape features, but it has always maintained its minimalist look and feel. That’s the secret to its success – this is a world that is understandable to everyone, from children right through to seniors who grew up before the computer age. While the richness and physics of this world have evolved, the quaint, blocky look of it hasn’t.

The game has since sold many millions of copies, inspired LEGO sets, books, films, and has a huge and active fan base on YouTube. Conventions are held regularly, and Notch achieved rock star status years ago. Minecraft is the very definition of viral Internet success, massively disrupting traditional corporate and marketing models. Without shrink wrap or advertisements, it took over the gaming world.

So, many folks were left scratching their heads over this Microsoft acquisition. On the surface (pardon the pun), it seems like an unlikely fit. However, I don’t think that at all. I see it as the happy marriage announcement of two dear old friends.

Around the turn of the millennium, I was a corporate trainer, with MCT & MCSD certifications (the ‘M’ is for Microsoft). I often found myself sticking up for Microsoft amidst the disdain of my colleagues. I would use Bill Gates’ early years as an example of out-of-the-box thinking, like the Tandy Model 100, the first true notebook, and the last machine that he programmed personally. The best Christmas present I ever got was DOS 3.0, and QuickBASIC was one of the best languages I ever used (and I’ve used 50 or more). Maybe Apple and others were more stylish, but it was Microsoft that really put the excitement into personal computing.

Microsoft hopes to gain leverage in the mobile and youth markets. I cannot think of any better vehicle than Minecraft. Far from seeing it as the death knell for Minecraft, I see Microsoft’s acquisition as a renaissance and a real chance for positive, imaginative, and compelling games to get the much larger piece of the pie that they deserve.

“If you talk about STEM education, the best way to introduce anyone to STEM or get their curiosity going on, it’s Minecraft”
– Microsoft CEO Satya Nadella

Also, there’s Microsoft’s augmented reality headset – HoloLens. There appears to be a plan coming together:
Minecraft + HoloLens + ElementaryProgrammingSkills = Syntonic Learning

If this is what Microsoft has in mind, it could be the most disruptive technology yet. Well done Redmond, and best wishes to Notch in all his future endeavors.

Morse Code and Minecraft

Several clever machinations have been implemented for the game Minecraft. These often include circuits and devices that mimic their real-world counterparts. They even include complete calculators and computer sub-systems (eg ECC memory and ALUs). Perhaps one of the more compelling inventions is the addition of Morse code operations. There are numerous implementations, all of which are interesting and a few are excellent.

I have a particular fondness for Morse code. See “Morse Code and Me”.

Often in the 18th and 19th centuries, ‘semaphores’ or codes were played with in an attempt to communicate over longer distances than the human voice could carry. With the rapid development of electricity and magnetism theory and practice in the early 1800s, the time was ripe for the telegraph. In 1837, Samuel Morse et al demonstrated a working telegraph setup in America (there were earlier versions in Europe). The principle was that codes tapped on a mechanical key would be converted to electrical signals, then sent along a wire to another key at the other end which would receive (repeat) the code. Since this transmission happened at almost the speed of light, communication at a distance became essentially instantaneous. Later refinements to and extensions of this principle led to the telephone, radio, television, and of course the Internet.

Why is Morse a good fit for Minecraft ? Basically, it is in keeping with Minecraft’s quaint and minimalist nature. Crafting rudimentary equipment and stringing telegraph wires matches this game (it’s simple and fun). Minecraft already has support for elementary circuits and power via “Redstone”. Morse code is at home in a rustic world of dirt, stone, and wood. You can’t get much more binary and ‘blocky’ than a string of dots and dashes. In contrast, popup chat windows are harshly anachronistic. Telegraphy also fits in well with Minecraft’s ‘bootstrapping’ model; you don’t absolutely require it, but it speeds gameplay and opens up new possibilities once the technology is achieved.

Educational use of Minecraft benefits greatly from this bootstrapping model. Finding, crafting, and mastering tools and gadgets, and thus learning as you go is half the story. Exploring endlessly rich and surprising 3D worlds is the other half.

Also, learning a language is never a bad thing, and Morse code is one of the simplest ‘languages’ there is. Telegraphy is laden with historical significance, another opportunity for learning.

What are the possible uses for telegraphy within Minecraft ? First, as I said, it’s a more primitive alternative to text chat windows in multi-player modes. Next, it would enable the transfer of information across distance, for example maybe a crude sort of ‘fax machine’ for maps, books, signs, etc. Next, automated beacons could be built, perhaps they might even show up on maps. Next, remote control of processes and apparatus would extend the player’s reach. Next, short messages could be written right into the landscape (for example dirt for dots and stone for dashes). How about time capsules or ‘notes to self’ ?

So, Morse code would fit very well into the Minecraft world and mindset, with many novel and fun uses. The educational benefits of this addition would be welcome and easily realized.