Category Archives: Forth

The picoXpert Story

picoXpert was one of the first (if not THE first) handheld artificial intelligence (AI) tools ever. It provided for the capture of human expert knowledge and later access to that knowledge by general users. It was a simplistic, yet portable implementation of an Expert System Shell. Here is the brief story of how it came to be.

When I was about 10, my grandfather (an accomplished machinist in his day) gave me his slide rule. It was a professional grade, handheld device that quickly performed basic calculations using several etched numeric scales with a central slider. I was immediately captivated by its near-magical power.

In high school, I received an early 4-function pocket calculator as a gift. Such devices were often called ‘electronic slide rules’. It was heavy, slow, and sucked battery power voraciously. I spent many long hours mesmerized by its operation. I scraped my pennies together to try to keep up with ever newer and more capable calculators, finally obtaining an early programmable model in 1977. Handheld machines that ‘think’ were now my obsession.

I read and watched many science fiction stories, and the ones that most fired my imagination were those that involved some sort of
portable computation device.

By 1980, I was building and programming personal computers. These were assembled on an open board, using either soldering or wire wrap to surround an 8-bit microprocessor with support components. I always sought those chips with orthogonality in memory and register architecture. They offered the most promise for the unfettered fields on which contemporary AI languages roamed. I liked the COSMAC 1802 for this reason. It had 5,000 transistors; modern processors have several billion. The biggest, baddest, orthogonal processor was the 16- or 32-bit Motorola 68000, but it was too new and expensive, so I used its little brother, the 6809, which was an 8-bit chip that looked similar to a 68000 to the programmer.

I spent much of the 1980s canoeing in Muskoka and Northern Ontario, with a Tandy Model 100 notebook, a primitive solar charger, and paperback editions of Asimov’s “Foundation” trilogy and sequels on-board (I read them five times).

By the mid 1990s, Jeff Hawkins had created the Palmtm handheld computer. The processor he chose was a tiny, cheap version of the 68000 called the ‘DragonBall’. I don’t know which I found more compelling – this little wonder or the fact that it was designed by a neuroscientist. I finally had in my hand a device with the speed, memory, and portability to fulfill my AI dreams.

The 1990s saw the death of Isaac Asimov (one of my greatest heroes), but also saw me finally gaining enough software skills to implement a few Palm designs. These were mainly created in Forth and Prolog.  The Mars Pathfinder lander in 1997 was based on the same 80C85 microprocessor used in the Tandy Model 100 that I had used years earlier. This fact warmed my heart.

In 2001, I formed Picodoc Corporation, and released picoXpert.

Here are: the original brochure, a primer on Expert Systems, and a few slides.

It met with initial enthusiasm by a few, such as this review:

Handheld Computing Mobility
Jan/Feb 2003  p. 51
picoXpert  Problem-solving in the palm of your hand
by David Haskin

However, the time for handheld AI had not yet come. After a couple of years of trying to penetrate the market, I moved on to other endeavours. These included more advanced AI such as Neural Networks and Agent-Based Models. In 2011, I wrote Future Psychohistory to explore Asimov’s greatest idea in the context of modern computation.

Picodoc Corporation still exists, although it has been dormant for many years. It’s encouraging to see the current explosion of interest in AI, especially the burgeoning Canadian AI scene. For those like me, who have been working away in near anonymity for decades, it’s a time of great excitement and hope. Today, I’m mainly into computational citizen science, and advanced technologies, such as blockchain, that might be applied to it.

Forth: A Syntonic Language

Educators sometimes hold up an ideal of knowledge as having the kind of coherence defined by formal logic. But these ideals bear little resemblance to the way in which most people experience themselves. The subjective experience of knowledge is more similar to the chaos and controversy of competing agents than to the certitude and orderliness of p’s implying q’s. The discrepancy between our experience of ourselves and our idealizations of knowledge has an effect: It intimidates us, it lessens the sense of our own competence, and it leads us into counterproductive strategies for learning and thinking.¹
– Seymour Papert

People are sentient and we know that we are sentient. We are also social creatures. The human mind is aware of ‘agency’, in both itself and others. Since the beginning of recorded history, and probably long before, we have even ascribed agency to objects and events in the natural world. Once we attain consciousness, it becomes powerful and efficient to reuse this mind machinery to understand the world around us in familiar terms. Mirror neurons in the human brain enable us to read or infer intention in others. They do not merely form patterns of thought – they reflect them.

Syntonic is a word sometimes found in music theory, but in psychology, syntonicity means having a strong emotional connection with the environment. It is understanding something through identification with it. It is achieved by putting oneself “in another’s shoes”, and it is key to human learning. One form of this is imitation, and children do this all the time, with everything from trees and animals to teapots and airplanes. Another form is metaphor, where an established understanding is substituted for a new and unfamiliar thing. See Ancient Metaphors for examples from the IT world.

When we mature, many of us discover computer programming and would like to use the same built-in learning method that allowed us to develop ‘common sense’. However, most programming languages and their text books are so abstract, formal, and doctrinal, that it’s nigh-on impossible to get very far this way. As a result, computational thinking recedes off into the ivory tower, and the world is much worse off because of it.

This type of learning is well suited to therapeutic applications due to its natural, informal flow. Starting with the ‘hard wired’ facilities we all have, ‘bootstrapping’ (discussed below) is a very individual and non-doctrinal process.

An interesting and more in-depth exploration of syntonicity in programming languages was written by Stuart Watt.²

 

Forth words

I have found Forth to be a very syntonic language.

Syntax and structure are the first face that we see of a new language. Some are fairly natural (e.g. BASIC), most are verbose and intricate, and some are at the extreme end of mathematical formality (e.g. Haskell). Forth is positively Spartan by comparison, with the majority of keywords being three letters or less, and the ‘: … ;’ pair providing most of the structure and context. Some long time users of Forth do not even like its syntax and quickly override keywords (a trivial task in Forth), inventing their own to craft the language to be more in tune with their way of thinking. Ironically, this makes them love Forth more, not less. It is an extremely simple language. At its core, Forth is just a mechanism to reduce the cost of calling a subroutine to merely referring to it by name. But this is not what makes Forth syntonic.

“syntonicity is not directly a property of the syntax, or even the semantics or pragmatics of a language, but a property of the stories that we tell about it” (Watt p.5)

Forth is a stack-based, threaded, concatenative language with a tightly coupled interpreter and compiler. To explain these terms, and thus ‘tell a story of Forth’, we’ll use a metaphor:
Forth – a hiker (named ‘Arkady’ for convenience)
Stack – her backpack
Thread – the trail
Interpreter – her mind (neurology)
Compiler – her logbook

Notice that there is no clear distinction between traveler (Forth) and journey (program). Arkady’s journey is of more interest to her than is her destination. She is open to distraction, side-trips, and unexpected eventualities. Even mistakes are sometimes learning opportunities. This is the way of Forth – less focus on design and protocol, more on exploration, discovery, and emergence.

Life is not a problem to be solved, but a reality to be experienced.
– Soren Kierkegaard

While most languages are applicative (functions are applied to data), Forth is concatenative, which is a much more natural and emergent process. There are also few safeguards in Forth; the entire environment (machine) is wide open. This freedom is anathema to authorities of language design, which might partly explain why Forth-like languages are not more widespread. It’s much safer to clone an application from an existing template and/or framework than to invent a new one. Of course, the result is a copy, not an original. Incidentally, in much research funding, the outcome must almost be predicted along with careful metrics and timetables (ask anyone who has prepared a grant application). This squelches creativity and serendipity. It also puts a premium on positive results over negative (and perhaps greatly informative) ones. In fairness, just so we don’t make the mistake of assuming that everyone agrees that doctrine is a bad thing, there has been a large and successful effort to come up with a more standardized, ANS Forth.

Her backpack (the stack) is a readily available, last in, first out (LIFO) pile of items that she adds or removes as required. As a good hiker, Arkady is always aware of what’s in her backpack. She keeps often-used items (tools, map, compass) near the top. Longer term necessities (food, tent) are towards the bottom.

She walks along the trail (thread), step-over-step. Some steps are short and obvious, like avoiding big rocks and snakes. Some are more complex and subtle, such as crossing streams and staying downwind of bears. Many are actually composites of smaller steps. Some require more items from the backpack than others. At each step, she has the same backpack, although the exact contents may vary. She may sometimes just take the top few items, drop the pack, and head off down an interesting side trail. This is how a Forth program is built up. New ‘words’ are written to accomplish tasks. These words are then threaded together as higher level words invoke a series of them. Eventually, a single, highest level word is the starting point for the whole program, which is just a chain of subroutine calls.

Using all of her neurology, from senses to brain to common sense, logic, and rationality, she ‘interprets’ her world. She learns new facts and skills as she walks, gradually ‘bootstrapping’ a deeper understanding of her environment and more powerful and efficient use of resources (food, energy, time). To learn if and how something works, she prods it and observes the results. If she finds an unfamiliar mineral, she could bring whatever tools she is carrying to bear. Or, she could invent an entirely new tool. She could craft a crude microscope from the magnifying lenses and rolled up map in her backpack. If she has time, a mass spectrometer may be suitable. In Forth, quick tests and trials are inexpensive and easy. There is no odious edit-compile-run loop to get in the way. For example, to find what OVER does, put a few numbers on the stack, type OVER, and examine the stack to see the result. Stumbling around with her nose in manuals and books would perhaps cause Arkady to miss a fossil or patch of berries. It is also a good way for her to become lunch for a mountain lion.

A word about bootstrapping. Sometimes, when a skill is applied to produce a result, new knowledge (or a new tool) is gained in the process. This new knowledge opens the door to skill-set improvement or refinement, which enables hitherto impractical or unseen possibilities. In turn, even more new knowledge is obtainable, and the process repeats. The result is flint knife to bow and arrow to alchemy to chemistry to electronics to spaceflight. Bootstrapping is the essence of Forth. Sadly, multitasking, object-orientation, dissociation and data-hiding, and that ultimate chestnut, ‘leveraging’ are all much more in vogue today. Multitasking in human thinking is greatly overrated. Deeper, syntonic thinking enables more creativity and innovation. If Forth was more widely used and understood, you might well be reading this on Mars right now. If it had been invented earlier, you might be reading this on Triton.

From time to time, Arkady considers new observations, knowledge, thoughts, and ‘steps’ to be well-established and important enough to jot down in her logbook. This log serves as a permanent record of all she has learned on her journey. It is in fact, a purified, corrected, ‘frozen concentrate’ of her trail.

Arkady eventually arrives at the end of her journey. It may have been cut short for various reasons or maybe her destination is different (maybe even better) than she originally intended. This is a function of her curiosity as much as weather or circumstance. In any case, she still has her logbook, for her or others to use as a future guide (be they human or machine).

Some would say that it’s wrong to argue for Forth going forward. After all, it’s an old language, and was originally created for control of machinery in an era of sparse computational resources. As programs and systems grow ever-more complex, capable, and intelligent, such a language has outlived its usefulness. However, I must disagree. The future is not just about frameworks, big data, and augmented reality. It will be at least as important to build new, ad hoc, ‘micro’ systems rapidly and locally in a time of dizzying acceleration of technology. Creating, testing, and problem solving starting from first principles will always be valuable.

Understanding the world requires thinking in new languages. Forth is to computer science what math is to physics. Computational thinking, syntonicity, learning-by-doing*, and good old human common sense are not headed for the dust bin of history any time soon. Here are some further thoughts on learning.

 

Greek Pattern 1

 

 

* I am, however, perhaps not a very good constructionist, try as I might.
I once tried (unsuccessfully) to acquire a minor Logo implementation.
I believe in learning-by-doing something else !  A wide, diverse search is
often the most efficient (just ask the ants).

(1) Papert, S. (1980). Mindstorms: Children, Computers, and Powerful Ideas. New York: Basic Books
(2) Watt, S. (1998). Syntonicity and the psychology of programming – Psychology of Programming Interest Group 10th Annual Workshop.

Concatenative Biology

There are several ways to categorize programming languages. One is to distinguish between applicative and concatenative evaluation. Most languages are applicative – functions are applied to data. In contrast, a concatenative language moves a single store of data or knowledge along a ‘pipeline’ with a sequence of functions each operating on the store in turn. The output of one function is the input of the next function, in a ‘threaded’ fashion. This sequence of functions is the program. Forth is an example of a concatenative language, with the stack serving as the data store that is passed along the thread of functions (‘words’). “Forth is a simple, natural computer language” – Charles Moore, inventor of Forth.

One of the great advantages of concatenative languages is the opportunity for extreme simplicity. Since each function really only needs to know about its own input, machinery, and output, there is a greatly reduced need for overall architecture. The big picture, or state, of the entire program is neither guiding nor informing each step. As long as a function can read, compute, and write, it can be an entity unto itself, with little compliance or doctrine to worry about. In fact, in Forth, beyond the stack and the threaded execution model, there’s precious little doctrine anyway!  Program structure is a simple sequence, with new words appended to the list (concatenated). The task of the programmer is just to get each word right, then move on to the next.

In nature, the concatenative approach is the only game in town. Small genetic changes occur due to several causes, random mutation being one of them. Each change is then put through the survivability sieve of natural selection, with large changes accumulating over large time scales (evolution). (Evolution is active across the entire spectrum of abstraction levels. Hierarchies emerge naturally, not through doctrine or design.)  Concatenation is the way by which these small changes are accumulated. Much of the epic and winding road of human evolution is recorded in our DNA, which is billions of letters long.

Concatenative DNA

This process can be seen happening right now in molecular biology. Consider the ribosome. This is the little machine inside a cell that reads a piece of RNA (a chain of nucleotides) and translates it into a protein (a chain of amino acids). There is no Master Control Program assigning names, delegating work, and applying functions. There is only a concatenative program, built up over the ages by evolution. So, basic life uses a fairly powerful and successful form of computation: DNA for storage, RNA for code, ribosome for computing, protein for application.
(and natural selection for testing) 🙂

We flatter ourselves when we talk of our ‘invention’ of levers, gears, factories, and computers. Nature had all that stuff and much more long before we ever came down from the trees. Math, engineering, and science are great not because of their products, but rather because they enable 3-pound hominid brains to explore nature and ponder the possibilities.

Forth – not quite dead

In my previous post, Thinking in Forth, I said that Forth is a dead language. I did this to make two points. The first is that I don’t have to revive or advocate a dead language, hurray, because that’s boring. The second point is that even a dead language still has value, like Latin does. However, just because I consider it to be a dead language doesn’t mean that everyone else does too.

Forth words

Forth is an excellent choice for education for several reasons:

  • it’s small, maybe 25 required core words
  • most of these words are one, two, or three letters long, so familiarity with English is not a prerequisite for learning Forth
  • it’s simple with no built-in abstractions – source code maps directly to assembly code (a Forth word is just a list of calls to previously defined words)
  • it has little syntax to learn
  • there is only one data type in basic Forth – integers (called ‘cells’)
  • useful programs can be written early in the learning process
    (a Forth program is just a list of one or more Forth words)
  • it’s very ‘bootstrap-able’ – one learns by concatenation, not starkly delineated ‘levels’
  • paradigms such as object-oriented and functional programming are consistent with Forth (and in fact many of them have often already been very well implemented and documented)
  • concatenative programming is much more pure and natural
    See Threaded Nature
  • syntonic programming is easier learn and use
    See Forth: A Syntonic Language
  • Forth is useful for building models and simulations
  • open and non-proprietary versions of Forth are freely available for a wide range of platforms

There are several efforts at delivering basic, classic Forth to beginning programmers. One that I particularly like is 4E4th which runs on a tiny, inexpensive protoyping card.
The Raspberry Pi can easily run a bigger, more capable Forth.

Several commercial enterprises continue to promote Forth. Some can be found towards the end of the excellent and extensive Forth page on Wikipedia

The creator of Forth, Chuck Moore, is a tireless, brilliant, and dynamic force. He has never stopped developing cutting edge technology and innovation based on Forth. For a look at some of his latest work, here’s a talk he gave in September 2013 on the ga144 chip. It has 144 cores, with energy measured in picoJoules per instruction and power in nanoWatts per core, and is of course programmed in Forth. That’s hardly dead technology.

So, while I do not aspire to revive Forth or advocate it generally, I may have been in error in describing it as dead. I come to praise Forth, not to bury it.

Thinking in Forth

I consider Forth to be a dead language, at least in the same sense that Latin is a dead language – i.e. nobody uses it in daily life anymore. I now only use it and write about it in the context of computational thinking. Forth may be a dead language, but that doesn’t preclude it from being vibrantly important. If you don’t like irony, you’re reading the wrong blog.

Forth was developed for over a decade by Charles (Chuck) Moore before being written down formally in 1970. Moore was a radio telescope guy at the start. He needed an expressive, helpful computer language for his own work, much as Newton needed calculus. The story is best told in his own words http://www.colorforth.com/HOPL.html

What makes Forth important is just that – it was created to serve as a computational assistant. Using it is very conversational and immersive. One doesn’t think for an hour, consult authority and standards, edit for an hour, then compile, then test, then go for lunch (or go home), etc. There is a non-stop dialectic, a friendly and thoughtful argument, with you handling the imagination and it handling the bits and bytes.
And a program emerges.

Forth words

I started using Forth in the late 1970s. Using it, I learned binary and hexadecimal numbers and arithmetic, register manipulation (the main task of a CPU), memory manipulation (the main task of a computer), and many concepts and paradigms of programming. These included imperative (states, flow control), functional (stateless, mathematical), object oriented (encapsulation, re-use), factoring, concatenative, declarative (mostly through a Prolog written in Forth), machine control, etc, etc,. I read several books and many articles, but the real learning happened by doing. What’s key here is the fact that I’ve never been a great programmer (very good, but never great). Anyone can gain tremendous benefit from spending even a few hours learning Forth. It enables one to invent an ad hoc language to solve an ad hoc problem. It’s not magic and other worldly like Excalibur. It’s more like an early hominid’s stone tool – it teaches one to think, especially to think about the next tool.

My Forth days are long in the past. Today, I write mainly in Haskell, Assembler, Prolog, and English. For database work, I use SQL, key-value stores, and JSON. Whenever I think about any problem, I think computationally. I’ve spent almost four decades thinking in Forth.

I intend to write many posts and maybe even a few articles on Forth, it will be a mainstay of this blog.