The kcats Programming Language

Table of Contents

kcats.png kcats-repl.gif

1. Summary

Kcats is a simple, powerful, beginner-friendly programming language.

2. Examples

2.1. Produce fibonacci sequence

Here we create a generator of the infinite fibonacci sequence, and collect the first 20 results.

[1 0                ;; start with the first two numbers as state
 [•👥 + 🔀 👥]      ;; add the two numbers but keep a copy of the 2nd one,
                    ;; then copy the result  
 🔳 🔳 [🔀] 🪆      ;; create a recurrence 
 20 taker           ;; limit the generator to 20 items
 collect]           ;; collect the results into a list
reap                ;; discard leftover state, keeping result
[1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765]

2.2. Sequence operations

Find highest priced of all products rated 4.5 or higher, and in stock

[[[name "Mouse"] [price 50] [rating 4.4] [inStock ✅]]
 [[name "Laptop"] [price 1000] [rating 4.7] [inStock ✅]]
 [[name "Desktop"] [price 1500] [rating 4.9] [inStock 🔳]]
 [[name "Tungsten Cube"] [price 2500] [rating 3.4] [inStock ✅]]
 [[name "Keyboard"] [price 70] [rating 4.8] [inStock 🔳]]
 [[name "Monitor"] [price 200] [rating 4.5] [inStock ✅]]
 [[name "Camera"] [price 600] [rating 4.3] [inStock ✅]]]

[[[rating] 🔍 4.5 >=]            ;; look up the rating, check if it's at least 4.5
 [[inStock] 🔍]]                 ;; look up whether it's in stock
[🛡️ ▶️] 💯                       ;; make a program that returns whether both of the above are true
[2️⃣ ✂️1️⃣] 🎒                     ;; package the data with the program
🧲 ▶️                            ;; keep only those items that match
[[[price] 🔍] max-by] reduce ▶️  ;; Find the max by price
[[name "Laptop"]
 [price 1000]
 [rating 4.7]
 [inStock ✅]]

3. Benefits

3.1. Easy for beginners

Kcats is designed to require learning as few concepts as possible. It's made with:

  • Stacks
  • Data values (numbers, strings, bytes)
  • Lists
  • Functions
  • Pipes

All of these are combined in various ways to achieve any type of expression, the same way a few types of lego bricks can be combined to make any object.1

Kcats code has a simple structure with only function names, data literals and quotation marks. 2

Format code however you want, kcats doesn't care.

Interactive development is encouraged. The documentation for the standard library can be queried with the language itself. See 5.4.

3.2. Powerful

Kcats' metaprogramming facilities (programs that write programs) allow you to express yourself succinctly.

3.3. Excels in the small

Many programming languages require you to write special setup code before you can do actual work. With kcats, there's no "ceremony" - you just write what you want to do. Here's a program that reads a database file of contacts, and finds contacts that are at least 55 years old. The program is only 9 words.

"contacts.kcats" slurp read [[age] 🔍 55 >=] 🧲 ▶️   

3.4. Features

  • [X] Automatic memory management
  • [X] Multithreading
  • [X] Error handling
  • [X] Metaprogramming
  • [X] Channels/CSPs
  • [X] Introspection
  • [X] Filesystem I/O
  • [X] TCP/IP Socket I/O
  • [X] Generators
  • [X] Sequence library
  • [X] Cryptographic primitives
  • [X] Builtin debugging tools
  • [X] Serialization

3.5. Tradeoffs

Kcats is meant to handle personal automation tasks, where expressivity and simplicity matter far more than raw performance. It's not currently suitable for processing huge datasets or high performance number crunching, and likely never will be.

{"prompt":"What design goal keeps Kcats easy to learn?","answer":"It is built so users only need to learn a very small set of concepts."}
{"prompt":"True or false: Kcats aims to make users learn only a handful of concepts.","answer":"True. Minimal conceptual load is a core design goal."}
{"prompt":"How does Kcats make itself beginner-friendly?","answer":"By requiring as few new concepts as possible."}
{"prompt":"Which five fundamental building blocks does Kcats rely on?","answer":"Stacks, data values (numbers/strings/bytes), lists, functions and pipes."}
{"prompt":"Yes or no: Are stacks, data values, lists, functions and pipes the basis of all Kcats code?","answer":"Yes. Those five elements form its core."}
{"prompt":"What components does Kcats combine to build all programs?","answer":"Stacks, data values, lists, functions and pipes."}
{"prompt":"What lets Kcats express any type of expression despite few building blocks?","answer":"The core blocks can be freely composed like Lego bricks, covering all expressions."}
{"prompt":"True or false: With just its five core elements, Kcats can build any expression.","answer":"True. Their composability gives full expressive power."}
{"prompt":"How does Kcats achieve expressive power with so few constructs?","answer":"By allowing its small set of building blocks to be combined in countless ways."}
{"prompt":"What three parts make up Kcats code syntax?","answer":"Only function names, data literals and quotation marks."}
{"prompt":"True or false: Kcats syntax includes special punctuation beyond quotation marks.","answer":"False. Besides quotation marks, there are only names and literals."}
{"prompt":"Which items appear in Kcats source besides data literals and quotation marks?","answer":"Function names; nothing else is needed."}
{"prompt":"How strict is Kcats about code formatting?","answer":"It doesn’t care; you can format code any way you like."}
{"prompt":"True or false: Kcats enforces a specific indentation style.","answer":"False. Whitespace and formatting are ignored."}
{"prompt":"Why can you freely reformat Kcats code?","answer":"Because the language ignores formatting and whitespace."}
{"prompt":"What development style does Kcats encourage and how can you see library docs?","answer":"Interactive use is encouraged; you can query the standard-library documentation from within Kcats itself."}
{"prompt":"Yes/no: Can you look up Kcats std-lib docs directly from a running REPL?","answer":"Yes."}
{"prompt":"How does Kcats support exploratory programming?","answer":"By encouraging interactive work and letting code retrieve its own documentation."}
{"prompt":"In Kcats, what is special about names like \"+\"?","answer":"Symbols can appear in function names; they’re just identifiers and can be renamed freely."}
{"prompt":"True or false: Symbols embedded in Kcats function names are part of core syntax.","answer":"False. They’re only in the name, not the syntax."}
{"prompt":"Why can \"+\" be renamed in Kcats without language changes?","answer":"Because symbols in function names are just identifiers, not hard-wired syntax."}
{"prompt":"What facility lets Kcats programs write other programs?","answer":"Its built-in metaprogramming features."}
{"prompt":"Yes/no: Does Kcats provide metaprogramming to write code that writes code?","answer":"Yes."}
{"prompt":"How does Kcats enable succinct expression?","answer":"Through metaprogramming facilities that let programs generate programs."}
{"prompt":"What kind of boilerplate is needed before doing work in Kcats?","answer":"None; you simply write the operations you need."}
{"prompt":"True or false: Kcats requires special setup code before real work.","answer":"False. There is no ceremony."}
{"prompt":"Why is Kcats described as \"no-ceremony\"?","answer":"Because you can jump straight to the task without any setup code."}
{"prompt":"List built-in capabilities provided by Kcats.","answer":"Examples: automatic memory management, multithreading, filesystem and socket I/O, channels, crypto, debugging, serialization."}
{"prompt":"Yes/no: Does Kcats come with multithreading, channels and cryptographic primitives out of the box?","answer":"Yes."}
{"prompt":"Which language supplies memory management, concurrency tools, I/O & debugging as built-ins?","answer":"Kcats."}
{"prompt":"For what main domain is Kcats intended?","answer":"Personal automation where expressivity and simplicity trump raw speed."}
{"prompt":"True or false: Kcats prioritizes expressivity over raw performance.","answer":"True."}
{"prompt":"Why is Kcats suited to personal automation scripts?","answer":"Because it favors concise, simple expression rather than performance."}
{"prompt":"What kinds of tasks is Kcats currently unsuited for?","answer":"Processing huge datasets or high-performance number crunching."}
{"prompt":"Yes/no: Is Kcats recommended for large-scale data crunching?","answer":"No."}
{"prompt":"When would Kcats likely fail to meet performance needs?","answer":"When handling very large datasets or intensive numerical workloads."}

4. Getting Started

4.1. Builds

4.1.1. Download pre-built binaries

See Releases. Binaries currently do not have installers, you will need to place the executable in a convenient place and chmod it (on mac/linux).

4.1.2. Build from source

You can skip this section if you're using pre-built binaries described above.

4.1.2.1. Dependencies
  • emacs
  • rustc
  • cargo
4.1.2.2. Creating the source

The source is contained within these org files, along with annotations and other non-code information. To generate the source code (that the runtime or compiler needs as separate files) before running or building, run the following command:

emacs --batch --load org --load tangle.el

4.1.2.3. Building

See Building

4.2. Using

See Using

4.3. Developing

4.3.1. Browsing the source

It can be checked out from git or viewed here: Production Source

4.3.2. Emacs IDE

See emacs-ide.org in the source tree for more info.

5. Fundamentals

5.1. Key terminology

item
A unit of information (aka value) of various types: numbers, strings, byte arrays, words, characters, and lists. (examples: 5, "Bob", swap, [1 2 3])
list
An item that contains other items, in a particular order (delimited by square braces). Example: [a "foo" 2].
program
a list of instructions intended to be carried out by a machine. (example: [[odd?] filter]).
stack
A list with a first-in, first-out interface. This is where the program stores all the data it needs to manipulate.
word
causes the program to do something, usually taking some items from the top of the stack, and using them to create new stack items. Words can be represented as actual english words, or math symbols or even emoji. (examples: 🔀, +, over)
axiom word
A word not defined in terms of other words.
definition
what a word is supposed to do, represented either in the base language for axiom words, or as a program.
dictionary
a set of available words and their definitions.
environment
the entire state of an executing program, which includes a stack, program, and dictionary.
pipe

A conduit to communicate between environments, and to the outside world. Items are put into pipes and emerge somewhere else (another environment, a file on disk, a remote machine, etc).

{"prompt":"What is an item in kcats?","answer":"A unit of information of various types: numbers, strings, byte arrays, words, characters, or lists."}
{"prompt":"True or false: In kcats, an item may be a word or a list.","answer":"True. Any value type, including words and lists, qualifies as an item."}
{"prompt":"Which kcats term refers to basic values like 5 or \"Bob\"?","answer":"Item."}
{"prompt":"What is a list in kcats?","answer":"An item that holds other items in order, written inside square brackets."}
{"prompt":"Yes or no: A kcats list is itself an item.","answer":"Yes. Lists are items that contain other items."}
{"prompt":"How does kcats describe an ordered collection like [a \"foo\" 2]?","answer":"As a list."}
{"prompt":"Define a program in kcats.","answer":"A list of instructions intended to be carried out by a machine."}
{"prompt":"True or false: In kcats, programs are represented as lists.","answer":"True. A program is literally a list of instructions."}
{"prompt":"What structure does kcats use to hold instructions for execution?","answer":"A program list."}
{"prompt":"What is the stack in kcats?","answer":"A list with a first-in, first-out interface where the program stores all data it manipulates."}
{"prompt":"Yes or no: The kcats stack provides FIFO access.","answer":"Yes. The stack is a FIFO list."}
{"prompt":"Why is the stack important in kcats execution?","answer":"It stores the data the program reads and writes during execution."}
{"prompt":"What is a word in kcats terminology?","answer":"An instruction that acts on stack items, producing new items; names can be text, symbols, or emoji."}
{"prompt":"True or false: kcats words can be emoji like 🔀.","answer":"True. Words may be English, symbols, or emoji."}
{"prompt":"How does a word interact with the stack in kcats?","answer":"It consumes items from the top and pushes resulting items back."}
{"prompt":"What is an axiom word in kcats?","answer":"A word whose behavior is not defined by other words; it is primitive."}
{"prompt":"Yes or no: An axiom word depends on other word definitions.","answer":"No. It is defined independently."}
{"prompt":"When is a kcats word called an axiom?","answer":"When it is primitive, not defined in terms of other words."}
{"prompt":"What does \"definition\" mean for a kcats word?","answer":"The specification of what a word does, written in base language for axioms or as a program for others."}
{"prompt":"True or false: Every kcats word has a definition.","answer":"True. Each word is associated with some defining description or code."}
{"prompt":"How can a kcats word be defined if it isn't an axiom?","answer":"By a program composed of other words."}
{"prompt":"What is a dictionary in kcats?","answer":"The set of available words along with their definitions."}
{"prompt":"Yes or no: The kcats dictionary holds word definitions.","answer":"Yes. It maps words to their definitions."}
{"prompt":"Which component lists all defined words in a kcats environment?","answer":"The dictionary."}
{"prompt":"What does \"environment\" encompass in kcats?","answer":"The complete execution state: stack, current program, and dictionary."}
{"prompt":"True or false: In kcats, the environment includes only the stack.","answer":"False. It also contains the program and dictionary."}
{"prompt":"Why is the environment crucial during kcats execution?","answer":"It holds everything needed to run: stack data, running program, and available word definitions."}
{"prompt":"What is a pipe in kcats?","answer":"A conduit for items to move between environments or to external destinations like files or remote machines."}
{"prompt":"Yes or no: Pipes allow kcats environments to communicate.","answer":"Yes. Pipes transfer items between environments or to the outside world."}
{"prompt":"How does kcats send data to another environment or file?","answer":"Through a pipe, which carries items to the target destination."}

5.2. The stack

Kcats uses a stack to keep track of all the values it needs to work with, instead of variables and function parameters. To manipulate data, you put it onto a stack, and then words operate on the items at the top of the stack (which might remove, shuffle, replace, or add new items). If you're familiar with functions in other languages, that's all words are - they're a function of the current stack, and they return a new stack.

Here's a simple example. If we mentally execute the program below, we first put 1 onto the stack. Then we put 2 onto the stack. 2 is now on top of 1. Then we put the word + onto the stack, where it will consume the 2 and the 1, and leave their sum, 3.

1 2 +
3

Multiple steps are accomplished just by adding more words and data. For example, in the program below we can add 1 and 2 (leaving 3 on the stack), and then multiply by 5, leaving 15.

1 2 + 5 *
15

Here's how it would look step by step (where the | separates the program that hasn't run yet - on the right, from the stack on the left). The stack's top item is just to the left of the |.

;; stack |  remaining program
;; ------|--------------------
         | 1 2 + 5 * 
       1 | 2 + 5 * 
     1 2 | + 5 *
       3 | 5 *
     3 5 | *
      15 |  

When there is nothing remaining to the right of the |, the program is finished. The result is what is left on the stack (in this case 15).

Note the stack can end up with multiple items. When it's printed, it will always start with the top of the stack - the last thing in is the first thing out.

1 2 3
3 2 1

Lists are denoted with square brackets, like [1 2 3]. When encountered, they just go onto the stack as a single unit. Words can operate on lists once the list is on the stack. You can see below the word 🔗 joins two lists into one.

[1 2 3] [4 5] 🔗
[1 2 3 4 5]
{"prompt":"What structure does kcats use to store values instead of variables?","answer":"A single stack."}
{"prompt":"True or false: kcats relies on variables to hold data.","answer":"False. It keeps all data on a stack."}
{"prompt":"How does kcats keep track of the values it works with?","answer":"By pushing them onto a stack rather than using variables or parameters."}
{"prompt":"Where must data be placed before words can operate on it in kcats?","answer":"On the stack; words work on the top items."}
{"prompt":"True or false: Words in kcats act only on items at the top of the stack.","answer":"True. They consume or rearrange the top elements."}
{"prompt":"Why must you push data to the stack before executing a word?","answer":"Because words can only access and manipulate the top of the stack."}
{"prompt":"In kcats, what do words conceptually take as input and produce as output?","answer":"They take the current stack and return a new stack."}
{"prompt":"Yes or no: A kcats word can be viewed as a stack-to-stack function.","answer":"Yes. Each word maps one stack state to another."}
{"prompt":"How are words analogous to functions in other languages?","answer":"They accept the current stack as argument and yield an updated stack."}
{"prompt":"What does the '+' word do to the top two numbers on the stack?","answer":"It removes them and pushes their sum."}
{"prompt":"True or false: '+' in kcats leaves the stack unchanged.","answer":"False. It pops two numbers and pushes their sum."}
{"prompt":"How does '+' transform the stack contents?","answer":"By replacing the top two numbers with their sum."}
{"prompt":"When does a kcats program end?","answer":"When no words remain to execute; the remaining stack is the output."}
{"prompt":"True or false: Execution continues in kcats until the stack is empty.","answer":"False. It stops when the program stream is empty; leftover stack items are results."}
{"prompt":"What determines the result of a completed kcats program?","answer":"The contents left on the stack once all words have run."}
{"prompt":"If a stack holds multiple items, which item is printed first?","answer":"The top of the stack, i.e., the most recently pushed."}
{"prompt":"Yes or no: kcats prints the bottom of the stack first.","answer":"No. It prints from the top downward."}
{"prompt":"Which order does kcats use when printing a stack with several values?","answer":"Last-in-first-out: top item first, then the rest in order."}
{"prompt":"How do you write a list literal in kcats?","answer":"Enclose items in square brackets, e.g. [1 2 3]."}
{"prompt":"True or false: The tokens inside [1 2 3] are pushed individually.","answer":"False. The entire bracketed list is pushed as one value."}
{"prompt":"Which notation tells kcats to treat several items as one list value?","answer":"Wrapping them in square brackets."}
{"prompt":"What does the 🔗 word do to the top two lists?","answer":"It concatenates them into a single list."}
{"prompt":"Yes or no: 🔗 merges two lists into one.","answer":"Yes. It links the two lists into one longer list."}
{"prompt":"How can you join two lists sitting on the stack?","answer":"Use the 🔗 word to concatenate them."}

5.3. Emoji

You'll notice in the earlier examples there's a fair number of pictograms (emoji). Kcats uses emoji pictographs instead of english words for some commonly used functions. They are treated just like any other word, except more colorful and succinct.

To get started we'll explain what a few of these emoji mean, that are used in the next section.

  • 🔀 swaps the top two stack items.
  • ▶️ executes a program snippet.
  • 👥 duplicates the top stack item.
  • 🚜 Run the same program snippet on each item in a list. (Why a tractor? Tractors have different attachments, to perform the same task like tilling, harvesting etc, on every row of the field)
  • 🪄 Magically makes the top item disappear while executing the program beneath, then magically makes the item reappear.

Words in the dictionary that are made with emoji will have english documentation that you can search for, in case you forget which symbol it is. For example:

dictionary lingo [[1 doc] 🔍 "swap" contains?] 🧲 ▶️ 
[[•🔀 [[definition builtin_•🔀]
      [doc "The crossing arrows with dot represents swapping one stack position down (2nd and 3rd items)"]
      [examples [[[1 2 3 •🔀] [2 1 3] "Swap 2nd and 3rd items"]]]
      [namespace #b64 "core"]
      [spec [[[item a]
              [item b]
              [item c]]
             [[item a]
              [item c]
              [item b]]]]]]
 [🔀 [[definition builtin_🔀]
     [doc "The crossing arrows denote swapping the top two stack items."]
     [examples [[[1 2 3 🔀] [1 3 2] "Swap top two items"] [[🔳 ✅ 🔀] [✅ 🔳]
                                                          "Swap boolean values"]
                [[42 "hello" 🔀] ["hello" 42]
                 "Swap number and string"]
                [[🔳 "" 🔀] ["" 🔳]
                 "Swap Nothing and empty string"]
                [["a" "b" 🔀] ["b" "a"]
                 "Swap strings"]
                [[[1 2]
                  [3 4]
                  🔀]
                 [[3 4]
                  [1 2]]
                 "Swap lists"]
                [[[1 [2]]
                  [3 [4]]
                  🔀]
                 [[3 [4]]
                  [1 [2]]]
                 "Swap nested lists"]
                [[[[🔳]] 🔳 🔀] [🔳 [[🔳]]]
                 "Swap deeply nested empty list with empty list"]
                [[1 2 3 🔀 🔀] [1 2 3] "Double swap"] [[1 2 🔀 3 🔀] [2 3 1] "Interleaved swaps"] [[1
                                                                                                2 🔀 🗑️]
                                                                                               [2] "Swap then trash"]
                [[42 42 🔀] [42 42]
                 "Swap identical numbers"]
                [[🔳 🔳 🔀] [🔳 🔳]
                 "Swap identical empty lists"]
                [[[1 2]
                  [1 2]
                  🔀]
                 [[1 2]
                  [1 2]]
                 "Swap identical complex lists"]
                [[[🔀] [🗑️ "Need 2 items to swap"]
                  🩹 ▶️]
                 ["Need 2 items to swap"] "Empty stack"]
                [[[1 🔀]
                  [🗑️ "Need 2 items to swap"]
                  🩹 ▶️]
                 ["Need 2 items to swap"] "Only one item on stack"]]]
     [namespace #b64 "core"]
     [spec [[[item a]
             [item b]]
            [[item b]
             [item a]]]]]]]

So now we know 🔀 does the swapping.

You don't need to understand how that program above works yet, just know that you can run it yourself and replace "swap" with whatever word or phrase you want to search.

{"prompt":"What does kcats use emoji pictographs for?","answer":"To stand in for some common functions; the emoji act like any other word."}
{"prompt":"True or false: kcats sometimes replaces English function names with emoji words.","answer":"True. Emoji substitute for English names and behave as normal words."}
{"prompt":"Why are emoji used in kcats vocabulary?","answer":"They provide a colorful, concise alternative to English function names while functioning normally."}
{"prompt":"What does 🔀 do in kcats?","answer":"It swaps the top two items on the stack."}
{"prompt":"True or false: 🔀 duplicates the top stack item.","answer":"False. 🔀 swaps the top two items."}
{"prompt":"Which emoji swaps the topmost pair of stack items?","answer":"🔀."}
{"prompt":"What is the purpose of ▶️ in kcats?","answer":"It executes a program snippet."}
{"prompt":"Yes or no: ▶️ runs a snippet of code.","answer":"Yes. ▶️ executes the given program snippet."}
{"prompt":"Which emoji is used to run a program snippet?","answer":"▶️."}
{"prompt":"What does 👥 accomplish on the stack?","answer":"It duplicates the top stack item."}
{"prompt":"True or false: 👥 duplicates the top stack item.","answer":"True. 👥 copies the value on top of the stack."}
{"prompt":"Which emoji duplicates the value at the top of the stack?","answer":"👥."}
{"prompt":"What operation does 🚜 perform in kcats?","answer":"It applies the same program snippet to every item in a list."}
{"prompt":"Yes or no: 🚜 iterates a snippet over each list element.","answer":"Yes. 🚜 runs the snippet on each list item."}
{"prompt":"How do you apply one snippet across all elements of a list?","answer":"Use 🚜 to run the snippet for every list item."}
{"prompt":"What magic does 🪄 perform in kcats?","answer":"It hides the top stack item while the next program runs, then restores the item afterward."}
{"prompt":"True or false: 🪄 permanently removes the top item.","answer":"False. 🪄 only hides it during execution and then brings it back."}
{"prompt":"How can you execute code without the top item but get it back later?","answer":"Use 🪄, which temporarily removes then restores the top stack item."}
{"prompt":"Can you find English docs for emoji words in the kcats dictionary?","answer":"Yes. Every emoji-defined word includes searchable English documentation."}
{"prompt":"True or false: Emoji entries lack searchable English docs.","answer":"False. Emoji words come with English documentation you can search."}
{"prompt":"Where can you look up meanings for emoji words you forget?","answer":"In the dictionary's English documentation attached to each emoji word."}

5.4. Exploring

5.4.1. Overview

Now that we know the very basics, we can explore and learn as we go. Kcats lets you treat the standard library (the dictionary) as data, and you can process it with… itself. Documentation is in there. You just need to know how to ask for it. So here's how you do it, and you'll understand how exactly it works later.

In all the examples in this document, you can run them on the command line, by running kcats -p, followed by the program in single quotes, like this:

kcats -p 'dictionary lingo [first] 🚜 ▶️ [] sort'

In case you want to view it in its entirety, the standard library is part of the source, it lives in the lexicon file.

5.4.2. What words or functions are available?

This program retrieves the dictionary of the current environment, and prints just the name of each, sorted in alphabetical order.

dictionary lingo [first] 🚜 ▶️ [] sort
[#️⃣⛏️ * + - / < <= = > >= abs addmethod advance and animate assemble assert assign
 association association? assocify
 attend autoformat bail bailer bits both both? break breakpoint butlast bytes? cache
 capture catcher ceiling character close collect combinations compare
 compare-by confine contains? cram cut database days dec decache decide decodejson
 definition dictionary dictmerge dipped draft drain dropper dump each
 emit empty empty? encode encodeitem encodejson ends? entry environment environment?
 error? eval-step evaluate evaluator even? exp extractor fail file-in file-out
 finished? first flatten flip floor fold format frequencies future generator get group
 handle handoff hashbytes heatmap hours inc indexed indexer
 indexof inscribe inspect integer? integers interpose intersection keep key label
 last let liberator lingo list? log max max-by method? milliseconds
 min min-by minutes mod module namespace negative? number number? odd? or over pad
 pair pair? parse parse-edn parse-utf8 partition persist
 pipe-in pipe-out pipe? pop positive? prepend primrec print produce quot radix random
 range read reap receiver reduce rem remove repetition
 resolve rest restore resume retry reverse round second seconds select sender serversocket
 set set? sign skipper sleep slice slurp socket
 something? sort sort-indexed spawn spit splitter sprint sqrt stage standard starts?
 stepper string string? taker timer times timestamps top tos
 tracer triplet tunnel type unassign under unnamespace until update updates use using
 value verify walk when within? word word? words
 xor zero? zip ••🐋 ••👥 ••📮 ••🗑️ ••🛡️ ••🪄 •🐋 •👥 •📮 •🔀 •🗑️ •🛡️ •🪄 ↔️ ⏳ ▶️ ☯️
 ⚓ ⚖️ ⛏️ ✅ 🌀 🍫 🎁 🎒 🎭 🏷️⛏️ 🐋 👥 💉 💯 📏 📣 📤 📮 📸 🔀
 🔍 🔗 🗑️ 🚜 🛟 🛡️ 🧤 🧦 🧲 🧹 🩹 🪄 🪆 🪜]

Even though the rest of this document will explain a lot of these words and how they work, the above program does the following:

  • dictionary retrieves the currently active dictionary
  • lingo returns the active words of the dictionary on the stack
  • [first] 🚜: for each word/definition pair in the dictionary, take the first, which is the word.
  • [] sort: sort takes a program to transform each item in the list it's sorting, to use for comparison. We want to use the word itself for comparison, so we don't need to transform it at all, hence the empty program.

5.4.3. What inputs/outputs does a particular word have?

The specification of a word's input and output types is stored in the dictionary too. It's in the property called spec. Let's say you're interested in the word swap.

dictionary lingo [🔀 spec] 🔍
[[[item a]
  [item b]]
 [[item b]
  [item a]]]

What this program does is fetches the dictionary words, then looks up the swap definition, then within that definition, looks up the spec property.

In the result, what we have here is two lists - the spec of the input, and the spec of the output.

The input spec is [[item a] [item b]]. The output spec is [[item b] [item a]]. What it's telling you is that it requires two items on the stack, any two, we'll call them a (on top) and b beneath. There may be more items below that but they won't be touched. When swap is finished, a and b will have their places swapped so that b is on top. And in fact that's what we get:

"b" "a" 🔀
"b" "a"

For more details see 5.5.3.

Remember the top of the stack is printed first, and so b is now on top.

The format of an input or output spec is a list of either a type or a type/name pair. For example, an input spec of [[integer age] [string name]] means the function takes an integer representing an 'age' on top of stack, and a string representing a name beneath that. The names are for documentation only. You can also leave out any name eg [integer string] is functionally the same spec, just less descriptive. Not all inputs or outputs need to be named, [integer [string name]] is also a valid input spec.

Some words have arbitrary stack effects because, for example, they run an arbitrary program or replace the stack. The output spec for these types of words are specified as [*]. Some examples of such words are ▶️ or restore.

5.4.4. What are some example usages of a word?

dictionary lingo [🔀 examples] 🔍
[[[1 2 3 🔀] [1 3 2] "Swap top two items"]
 [[🔳 ✅ 🔀] [✅ 🔳] "Swap boolean values"]
 [[42 "hello" 🔀] ["hello" 42] "Swap number and string"]
 [[🔳 "" 🔀] ["" 🔳] "Swap Nothing and empty string"]
 [["a" "b" 🔀] ["b" "a"] "Swap strings"]
 [[[1 2] [3 4] 🔀] [[3 4] [1 2]] "Swap lists"]
 [[[1 [2]] [3 [4]] 🔀] [[3 [4]] [1 [2]]] "Swap nested lists"]
 [[[[🔳]] 🔳 🔀] [🔳 [[🔳]]] "Swap deeply nested empty list with empty list"]
 [[1 2 3 🔀 🔀] [1 2 3] "Double swap"]
 [[1 2 🔀 3 🔀] [2 3 1] "Interleaved swaps"]
 [[1 2 🔀 🗑️] [2] "Swap then trash"]
 [[42 42 🔀] [42 42] "Swap identical numbers"]
 [[🔳 🔳 🔀] [🔳 🔳] "Swap identical empty lists"]
 [[[1 2] [1 2] 🔀] [[1 2] [1 2]] "Swap identical complex lists"]
 [[[🔀] [🗑️ "Need 2 items to swap"] 🩹 ▶️] ["Need 2 items to swap"] "Empty stack"]
 [[[1 🔀] [🗑️ "Need 2 items to swap"] 🩹 ▶️] ["Need 2 items to swap"] "Only one item on stack"]]

This is a list of examples, and each example is a pair or triple:

  • A program that calls the given word
  • A program that doesn't call the word that gives the same result 3
  • An optional description of what the example is demonstrating

5.4.5. Continuing exploration

Use the same technique to explore other words. You can simply replace the word in the code snippets above with some other word. Here's how you find the examples for =, which tests for equality of two items - just replaced swap with =.

dictionary lingo [= examples] 🔍
[[[1 2 =] [[]] "Different Numbers are not equal"]
 [[1 1 =] [] "Same numbers are equal"]
 [[1 1 =] [] "Same value integer and float are equal"]
 [[[1] [] =] [[]] "Number and Nothing are unequal"]
 [[[1 [[]]] [1 [[]]] =] [] "Same nested list with numbers are equal"]
 [[[1 ["foo"]] [1 ["foo"]] =] [] "Same nested list with string are equal"]
 [["hi" "hi" =] [] "Same strings are equal"]
 [["hi" "there" =] [[]] "Different strings are unequal"]
 [[\h \h =] [] "Same characters are equal"]
 [[\h \i =] [[]] "Different characters are unequal"]
 [["hi" encode "hi" encode =] [] "Same bytes are equal"]
 [["hi" encode "there" encode =] [[]] "Different bytes are unequal"]
 [[[] ✅ =] [[]] "Different booleans unequal"]
 [[[1 ["foo"]] [1 ["bar"]] =] [[]] "Nested lists with different strings are unequal"]
 [[[] [] =] [] "'Nothing' is equal to itself"]
 [[[] [] association =] [] "List/Association empty container types are equal"]
 [[[] [] set =] [] "List/Set empty container types are equal"]
 [[[[a b]] [[a b]] association =] [[]] "Nonempty List/Association types are unequal"]
 [[[1 2 3] set [3 1 2] set =] [] "Sets constructed from different lists are equal"]]

5.5. Data types

5.5.1. Types

5.5.1.1. Words

In kcats, words can be either in the dictionary, or not.

Dictionary words are verbs, and they are used directly in the execution of programs. For example: 👥 and 🔀

1 👥 2 🔀
1 2 1

Non-dictionary words are only used inside lists, often as field names. These words can't be executed, because they're not in the dictionary and not intended to be. They're used more like you'd use strings or keywords in other programming languages.

[foo bar baz] [quux] 🍫 📮 
[foo bar baz quux]

Note the use of 🍫 here. What does the chocolate bar mean? Let's find out!

dictionary lingo [🍫] 🔍
[[definition builtin_🍫]
 [doc "The unwrapped chocolate bar denotes unwrapping a container to place all the items inside on the stack."]
 [examples [[[[1] 🍫] [1] "Unwrap a list of one item"]
            [["foo" ["bar" "baz"] 🍫] ["foo" "bar" "baz"] "Unwrap a list of strings with existing stack items"]
            [["foo" 🔳 🍫] ["foo"] "Unwrap an empty list is a no-op"]
            [[[1 2 3] 🍫] [1 2 3] "Unwrap a list of multiple items"]]]
 [namespace #b64 "core"]
 [spec [[list] [*]]]]

Ok from the examples we see it unwraps the list on top of the stack, splicing its contents into the stack. (the emoji is a partially unwrapped chocolate bar, so that's where the unwrapping concept comes from).

What about 📮?

dictionary lingo [📮] 🔍
[[definition builtin_📮]
 [doc "The postbox denotes putting an item into a container"]
 [examples [[[🔳 1 📮] [[1]] "Put integer into empty list"] [[[1 2 3] 4 📮] [[1 2 3
                                                                           4]] "Put integer into list"] [["foo" \d 📮] ["food"] "Put character into string"]
            [["foo" encode 32 📮 string] ["foo "] "Put byte into byte array"]]]
 [namespace #b64 "core"]
 [spec [[item receptacle]
        [receptacle]]]]

We see from the examples that 📮 puts an item into a container. What's wrong with just trying to 📮 quux directly into the list?

[foo bar baz] quux 📮
[[asked [quux]]
 [handled 🔳]
 [reason "word is not defined"]
 [type error]
 [unwound [quux 📮]]]
[foo bar baz]

The problem is that as soon as kcats reaches quux, it tries to execute it as a verb. quux is not in the dictionary, so it fails to execute.

What we want is to get quux onto the stack by itself, without actually executing it. We can do that with [quux] 🍫. The word 🍫 removes the list wrapper and leaves a bare word on the stack. Another way to go about this is to use 🔗, so we don't need 🍫:

[foo bar baz] [quux] 🍫 📮
[foo bar baz quux]
[foo bar baz] [quux] 🔗
[foo bar baz quux]
{"prompt":"What two categories can words fall into in kcats?","answer":"They are either dictionary words or non-dictionary words."}
{"prompt":"True or false: every word in kcats belongs to the dictionary.","answer":"False. Words can be dictionary or non-dictionary."}
{"prompt":"How does kcats classify its words?","answer":"It distinguishes between dictionary verbs and non-dictionary words used only inside lists."}
{"prompt":"In kcats, what role do dictionary words play during execution?","answer":"They act as verbs that are executed directly."}
{"prompt":"Yes or no: dictionary words in kcats are executed as verbs.","answer":"Yes, dictionary words are verbs executed immediately."}
{"prompt":"Why are dictionary words important when a kcats program runs?","answer":"Because they provide the executable verbs the interpreter runs."}
{"prompt":"Where are non-dictionary words typically used in kcats?","answer":"Inside lists, often as field names, acting like strings or keywords."}
{"prompt":"True or false: non-dictionary words can be executed as verbs.","answer":"False. They are not executable; they serve as data inside lists."}
{"prompt":"How do non-dictionary words behave compared to dictionary verbs?","answer":"They're purely data, never executed, similar to strings or keywords."}
{"prompt":"What does the 🍫 verb do in kcats?","answer":"It unwraps the top list, pushing all its items individually onto the stack."}
{"prompt":"True or false: 🍫 splices the contents of a list onto the stack.","answer":"True. It removes the list wrapper and spreads its elements."}
{"prompt":"How does 🍫 affect the list on the top of the stack?","answer":"By unwrapping it so each contained item appears separately on the stack."}
{"prompt":"What action is performed by the 📮 verb in kcats?","answer":"It inserts an item into a container (list, string, byte array, etc.)."}
{"prompt":"Yes or no: 📮 removes items from a container.","answer":"No. It puts an item into a container."}
{"prompt":"When would you use 📮?","answer":"When you need to place a value inside an existing container."}
{"prompt":"What happens if kcats encounters a word not in the dictionary during execution?","answer":"It raises a \"word is not defined\" error."}
{"prompt":"True or false: a bare non-dictionary word will run without issues.","answer":"False. It triggers an undefined-word error."}
{"prompt":"How can you place the word \"quux\" on the stack without executing it?","answer":"Wrap it in a list and run 🍫, e.g., [quux] 🍫."}
{"prompt":"Yes or no: using [word] 🍫 lets you push a bare word as data.","answer":"Yes. The list protects the word and 🍫 unwraps it onto the stack."}
{"prompt":"Why combine a single-item list with 🍫?","answer":"The list blocks execution and 🍫 then unwraps it, leaving the word as data on the stack."}
{"prompt":"Which verb can replace 🍫 when adding \"quux\" to a list?","answer":"🔗; it avoids the extra unwrap step."}
{"prompt":"True or false: 🔗 can be used instead of 🍫 to add a word from a list.","answer":"True. 🔗 eliminates the need for a prior 🍫."}
{"prompt":"How can 🔗 simplify inserting a non-dictionary word into a container?","answer":"By linking the list contents directly, so you skip the 🍫 unwrap."}
5.5.1.2. Booleans

Most programming languages have special built-in values true and false with privileged status in the language. Kcats does not. Instead, kcats uses container emptiness to determine logical truth - an empty container acts like false, and anything else acts like true. For convenience, the word (which is just a regular word that puts itself on the stack) is conventionally used throughout the standard library to represent an affirmative value, but it has no special status in the language's semantics.

Here's an example of checking boolean values:

[] ["yes"] ["no"] ↔️
"no"

versus

"anything" ["yes"] ["no"] ↔️
"yes" "anything"

Notice that when we branch on a boolean value, if we get the negative value, it's not preserved (because we already know what it must be - an empty container), but the affirmative value is kept.

But wait, what about this?

3 odd?

Some action words will return the word (meaning affirmative), but as we said earlier, it's not really a boolean, it's just the word .

Use empty list [] to indicate logical negative. There is an emoji you can use interchangeably with []: 🔳. It nicely complements the check mark (as in, an empty checkbox).

{"prompt":"What built-in boolean literals does kcats provide?","answer":"None. Kcats has no built-in true/false values."}
{"prompt":"True or false: kcats defines special true and false values.","answer":"False. Kcats has no dedicated boolean literals."}
{"prompt":"How does kcats differ from most languages regarding boolean literals?","answer":"It doesn't supply any; there are no privileged true/false constants."}
{"prompt":"How does kcats decide if a value is true or false?","answer":"By container emptiness: an empty container is false, anything else is true."}
{"prompt":"True or false: Any non-empty container counts as true in kcats.","answer":"True. Only the empty container is considered false."}
{"prompt":"Which rule governs truthiness in kcats?","answer":"Container emptiness: empty means false, anything else means true."}
{"prompt":"What is the purpose of the ✅ word in kcats?","answer":"It’s a conventional affirmative token but has no special semantics."}
{"prompt":"Yes/no: The ✅ word has privileged boolean status in kcats.","answer":"No. It is just an ordinary word placed on the stack."}
{"prompt":"Why can programmers use ✅ to signal truth in kcats?","answer":"Because it’s a normal word that pushes itself; by convention it marks affirmation."}
{"prompt":"After a false branch, what happens to the test value in kcats?","answer":"The empty container (false) is dropped from the stack."}
{"prompt":"True or false: Kcats keeps the negative test value after branching.","answer":"False. The negative (empty) value is discarded; only the true value is kept."}
{"prompt":"How are stack contents affected by branching on truth in kcats?","answer":"If the test was false, the empty container is removed; if true, the affirmative value remains."}
{"prompt":"What value do predicates like odd? return when the test succeeds?","answer":"They push the word ✅ onto the stack to signal affirmation."}
{"prompt":"Yes/no: Some kcats predicates return ✅ for an affirmative result.","answer":"Yes. Several action words yield ✅ when their condition is met."}
{"prompt":"Which token is returned by certain action words to indicate success?","answer":"The word ✅."}
{"prompt":"Which value represents logical false in kcats?","answer":"The empty list [], also shown as 🔳."}
{"prompt":"True or false: 🔳 is an alias for [] when expressing false in kcats.","answer":"True. The empty-box emoji can be used in place of the empty list."}
{"prompt":"Which emoji can you use instead of [] to denote false in kcats?","answer":"The empty box emoji 🔳."}
5.5.1.3. Numbers

Integers and floats are supported (64 bit).

Supported math operations include +, -, *, /, mod, rem, min, max, abs, inc, dec, <, >, <=, >=, ceil, sqrt, odd?, even?.

5.5.1.4. Containers
  1. Overview

    Containers are types that contain other items. Some containers are homogenous. For example, strings are a container for characters and if you try to put anything else in it, that's an error. Other containers can hold any kind of item, for example lists and sets.

    {"prompt":"What is a container in kcats?","answer":"A container is a type whose value holds other items."}
    {"prompt":"True or false: in kcats a container is a type that holds other items.","answer":"True."}
    {"prompt":"How does kcats define a container type?","answer":"As a type whose instances contain other items."}
    {"prompt":"What happens if you put a non-character into a string in kcats?","answer":"It raises an error because strings are homogeneous containers for characters only."}
    {"prompt":"True or false: strings in kcats can store any type of item.","answer":"False. Strings are homogeneous and may only hold characters."}
    {"prompt":"Which container type is homogeneous, accepting only characters?","answer":"The string type."}
    {"prompt":"Which containers can hold any kind of item in kcats?","answer":"Lists and sets; they are heterogeneous containers."}
    {"prompt":"True or false: lists in kcats are restricted to a single item type.","answer":"False. Lists can contain items of any type."}
    {"prompt":"How do lists and sets differ from strings regarding item types they hold?","answer":"Lists and sets are heterogeneous and accept any item type, whereas strings accept only characters."}
    
  2. Lists

    Lists are multiple items bound up into a single unit, where their order is maintained. Lists are heterogenous and can accept any type, including other lists.

    1. Comprehension

      See the word 🪜, which converts a program into one that runs on each item in a list.

      0 [12 6 13 7 5] [+] 🪜 ▶️
      
      43
      

      Similar to 🪜, but more strict, is 🚜, which only allows the program to work on a given item and can't permanently alter the rest of the stack. Use that to transform each item in a list, in the same way (in this case showing the remainder when dividing by 5).

      [12 6 13 7 5] [5 mod] 🚜 ▶️
      
      [2 1 3 2 0]
      
      {"prompt":"What does a kcats list represent?","answer":"A single stack item that bundles multiple elements while preserving their order."}
      {"prompt":"True or false: A kcats list preserves the order of its elements.","answer":"True. Lists maintain the original ordering of their items."}
      {"prompt":"How is a group of items stored so their order stays intact in kcats?","answer":"They are placed in a list, which binds them into one ordered unit."}
      {"prompt":"Can kcats lists hold mixed types?","answer":"Yes. Lists are heterogeneous and may contain any type, including other lists."}
      {"prompt":"True or false: kcats lists must hold values of a single type.","answer":"False. They accept any mixture of types, even nested lists."}
      {"prompt":"Which feature lets a list in kcats contain numbers, strings, or sub-lists together?","answer":"List heterogeneity—any value type is allowed inside a list."}
      {"prompt":"What does the word 🪜 do in kcats?","answer":"It turns a program into one that runs on every item of a list (list comprehension)."}
      {"prompt":"Yes/No: 🪜 executes a given program separately on each element in a list.","answer":"Yes. 🪜 maps the program across the list items."}
      {"prompt":"How can you apply a program to each element of a list in kcats?","answer":"Use the word 🪜 to convert the program into an element-wise version."}
      {"prompt":"How does 🚜 differ from 🪜 in kcats?","answer":"🚜 also applies a program to each list item but forbids the program from permanently changing the rest of the stack."}
      {"prompt":"True or false: Programs run under 🚜 may modify items outside the current element.","answer":"False. 🚜 restricts programs so only the current element can be affected."}
      {"prompt":"Why is 🚜 described as more strict than 🪜?","answer":"Because it isolates each element’s run, preventing lasting changes to the surrounding stack."}
      {"prompt":"What common use does 🚜 have with respect to list elements?","answer":"It’s used to transform every item in a list, returning a list of the results."}
      {"prompt":"Yes/No: 🚜 can serve as a mapping operator over a list.","answer":"Yes. It applies a program to each element and collects the transformed items."}
      {"prompt":"When would you choose 🚜 for list processing?","answer":"When you need to apply the same operation to each element while keeping the rest of the stack intact."}
      
  3. Strings

    Strings are a container for character types, and work much like in other programming languages. Most of the functions that work on other containers will work on strings.

    "Hello World!" 📏
    
    12
    
    "Hello World!" first
    
    \H
    
    "Hello World" 0 5 slice
    
    "Hello"
    
  4. Bytes (byte array)

    Byte arrays are a sort of "lowest common denominator" data format. It's what you use to interact with files or sockets. Byte arrays are printed in base64 encoding and denoted as such with the #b64 tag. You can encode any item into bytes.

    "Hello World!" encode
    
    #b64 "SGVsbG8gV29ybGQh"
    
    [1 2 3] encode 
    
    #b64 "MSAyIDM"
    

    and you can treat those byte arrays as lists of integers:

    #b64 "SGVsbG8gV29ybGQh" 📤
    
    72 #b64 "ZWxsbyBXb3JsZCE"
    

    72 is the ASCII encoding for H.

  5. Associations

    An association is made from a list of pairs, like this:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    

    However there are some words you can use that make this list behave a bit differently than a normal list. For example:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [age] 25 assign
    
    [[age 25]
     [favorite-color "brown"]
     [name "Alice"]]
    

    Here we use assign to reset Alice's age - it does not simply add a new item to the list. It will find the existing key and replace it. It will create a new item only if the key didn't already exist:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [department] "Sales" assign 
    
    [[age 24]
     [department "Sales"]
     [favorite-color "brown"]
     [name "Alice"]]
    

    Note that the order of the items is not preserved. Here we take a key/value pair out of the association but which one we get is arbitrary:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [department] "Sales" assign 📤 
    
    [favorite-color "brown"]
    [[age 24]
     [department "Sales"]
     [name "Alice"]]
    

    Once you treat a list as an association, it "sticks" (see 5.5.2 for details). It acts like an association from then on, and order is no longer maintained.

    We can improve upon our example that incremented Alice's age (presumably after her birthday) with the word update. That will run a program on the item of whatever key (or keys) you specify.

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [age] [inc] update
    
    [[age 25]
     [name "Alice"]
     [favorite-color "brown"]]
    

    Note that associations and lists look the same when printed, but testing them for equality will reveal they are not the same:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [age] [inc] update
    
    [[name "Alice"]
     [age 25]
     [favorite-color "brown"]]
    
    =
    
    🔳
    

    Here we are comparing an association with a list. The = operator has no way of knowing whether you want the list semantics (which does care about order), or the association semantics (which doesn't care about order). It defaults to the more strict rules, so they are not equal.

    The act of using a list as an association (by applying words to it like assign or update) will convert it to an association, but what if you just want to convert a list to an association, without doing anything else?

    You can use the word association to convert the list to an association:

    [[name "Alice"]
     [age 24]
     [favorite-color "brown"]]
    
    [age] [inc] update
    
    [[name "Alice"]
     [age 25]
     [favorite-color "brown"]]
    
    association =
    
    {"prompt":"What structure is used to create an association in kcats?","answer":"A plain list of key/value pairs becomes an association."}
    {"prompt":"True or false: An association can start from a list of pairs.","answer":"True. A list of key/value pairs can be interpreted as an association."}
    {"prompt":"How is an association formed from basic data in kcats?","answer":"By writing a list of pairs such as [[key value] …]; the runtime can treat it as an association."}
    {"prompt":"What does the word \"assign\" do to an association entry?","answer":"assign finds the key and replaces its value, adding the pair only if the key is absent."}
    {"prompt":"True or false: assign always appends a new pair even when the key exists.","answer":"False. assign replaces an existing key’s value; it only adds a pair if the key is missing."}
    {"prompt":"How does assign differ from simply adding a pair to a list?","answer":"assign updates the value for an existing key instead of creating a duplicate; otherwise it inserts the pair."}
    {"prompt":"Does a kcats association preserve item order?","answer":"No. Associations are unordered and operations may rearrange pairs arbitrarily."}
    {"prompt":"True or false: Item order is preserved in an association.","answer":"False. Order is not maintained once the data is treated as an association."}
    {"prompt":"Why can’t you rely on element order within an association?","answer":"Because associations are unordered; their implementation does not keep pairs in original sequence."}
    {"prompt":"When you pop a pair from an association, which one is returned?","answer":"An arbitrary key/value pair; the language makes no guarantee about which pair you’ll get."}
    {"prompt":"Yes or no: Popping from an association yields a predictable first element.","answer":"No. The pair returned is arbitrary because the collection is unordered."}
    {"prompt":"Which pair will a pop operation return when used on an association?","answer":"Any pair—the choice is unspecified since associations don’t store an ordered sequence."}
    {"prompt":"What happens after you use assign or update on a list?","answer":"The list is promoted to an association and retains association behaviour thereafter."}
    {"prompt":"True or false: After assign, the structure still behaves like a normal list.","answer":"False. Once assign (or update) is used, it behaves as an association."}
    {"prompt":"Why does order stop mattering once you apply association words to a list?","answer":"Because the list is converted to an association, whose semantics ignore ordering."}
    {"prompt":"What is the purpose of the word \"update\" in an association?","answer":"update runs a supplied program on the value of the specified key(s) and stores the result."}
    {"prompt":"Yes or no: update executes a program on targeted association values.","answer":"Yes. update applies the provided code to the values of the given key(s)."}
    {"prompt":"How does update modify an association entry?","answer":"It executes the given program on the value for each specified key, replacing each with the new result."}
    {"prompt":"Do printed representations distinguish lists from associations?","answer":"No. They look identical when printed, even though they are different types."}
    {"prompt":"True or false: A list and an association with identical pairs print differently.","answer":"False. Their printed forms are the same, though they differ semantically."}
    {"prompt":"Why can equality testing reveal a difference unseen in printed output?","answer":"Because print formats match, but list and association types differ, so comparison operators can show inequality."}
    {"prompt":"Why does the = operator report a list and identical-looking association as unequal?","answer":"= defaults to order-sensitive list semantics, so the unordered association fails to match."}
    {"prompt":"True or false: = compares using association semantics by default.","answer":"False. It uses stricter list semantics unless told otherwise."}
    {"prompt":"How does the default behaviour of = affect equality between list and association?","answer":"It applies list rules that care about order, causing a list and an association with the same pairs to be unequal."}
    {"prompt":"What does the word \"association\" do when applied to a list?","answer":"It converts (promotes) the list into an association without altering its data."}
    {"prompt":"Yes or no: The word \"association\" can promote a list to an association.","answer":"Yes. Invoking association changes the list’s type to an association."}
    {"prompt":"How can you convert a list to an association without changing its data?","answer":"Use the word association; it promotes the list to an association in one step."}
    
  6. Sets

    Sets are made to test for membership, and do not care about order.

    ["Larry" "Curly" "Moe"] set "Moe" contains?
    

    If you add an item to a set, but it's already there, nothing changes.

    ["Larry" "Curly" "Moe"] set "Curly" 📮
    
    ["Curly" "Larry" "Moe"]
    

    You can take from a set but since order doesn't matter, you get an arbitrary item.

    1 20 1 range set 📤
    
    7 [1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 17 18 19]
    
5.5.1.5. Errors

See 5.15

5.5.1.6. Pipes

See 5.18

5.5.2. Promotion

Data types are automatically converted when needed.

For example, if you have a list of pairs and you use the word 🔍, it assumes your intention is to use the list as an associative data type, so it will be automatically converted, and remain converted after lookup completes.

You can often tell by the spec when the return type is a promoted type:

dictionary lingo [assign spec] 🔍
[[[item value]
  [list keys]
  sized]
 [association]]

Here you can see that the spec for assign takes a sized and returns an association. This allows you to do things like this:

[[name "Susie"] [age 25]] [sport] "bowling" assign
[[age 25]
 [name "Susie"]
 [sport "bowling"]]

The initial value of [[name "Susie"] [age 25]] is not an associative, it's just a list. You could explicitly convert it using the word association but assign will do it for you, because it is a function that operates on an associative type.

Note that the conversion can fail, because converting to associative requires that you have a list of pairs. If you don't, that's an error:

["foo" "bar"] [age] 25 assign
[[actual "foo"]
 [asked [pair]]
 [handled []]
 [reason "type mismatch"]
 [type error]
 [unwound [assign]]]
25 [age] ["foo" "bar"]

The most common promotion is from list to associative but there are others.

{"prompt":"How are data types handled when a word needs a different type in kcats?","answer":"Kcats automatically converts (promotes) the data to the required type."}
{"prompt":"True or false: kcats will automatically promote a value to another data type when appropriate.","answer":"True. The runtime performs implicit type conversion when needed."}
{"prompt":"Why can you usually supply a mismatched type to a kcats word and still succeed?","answer":"Because kcats performs automatic type promotion when the word expects another type."}
{"prompt":"What happens when you apply 🔍 to a list of pairs?","answer":"The list is implicitly converted to an associative type and stays associative after lookup."}
{"prompt":"Yes or no: After using 🔍, the list you queried remains an associative structure.","answer":"Yes. The implicit promotion is permanent for that value."}
{"prompt":"How does kcats obtain an associative from a list of pairs during lookup?","answer":"By auto-converting the list to an associative type when 🔍 is invoked."}
{"prompt":"How can you spot that a word returns a promoted type just by reading its spec?","answer":"The spec explicitly shows the promoted return type, signalling automatic conversion."}
{"prompt":"True or false: A kcats word’s spec can indicate that its result is a promoted type.","answer":"True. The type signature reveals when promotion occurs."}
{"prompt":"Why might you examine a word’s spec before using it with mixed data types?","answer":"Because the spec tells you whether the word will return a promoted type."}
{"prompt":"What will assign do if given a plain list of pairs instead of an association?","answer":"assign automatically converts the list to an associative type before updating it."}
{"prompt":"True or false: You must call association before using assign on a list of pairs.","answer":"False. assign promotes the list to an association for you."}
{"prompt":"How does assign let you skip explicitly calling association?","answer":"It implicitly converts a list of pairs to an associative type as part of its operation."}
{"prompt":"What error occurs if kcats tries to convert a non-pair list to associative?","answer":"A type mismatch error is raised because the conversion to associative cannot succeed."}
{"prompt":"Yes or no: Implicit promotion to associative always works, even on lists without pairs.","answer":"No. It fails with a type mismatch if the list is not made of pairs."}
{"prompt":"Why can implicit conversion to associative sometimes fail?","answer":"Because it requires a list composed solely of pairs; otherwise kcats throws a type error."}
{"prompt":"Which promotion is most common in kcats’ automatic conversions?","answer":"Converting a list to an associative type is the most frequent promotion."}
{"prompt":"True or false: List→associative is the only automatic promotion kcats performs.","answer":"False. It is the most common, but other promotions exist."}
{"prompt":"When is list-to-associative promotion typically encountered?","answer":"It is the standard and most frequent automatic type upgrade performed by kcats."}

5.5.3. Spec

Kcats lets you see what the input an output for a given word should look like. You can specify what kinds of data should be at each place on the stack, both before and after the word executes.

Let's look at an example, for + that adds numbers:

dictionary lingo [+ spec] 🔍 
[[number number]
 [number]]

That means, the input is two numbers, and the output is one number. In terms of stack locations, the top of the stack comes first in the input or output spec.

When the actual stack values don't match, we get an error due to the spec check:

"foo" "bar" +
[[actual "bar"]
 [asked [number?]]
 [handled 🔳]
 [reason "type mismatch"]
 [trace 🔳]
 [type error]
 [unwound [+]]]
"bar" "foo"

Currently only input specs are enforced, and all specs are optional (they default to [[][]] which means "don't look at any stack items".

Specs can get more complicated, for example, instead of just specifying a type number, they can give a helpful name like [number x]. Names that match should be the equal values - for example:

dictionary lingo [👥 spec] 🔍
[[[item a]] [[item a]
             [item a]]]

The word 👥 takes an item called a, and leaves two items called a. So we can see 👥 duplicates the item on top of the stack.

Sometimes the word executes another program on the stack, that's not known until runtime, and therefore we can't predict what effect it will have. Specs show this unknown effect with *. For example:

dictionary lingo [🪄 spec] 🔍
[[program [item a]]
 [[item a]
  *]]

In this case, all we know about the output is [item a] will be on the top of the stack, what's beneath that depends on what the program does.

5.5.4. Traits

There are words that operate on multiple types, and it's helpful to talk about what those types have in common. Specs use these traits to describe groups of types that a word will accept or produce.

5.5.4.1. Dispenser

Containers from which you can take out items, one by one. Includes:

  • Strings - dispenses Characters
  • Bytes - dispenses Integers
  • Lists - dispenses Items
  • Associations - dispenses key/value pairs
  • Sets - dispenses Items
  • Out Pipes - dispenses Bytes
  • Tunnels - dispenses Bytes

    Supported words: 📤

5.5.4.2. Receptacle

Containers into which you can put items, one by one. Includes:

  • Strings - accepts Character
  • Bytes - accepts Integer
  • Lists - accepts Item
  • Associations - accepts key/value pairs
  • Sets - accepts Item
  • In Pipes - accepts Bytes
  • Tunnels - accepts Bytes

Supported words: 📮

5.5.4.3. Sized

Containers whose items can be counted. Includes:

  • Strings
  • Bytes
  • Lists
  • Associations
  • Sets

Just list the names of the words that use sized, since there's a lot:

dictionary lingo [second [spec] 🔍
       first set [sized] 🍫 contains?] 🧲 ▶️ 
[first] 🚜 ▶️ 🔳 sort
[assemble assign butlast cut empty environment fail get intersection pad persist
 prepend rest sort sort-indexed splitter 💯 📏 🔍 🔗
 🧲 🧹]
5.5.4.4. Ordered

Containers whose items are kept in a specific order. Includes

  • Strings
  • Bytes
  • Lists
dictionary lingo [second [spec] 🔍
       first set [ordered] 🍫 contains?] 🧲 ▶️
[[slice [[definition builtin_slice]
         [examples [[["foobar" 0 3 slice] ["foo"] "Slice a string with valid indices"] [["foobar"
                                                                                         0 7 slice]
                                                                                        [🔳] "Slice a string with index past end -> Nothing"]
                    [["foobar" encode 0 3 slice] ["foo" encode]
                     "Slice a byte array with valid indices"]
                    [[[a b c d e] 0 3 slice] [[a b c]] "Slice a list with valid indices"]]]
         [namespace #b64 "core"]
         [spec [[integer integer ordered] [ordered]]]]]
 [interpose [[definition [🔳 flip [🔀 pair 🔗 [pop] 🛡️] 🪜 🗑️ pop 🗑️]]
             [examples [[[[foo bar baz] "hi" interpose] [[foo "hi" bar "hi" baz]] "Interpose string between words"]
                        [[🔳 "hi" interpose] [🔳] "Empty list is a no-op"] [[[foo] "hi" interpose] [[foo]]
                                                                          "Single item list is a no-op"]]]
             [spec [[item ordered]
                    [ordered]]]]]
 [first [[definition [📤 •🗑️]]
         [examples [[[[4 5 6] first]
                     [4] "Get the first item of a list"]
                    [["foo" first]
                     [\f] "The first item of a string is the first character"]
                    [[🔳 first]
                     [🔳] "The first item of an empty list is Nothing"]]]
         [spec [[ordered] [item]]]]]
 [pop [[definition builtin_pop]
       [examples [[[["a" "b" "c"] pop]
                   [["a" "b"]
                    "c"]
                   "Pop last string from list"]
                  [[[1 2 3] pop]
                   [[1 2]
                    3]
                   "Pop last number from list"]]]
       [namespace #b64 "core"]
       [spec [[ordered] [item ordered]]]]]
 [second [[definition builtin_second]
          [examples [[[[4 5 6] second]
                      [5] "Get second item of list"]
                     [["foo" second [\o]] "Get second item of string"]
                     [[🔳 second]
                      [🔳] "Get second item of empty list -> Nothing"]]]
          [namespace #b64 "core"]
          [spec [[ordered] [item]]]]]
 [last [[definition builtin_last]
        [examples [[[[3 4 5 6] last]
                    [6] "Get last item of list"]
                   [["foo" last [\o]] "Get last item of string"]
                   [[🔳 last]
                    [🔳] "Get last item of empty list -> Nothing"]]]
        [namespace #b64 "core"]
        [spec [[ordered] [item]]]]]
 [ends? [[definition [[reverse] both starts?]]
         [examples [[["abcd" "cd" ends?] [] "String ends with matching string"] [["abcd"
                                                                                    "" ends?]
                                                                                   [] "String ends with empty string"]
                    [["abcd" "bb" ends?] [🔳] "String doesn't end with non-matching string"] [[[1 2 3
                                                                                               4]
                                                                                              [3 4]
                                                                                              ends?]
                                                                                             [] "List ends with matching list"]]]
         [spec [[ordered ordered]
                [boolean]]]]]
 [reverse [[definition builtin_reverse]
           [examples [[[[1 2 3] reverse]
                       [[3 2 1]] "Reverse a list"]
                      [["123" reverse]
                       ["321"] "Reverse a string"]
                      [[["foo" "bar" "baz"] reverse]
                       [["baz" "bar" "foo"]] "Reverse a list of string"]]]
           [namespace #b64 "core"]
           [spec [[ordered] [ordered]]]]]]
{"prompt":"What are traits used for in kcats specs?","answer":"Traits group the types a word can accept or produce."}
{"prompt":"True or false: kcats traits describe groups of types accepted or produced by words.","answer":"True. A trait names the set of types a word works with."}
{"prompt":"Why does kcats introduce traits in its specifications?","answer":"To concisely express shared properties of several types so words can be specified once."}
{"prompt":"What does the Dispenser trait represent?","answer":"A container you can take items out of one at a time."}
{"prompt":"Is a Dispenser a container you remove items from individually?","answer":"Yes. Dispensers let you take items one by one."}
{"prompt":"How is a Dispenser described in kcats?","answer":"As a container from which items can be taken one by one."}
{"prompt":"Name some types that are Dispensers.","answer":"Strings, Bytes, Lists, Associations, Sets, Out Pipes and Tunnels are all Dispensers."}
{"prompt":"True or false: Associations are Dispensers.","answer":"True. Associations belong to the Dispenser trait."}
{"prompt":"Which trait includes Strings, Bytes and Out Pipes?","answer":"The Dispenser trait."}
{"prompt":"Which word operates on Dispensers?","answer":"📤"}
{"prompt":"True or false: 📤 expects its argument to be a Dispenser.","answer":"True. 📤 works only with Dispenser containers."}
{"prompt":"Which trait must a container satisfy to be used with 📤?","answer":"Dispenser."}
{"prompt":"What is the Receptacle trait?","answer":"A container you can put items into one at a time."}
{"prompt":"Does the Receptacle trait describe containers you can insert into individually?","answer":"Yes."}
{"prompt":"How is a Receptacle defined in kcats?","answer":"As a container into which you can place items one by one."}
{"prompt":"List some types that are Receptacles.","answer":"Strings, Bytes, Lists, Associations, Sets, In Pipes and Tunnels are Receptacles."}
{"prompt":"True or false: In Pipes are Receptacles.","answer":"True. In Pipes belong to the Receptacle trait."}
{"prompt":"Which trait groups Strings, Sets and Tunnels for insertion operations?","answer":"Receptacle."}
{"prompt":"Which word operates on Receptacle containers?","answer":"📮"}
{"prompt":"True or false: The word 📮 requires a Receptacle.","answer":"True. 📮 only works with Receptacle containers."}
{"prompt":"Which trait is expected by 📮?","answer":"Receptacle."}
{"prompt":"What does the Sized trait indicate?","answer":"That a container’s items can be counted."}
{"prompt":"Is a Sized container one whose items are countable?","answer":"Yes."}
{"prompt":"When do you use the Sized trait?","answer":"When the container’s element count matters because its size can be measured."}
{"prompt":"Which containers are marked Sized?","answer":"Strings, Bytes, Lists, Associations and Sets are all Sized."}
{"prompt":"True or false: Sets are part of the Sized trait.","answer":"True. Sets are Sized containers."}
{"prompt":"Which trait covers Strings, Bytes and Associations for counting operations?","answer":"Sized."}
{"prompt":"Why does the doc only list the names of words that use Sized?","answer":"Because so many words use the Sized trait that only their names are shown."}
{"prompt":"True or false: There are so many Sized words that only their names are shown.","answer":"True. The list is long, so only names are provided."}
{"prompt":"How does the documentation handle the large number of words using Sized?","answer":"It lists only the names of those words instead of full details."}
{"prompt":"What defines the Ordered trait?","answer":"Containers whose items are kept in a specific order."}
{"prompt":"Is an Ordered container required to maintain item order?","answer":"Yes."}
{"prompt":"Why is a container labelled Ordered?","answer":"Because its elements have a defined sequence."}
{"prompt":"Which types are Ordered?","answer":"Strings, Bytes and Lists are the Ordered types."}
{"prompt":"True or false: Strings are part of the Ordered trait.","answer":"True. Strings are Ordered."}
{"prompt":"Which trait groups Strings, Bytes and Lists for order-sensitive words?","answer":"Ordered."}

5.6. Stack motion

Often you have all the data a word needs on the stack, but it's in the wrong order. There's lots of handy words to help there.

🔀
swap the top two items
🛟
float the 3rd item up to the top
sink the top item down to 3rd
flip

reverse the top 3 items

These words can also be combined with 🪄 and its variants to reach deeper into the stack.

{"prompt":"What are stack motion words used for in kcats?","answer":"They reorder items already on the stack when their order is wrong."}
{"prompt":"True or false: Stack motion words help rearrange data already on the stack.","answer":"True—stack motion helpers exist to reorder existing stack items."}
{"prompt":"Why would you use kcats stack-motion helpers?","answer":"To get arguments into the right order by shuffling the current stack contents."}
{"prompt":"What does 🔀 do in kcats?","answer":"It swaps the two items at the top of the stack."}
{"prompt":"Yes/no: Does 🔀 exchange the two items at the top of the stack?","answer":"Yes—it swaps the top pair."}
{"prompt":"How can you swap the top two stack values?","answer":"Use 🔀."}
{"prompt":"What does 🛟 do in kcats?","answer":"It brings the third stack item up to the top."}
{"prompt":"True or false: 🛟 brings the third stack element to the top.","answer":"True—🛟 floats the 3rd item upward."}
{"prompt":"How do you float the 3rd item to the top?","answer":"Use 🛟."}
{"prompt":"What does ⚓ do in kcats?","answer":"It pushes the top stack item down to the third position."}
{"prompt":"Yes/no: Does ⚓ push the top value down to the third spot on the stack?","answer":"Yes—⚓ sinks the top item to 3rd."}
{"prompt":"Which word moves the current top value to the third place?","answer":"⚓."}
{"prompt":"What does flip do in kcats stack motion?","answer":"It reverses the order of the top three stack items."}
{"prompt":"True or false: flip reverses the top three stack elements.","answer":"True—flip inverts the order of the first three values."}
{"prompt":"How do you reverse the order of the first three stack values?","answer":"Use flip."}
{"prompt":"How can you reach deeper into the stack with stack-motion words?","answer":"Combine them with 🪄 and its variants, which extend their reach."}
{"prompt":"Yes/no: Can you combine stack motion words with 🪄 to access deeper stack positions?","answer":"Yes—pairing with 🪄 lets them touch deeper items."}
{"prompt":"Which trick allows stack motion helpers to work deeper than three items?","answer":"Using 🪄 (and its variants) alongside them reaches further down the stack."}

5.7. Cloning and dropping

When you're done with an item, you can 🗑️ it, which eliminates it from the top of the stack. If you know a word will consume an item you need afterward, you can 👥 it so you have an extra copy.

5.8. Programs that write programs

5.8.1. Basics

The most important expressive feature of kcats is that you can manipulate programs exactly the same way as you can any other data.

One thing you can do with a list, is treat it like a program and ▶️ (execute) it. Notice that on the 5th and 6th line of the execution trace below, the word ▶️ takes the list from the top of the stack on the left, and puts its contents back on the right, making it part of the program remaining to be run!

;;   stack  |  remaining program
;; ---------|--------------------
            | 4 5 6 [* +] ▶️ inc
          4 | 5 6 [* +] ▶️ inc
        4 5 | 6 [* +] ▶️ inc
      4 5 6 | [* +] ▶️ inc
4 5 6 [* +] | ▶️ inc
      4 5 6 | * + inc
       4 30 | + inc
         34 | inc
         35 |

Note that, when * + gets moved back to the program, it went in front of inc.

The same way we used 🔗 to combine two lists, we can combine two small programs into one, and then ▶️ it:

4 5 6 [+] [*] 🔗 ▶️ 
44

Note that words inside lists don't perform any action when the list is put on the stack. You can think of it as a quotation - a message being being passed along, not acted upon.

One important theme of programming kcats is combining program snippets in various ways, and then ▶️ them to actually carry them out. There are lots of program "modifiers" to help.

{"prompt":"What is the most important expressive feature of kcats?","answer":"You can manipulate programs exactly the same way as any other data."}
{"prompt":"True or false: In kcats programs are manipulated just like data.","answer":"True. Programs are treated as ordinary data values."}
{"prompt":"How does kcats treat the relationship between code and data?","answer":"It blurs it; programs are handled identically to regular data."}
{"prompt":"How do you execute a list as a program in kcats?","answer":"Use the ▶️ word to run the list on top of the stack."}
{"prompt":"Yes or no: Does ▶️ execute the list on the stack?","answer":"Yes. ▶️ treats the list as code and runs it."}
{"prompt":"Which kcats word turns a list into running code?","answer":"▶️"}
{"prompt":"What does ▶️ do with a list when it executes it?","answer":"Takes the list off the stack and splices its contents in front of the remaining program."}
{"prompt":"True or false: ▶️ appends list contents after the program.","answer":"False. It inserts them before the next instructions."}
{"prompt":"How are a list's elements integrated into the running program by ▶️?","answer":"They are placed at the front, so they run before the following words."}
{"prompt":"How can you run two small programs together in kcats?","answer":"Link their lists with 🔗, then execute the combined list with ▶️."}
{"prompt":"Yes or no: Can 🔗 join two programs that you then run with ▶️?","answer":"Yes. 🔗 merges the lists; ▶️ executes the result."}
{"prompt":"Which sequence merges two program lists and then executes them?","answer":"Use 🔗 to combine them, followed by ▶️ to run."}
{"prompt":"What is the purpose of the 🔗 word in kcats?","answer":"It concatenates two lists or program snippets into one list."}
{"prompt":"True or false: 🔗 can be used to merge two lists into a single list.","answer":"True. 🔗 links its two list arguments together."}
{"prompt":"How do you merge two lists in kcats before further processing?","answer":"Apply 🔗 to concatenate them into one list."}
{"prompt":"What happens to words inside a list when the list is merely on the stack?","answer":"They do nothing; the list acts as a quotation until executed with ▶️."}
{"prompt":"True or false: Words inside a list execute automatically when the list is pushed.","answer":"False. Lists are inert quotations until ▶️ is used."}
{"prompt":"Why are lists compared to quotations in kcats?","answer":"Because their contents are passed around without action until explicitly executed."}
{"prompt":"What recurring theme does the kcats manual highlight about building code?","answer":"Composing program snippets with modifiers and then executing them with ▶️ is a central practice."}
{"prompt":"Yes or no: Is combining snippets and firing them with ▶️ fundamental in kcats?","answer":"Yes. Composition plus ▶️ lies at the heart of the language."}
{"prompt":"How does kcats encourage program composition?","answer":"It offers modifiers that let you build snippets, link them, and execute them with ▶️."}

5.8.2. Modifiers

What are modifiers? They are programs that modify other programs.

Here's an example from everyday life: "When you're following the cake recipe, if any ingredients are missing, go to the bake shop on Main Street to get them. And when it calls for brown sugar, use molasses instead". You're taking the existing instuctions (the recipe of how to make a cake), and wrapping it in larger instructions that specify things outside the scope of the recipe (where to get ingredients) and also change the recipe (substitute ingredients). When you follow instructions in everyday life, you're running a program, and we routinely find modifiers to programs out there in written English instructions.

Let's look at some building blocks of kcats that modify existing programs.

5.8.2.1. Looping and branching
  1. ⚖️

    ⚖️ takes 3 programs from the stack:

    • a condition program whose result decides which branch to take
    • the yes branch
    • the no branch

    It returns a new single program that handles the condition test and logical branching:

    [126 18 mod zero?] ["divides evenly!"] ["doesn't divide!"] ⚖️
    
    [126 18 mod zero? ["divides evenly!"] ["doesn't divide!"] ↔️]
    

    The resulting program runs the actual test, which yields a boolean value, and then calls the branching word ↔️. That word selects which of the two programs to run based on the test result. So what happens when we execute that program?

    [126 18 mod zero? ["divides evenly!"] ["doesn't divide!"] ↔️] ▶️
    
    "divides evenly!"

    Great, so what is that checkmark? Why is that there? That's the affirmative result from zero? which is still on the stack:

    0 zero?
    

    Our modifier ⚖️ does not drop this result. Why keep it? Remember any value except an empty list is affirmative, so there could have been important data there that we'd need later. See booleans.

    Let's look at one more example, where we want to test an argument and then do something with it (in this case, add one to a number if it's odd, or multiply it by 3 if it's even).

    3 [odd?] [🗑️ inc "odd, added one"] [🗑️ 3 * "even, times 3"] ⚖️
    
    [odd? [🗑️ inc "odd, added one"] [🗑️ 3 * "even, times 3"] ↔️] 3 2 1
    

    then if you run that, you get

    3 [odd?] [🗑️ inc "odd, added one"] [🗑️ 3 * "even, times 3"] ⚖️ ▶️ 
    
    [[asked [consume]]
     [handled []]
     [reason "not enough items on stack"]
     [type error]
     [unwound [inc "odd, added one"]]]
    

    Why didn't this work? The problem is odd? consumes its argument:

    3 odd?
    

    The original number gets lost after we check whether it's odd. We can prevent that in several ways. One is with 👥:

    3 👥 odd?
    
    ✅ 3
    

    Using 👥 can be a little fiddly, especially when multiple items need to be preserved.

    A more comprehensive method is to use the 🛡️ modifier. See 5.11 for how it works.

    3 [odd?] 🛡️
    [🗑️ inc "odd, added one"]
    [ 3 * "even, times 3"] ⚖️ ▶️
    
    "odd, added one" 4
    

    Finally, why do we need the 🗑️? Recall that the result of the condition check is not automatically dropped if the result is affirmative, because often your condition check is "is there anything to process". In that case your condition program can just focus on trying to get the input and not worry about checking whether there's anything there - ⚖️ does that for you just due to how boolean values work.

    ["foo" "bar" "baz"] ;; start with a list on stack
    [📤]  ;; results in an item from the list, or nothing if list is empty
    ["!!!" 🔗] ;; add excitement to the item if there was one
    🔳 ;; if not we're done
    ⚖️ ▶️ 
    
    "foo!!!" ["bar" "baz"]
    

    Or if the list actually is empty:

    🔳 ;; start with empty list on stack
    [📤]  ;; results in an item, or nothing if list is empty
    ["!!!" 🔗] ;; add excitement to the item if there was one
    🔳 ;; if not we're done
    ⚖️ ▶️ 
    
    []
    

    Notice that ⚖️ does in fact drop the condition result if it's negative. In this case the result of 📤 on an empty list is another empty list. That 2nd empty list is dropped by the ⚖️ logic and we're left with just the original empty list. The reason this value is dropped is that it's just an empty container and there's little to be gained by examining it, we essentially already know what it was, just as a result of reaching this code snippet.

    {"prompt":"What three programs must be on the stack before calling ⚖️?","answer":"A condition program, the yes branch program, and the no branch program."}
    {"prompt":"True or false: ⚖️ expects exactly three programs on the stack.","answer":"True. It needs the condition, yes branch, and no branch programs."}
    {"prompt":"Which elements does ⚖️ pull from the stack to build its result?","answer":"It takes the condition program plus the yes and no branch programs."}
    {"prompt":"What does ⚖️ return after consuming the three input programs?","answer":"A single program that runs the test and then branches accordingly."}
    {"prompt":"Yes or no: ⚖️ outputs a program that already includes the branching logic.","answer":"Yes. The returned program performs the test and chooses a branch."}
    {"prompt":"How does ⚖️ combine its inputs?","answer":"It packages them into one program that executes the test and selects a branch."}
    {"prompt":"Which word does the program built by ⚖️ use to choose between the yes and no branches?","answer":"It calls the branching word ↔️ to select the branch."}
    {"prompt":"True or false: After evaluating the condition, ⚖️ directly runs the yes branch.","answer":"False. It feeds the result to ↔️, which picks the branch."}
    {"prompt":"How is the branch selected inside the program produced by ⚖️?","answer":"The condition result is passed to ↔️, which runs the appropriate branch."}
    {"prompt":"What happens to a truthy condition result after ⚖️ evaluates it?","answer":"The affirmative result stays on the stack; ⚖️ does not drop it."}
    {"prompt":"Yes or no: ⚖️ discards the condition result when it is truthy.","answer":"No. Truthy results are kept on the stack."}
    {"prompt":"Why does ⚖️ keep truthy condition results?","answer":"Because any non-empty value may contain data needed later."}
    {"prompt":"What does ⚖️ do with a false (empty list) condition result?","answer":"It removes the negative result (empty list) from the stack."}
    {"prompt":"True or false: ⚖️ keeps an empty list result after branching.","answer":"False. Negative (empty list) results are dropped."}
    {"prompt":"When is the condition result discarded by ⚖️?","answer":"Only when the result is negative, i.e., the empty list."}
    {"prompt":"Which word can you use to duplicate an item before a condition test to prevent it being consumed?","answer":"Use 👥 to copy the item."}
    {"prompt":"True or false: 👥 helps preserve data that a condition test would otherwise consume.","answer":"True. 👥 duplicates the data before the test."}
    {"prompt":"How can you keep an argument after running an input-consuming predicate like odd??","answer":"Duplicate it with 👥 before calling the predicate."}
    {"prompt":"What modifier offers a comprehensive way to preserve multiple stack items during a condition test?","answer":"The 🛡️ modifier."}
    {"prompt":"Yes or no: 🛡️ can protect several values from being consumed by a condition program.","answer":"Yes. 🛡️ shields multiple stack items."}
    {"prompt":"Which technique is recommended over repeated 👥 when many items must be preserved?","answer":"Wrap the condition program with 🛡️."}
    {"prompt":"Which word do you insert to explicitly discard an unwanted truthy condition result?","answer":"Use 🗑️ to drop the value."}
    {"prompt":"True or false: You should use 🗑️ when you don't need the kept affirmative result.","answer":"True. 🗑️ discards the unneeded value."}
    {"prompt":"How do branch programs remove the leftover condition value that ⚖️ keeps?","answer":"They call 🗑️ to drop the leftover result."}
    
  2. 🪆

    Recurrence (🪆) is the fundamental looping construct of kcats. Every other loop modifier is built from 🪆.

    It takes 4 arguments:

    • A predicate program
    • An body program
    • An exit program
    • A combinator program

    The result is a single recurrence program on the stack that when executed with ▶️, may place another copy of itself on the stack. For emphasis: 🪆 builds a program that loops, it does not execute the program.

    The recurrence program, when executed, works as follows: first the predicate program runs (as if by ⚖️). If the predicate result is negative, the exit program runs, and then the whole program is finished.

    If the predicate result is affirmative:

    • The body program is executed
    • A copy of the original recurrence program is placed on the stack
    • The combinator is executed

    In typical loop use cases, the combinator program is simply [▶️]. That means, "execute the recurrence again". However some combinators might do fancier things, like use 🚜, to mean "run the recurrence program once for each item in the list below, collect the results".

  3. 🌀

    Looper (🌀) takes a loop body program, and returns a loop program, which may execute the body multiple times. The loop program expects a boolean (See 5.5.1.2) condition on the top of stack. If the condition is negative, the body doesn't run, and the loop is finished. If it's affirmative, the body runs, and the loop expects another boolean condition to be on top of the stack for the next iteration.

    Here's an example:

    1 ✅ [🗑️ 2 * 👥 100 <] 🌀 ▶️
    
    128
    

    Notice that 🌀 receives the body program and builds the loop program. The body program receives ✅ (affirmative) the first iteration. The body program sees the , and doesn't need it, so it drops it bringing 1 to the top. The body then multiplies by 2, clones it and checks if it's less than 100 (note you could write [100 <] 🛡️ in place of 👥 100 <). If so, the body runs again, and so on, until the number is not less than 100. Finally that false value is dropped automatically and the loop is done, leaving just the final number 128.


  4. Kcats also has a while loop (⏳), which is a bit higher level than 🌀. ⏳ is to 🌀, as ⚖️ is to ↔️. Instead of expecting a boolean value on top each time through, you provide a condition program similar to what ⚖️ requires. ⏳ runs the condition program, if it leaves a affirmative value, the loop continues, otherwise it is finished.

    1 [100 <] 🛡️ [🗑️ 2 *] ⏳ ▶️ 
    
    128
    
  5. until

    It's just like ⏳, but with the condition's logic reversed, so that it stops when the condition is true.

    1 [100 >=] [2 *] until
    
    128
    

    Unlike (which runs the body 0 or more times), until will always run it at least once.

    1 [][2 *] until
    
    2
    

5.9. Argument order

Kcats' stack-based nature can take a little getting used to, and the reversing of the order you wrote something can be a common stumbling block.

When writing programs like ⚖️, the arguments appear in a natural order in your code: conditional, true-branch, false-branch. However, remember that the stack is a last-in-first-out structure - when these items are pushed onto the stack, their order is reversed. If you inspect the stack at this point, you'll see the false-branch on top, followed by the true-branch, and then the conditional at the bottom.

1 2 3 [🗑️ odd?] ["it's odd"] ["it's even"] ;; ⚖️
["it's even"] ["it's odd"] [🗑️ odd?] 3 2 1

This is a theme in kcats, where argument order is designed to make the code readable - if a word takes multiple arguments, and the order matters, the "first" logical argument is not the top of the stack. Here's an example:

1 2 <

When we write 1 2 <, we mean "1 is less than 2". Even though the top of the stack is 2, we don't consider 2 the "first" argument.

5.10. Item hiding

Sometimes you have a program that you don't trust with a certain stack item. Perhaps there's a password on the stack, and you're running an untrusted program given to you by someone else.

What if there was a way to hide that password behind your back such that the program never even knew it was there, and then restore it after the untrusted program was finished?

🪄 takes an item on the top of the stack, and a program. It temporarily makes the item disappear, and runs the program. After the program is done, the item reappears on top of the stack.

1 2 "mypassword" [+] 🪄
"mypassword" 3

Notice the addition program could not access the password even if it tried. It isn't on the stack while it's executing, it's hidden away elsewhere in the runtime, temporarily.

To demonstrate we can use the word 📸, which takes a snapshot of the entire stack and places it on top of the stack.

"foo" "bar" "hidden!" [📸] 🪄 
"hidden!" ["bar" "foo"] "bar" "foo"

So here we see the snapshot ["bar" "foo"], the word "hidden" nowhere to be found. That's because when the snapshot was taken, it was hidden away and wasn't anywhere on the stack. Then "hidden" is placed back on top after the snapshot is done.

🪄 is very common in kcats, and it's used mostly in cases where you don't actually care if a program reads an item, you just want the item out of the way temporarily, and it's easier than finicky swapping. However in cases where there is a trust issue, no amount of swapping can fix the problem; you definitely should reach for 🪄.

5.11. Stack effect control

Kcats provides some facilities to let you avoid tedious cloning of items to keep from losing them. Most words consume items from the stack to produce new items. Sometimes you'll still need those old items again later.

Let's say you want to check if two items are equal, and if not, add them to a list.

Naively you might try this:

[] 5 6 [=] [] [[📮] 🪄 📮] ⚖️ ▶️
[[asked [consume]]
 [handled 🔳]
 [reason "not enough items on stack"]
 [trace [🪄]]
 [type error]
 [unwound [📮 🔳 📮]]]

The problem is that = consumes both the numbers, just leaving a boolean. What if we could specify that the program [=] isn't allowed to permanently consume any stack items, it's just allowed produce a result?

That's what 🛡️ does.

5 6 [=] 🛡️ ▶️ 
🔳 6 5

So we can just drop 🛡️ into the original program right after the program we want to modify, and that should fix things:

[] 5 6 [=] 🛡️ [] [[📮] 🪄 📮] ⚖️ ▶️
[5 6]

5.12. Down stack variants

There are words like •🪄, •🛡️, •🔀, •🗑️, •🐋. What are those?

It's a modification of the original where the effect is one stack element further down from the original. Each dot represents a stack item that is above where the word has its effect (or where the word ends its effect). What exactly is further down, depends on the word.

•🔀
swap the 2nd and 3rd stack items (instead of 1st and 2nd).
•🪄
hide the top two stack items (instead of just the 1st).
•🛡️
Protect all but the top stack item (instead of the whole stack).
•🗑️
drops the 2nd item (instead of the top item)
•🐋
Skips over top 2 stack items and returns result to top (instead of skipping only top item)
•📮
Puts the top item into the 3rd item

Similarly the deep variants are one level even deeper than that:

••🔀
swap the 3rd and 4th items
••🪄
hide the top 3 items
••🛡️
protect all but the top two items
••🗑️
drops the 3rd item
••📮
Puts the top item into the 4th item

5.13. Templating and destructuring

Templating and destructuring are two opposite operations. Templating takes a strucutred template that contains placeholders, and some values, and puts the values in the correct place in the template. Conversely, matching takes a filled-in structure, and a pattern containing placeholders, and extracts values back out from the filled-in structure according to the pattern.

5.13.1. Templating with 🎒

Let's take a look at a simple example:

"foo" "bar" "baz"
[[a 1️⃣]
 [b 2️⃣]
 [z 3️⃣]]
🎒 
[[a "baz"]
 [b "bar"]
 [z "foo"]]

Notice that the numbered placeholders 1️⃣ 2️⃣ etc refer to places on the stack: 1️⃣ is the top, 2️⃣ is the 2nd stack item etc. 🎒 takes values from the stack and inserts them into the template.

5.13.2. Destructuring directly to stack with #️⃣⛏️

Again, let's look at a simple example, that calculates the total cost of an inventory of smartphones.

[[smartphone [[pricing [[quantity 2]
                        [discount 0.15]
                        [price 699.00]]]
              [description "Worlds most advanced smart phone"]]]
 [laptop [[pricing [[quantity 4]
                    [discount 0.10]
                    [price 1699.00]]]
          [description "Worlds thinnest laptop computer"]]]] assocify

[[smartphone
  [[pricing [[price 1️⃣]
             [discount 2️⃣]
             [quantity 3️⃣]]]]]] assocify 
#️⃣⛏️ * * 
209.7

Notice that here we're doing the opposite of 🎒 - extracting values back out of the filed in structure and putting them on the stack. We mark the items to be extracted using numbered markers, that correspond to places on the stack.

But why do we need to call assocify here? Answer: If we don't, then kcats does list matching, where order matters. What happens if we forget assocify?

[[smartphone [[pricing [[quantity 2]
                        [discount 0.15]
                        [price 699.00]]]
              [description "Worlds most advanced smart phone"]]]
 [laptop [[pricing [[quantity 4]
                    [discount 0.10]
                    [price 1699.00]]]
          [description "Worlds thinnest laptop computer"]]]] 

[[smartphone
  [[pricing [[price 1️⃣]
             [discount 2️⃣]
             [quantity 3️⃣]]]]]]  
#️⃣⛏️ * * 
[[asked [price quantity =]]
 [handled 🔳]
 [reason "match failed"]
 [trace [#️⃣⛏️]]
 [type error]
 [unwound [⛏️ 🗑️ reverse 🍫 * *]]]

Here we see a mismatch because in the pricing association, in the data quantity is first, but in the pattern, price is first. The error message is telling us what specifically didn't match.

Wildcards are supported inside lists. Use the word _ to indicate that any value should match at that position:

[a 12 "foo" "mydata" 13]
[a _ _ 1️⃣ 13]
#️⃣⛏️
"mydata"

Wildcards aren't used in associations - if you want to match "any value" for an association key, just leave it out of the pattern. Matching "any key" for a given value, is not supported.

5.13.3. Destructuring to an association

If you want to destructure a bunch of values, or just don't want to put multiple values onto the stack, you can also destructure into an association. Instead of numbered placeholders, you'll use names. But in order to differentiate between placeholders and other words, you'll need to prefix the word with the tag character 🏷️, and use the word 🏷️⛏️ to execute the destructuring.

[[name "Alice"]
 [age 25]
 [sex f]
 [address [[street "Main St."]
           [number 123]
           [zip 12345]]]] assocify
[[name 🏷️cust-name]
 [age 🏷️cust-age]
 [address [[zip 🏷️cust-zip]]]] assocify
🏷️⛏️
[[cust-age 25]
 [cust-name "Alice"]
 [cust-zip 12345]]

5.14. Programmable runtime

5.14.1. Overview

One unusual and important feature of kcats is that you can program the runtime. What does that mean? It means you can control exactly how a program is executed.

Why would you want to do that? Well, one common use case is that you want to debug the program: you'd like to manually control execution while you examine the state of the program, to figure out where it's going wrong.

Another use case is security: you want to execute a program but not allow it to access things that programs normally can access (like the filesystem or network). You also don't want the program to be able to permanently redefine what words mean.

So how exactly do we alter how programs are executed? A kcats program keep track of 3 things as it is running, its state: the stack (the data the program is working with), the program (the remaining instructions left to be executed), and the dictionary (the words that have meaning in a program). We call the whole state an environment, and we can create and work with environments in kcats just like any other data, including executing the progams within them.

5.14.2. Environments

First let's look at how to create an environment:

[[program [1 2 +]]] environment
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program [1 2 +]]
 [stack 🔳]]

Notice how `environment` takes an association and fills out the `dictionary` and `stack`. Why does it say `270entries`? Normally dictionaries would be printed out in their entirety, just like any other data, but because they're large (there are hundreds of words) by default the runtime prints it as the count of the words.

So this is an environment. What words are useful here? Well, first of all we can treat it as an association, which it is. Here we replace the last item in the program so we're subtracting instead of adding.

[[program [1 2 +]]] environment
[program] [pop 🗑️ [-] 🔗] update
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program [1 2 -]]
 [stack 🔳]]

Ok, well, this is not that exciting, if environment is just another kind of association right? Point taken, but now let's use the word `eval-step`:

[[program [1 2 +]]] environment
eval-step
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program [2 +]]
 [stack [1]]]

That evaluates the environment one step! We can do two more steps to finish the program:

[[program [1 2 +]]] environment
[eval-step] 3 times ▶️ 
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program 🔳]
 [stack [3]]]

5.14.3. Debugging

5.14.3.1. The debugger

In kcats, we don't need an external debugger. We can debug our programs right in the kcats interpeter. We can specify the program to run and step through it.

Let's say this is the program we want to step through. This is how we'd normally run it:

0 1 3 inc 1 range [+] 🪜 ▶️
6

To debug, we put it into an environment item which we can then use debugging words like advance:

[[program [0 1 3 inc 1 range [+] 🪜 ▶️]]] environment
[advance] 8 times ▶️
eval-step
[advance] 2 times ▶️ eval-step eval-step
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program [🔀 [+] 👥 •🪄 🪜 ▶️]]
 [stack [1 [2 3]
         0]]]

Note that advance is like step-over in a traditional debugger, and eval-step is like step-into. So above we advance until we reach the word step in the program, and then we step into it. We end up showing the environment in the middle of execution. The stack has a program [+] on top, and the next word is execute which will run that program.

You can also use a until loop to run the program until an arbitrary condition is hit. Here's one that runs the program until the number =2 is on the top of stack (note the handy word tos shortcut)

[0 1 3 inc 1 range [+] 🪜 ▶️] stage
[tos 2 =] 🛡️ [🗑️ eval-step] until ▶️
[[dictionary [[modules [#b64 "core"]]
              [words 274_entries]]]
 [program [🔀 [+] 👥 •🪄 🪜 ▶️]]
 [stack [2 [3] 1]]]

You can do whatever you want with the environment data - you can retain the environment at every step, filter the steps, change them, and continue the execution from any arbitrary place.

It's particularly handy to save an environment at the "last known good" state and continue from there, instead of having to re-execute from the beginning each time.

Just as an example of what's possible, here we show only the states where + is about to be executed. stepper generator takes an environment and generates all the steps of execution. Note top means "top of program", so it's keeping the states where + is the next item in the program. We drop the dictionary from each one, because we don't need it and in this case it's always the same.

[0 1 3 inc 1 range [+] 🪜 ▶️] stage 
stepper generator

[top 🎁 [+]  = ] keep
;[[dictionary] unassign] each
collect 
🔳 🔳

Then just to show that all these environments work on their own, we'll add some code to select the first one and step it forward. So we've essentially gone back in time and rolled forward again.

[0 1 3 inc 1 range [+] 🪜 ▶️] stage 
stepper generator
[top 🎁 [+] =] keep
collect
first eval-step
[[dictionary [[lingo 270_entries]
              [modules [#b64 "core"]]
              [words 270_entries]]]
 [program [[2 3]
           [+] 🪜 ▶️]]
 [stack [1]]]
🔳

We can even mess with the stack and the program:

[0 1 3 inc 1 range [+] 🪜 ▶️] stage 
stepper generator
[top 🎁 [+] =] keep
collect

;; add this
first ;; to select the first env from above
[program 0] [-] 🍫 assign ;; change + to - right before it is run

;; now step forward again
eval-step
[[dictionary [[lingo 265_entries]
              [modules [#b64 "core"]]
              [words 265_entries]]]
 [program [[2 3] [+] 🪜 ▶️]]
 [stack [-1]]]
🔳
5.14.3.2. Other tools

The "good old fashioned" method of debugging a program is to put print statements throughout the program to get insight into what's happening.

You can do that in kcats too!

3 [odd?] 🛡️ [🗑️ "it was odd" print inc] ["it was even" print 2 *] ⚖️ ▶️
it was odd
4

print takes a string and will print it to standard out.

There's also a way to print the current stack, it's called dump, and it has has no stack effect, so it's safe to put anywhere.

1 2 3 [odd?] 🛡️ [🗑️ dump inc] [dump 2 *] ⚖️ ▶️
[3 2 1]
4 2 1

5.15. Error handling

In kcats, when a program encounters an error, two things happen: (1) an error item describing the problem is placed on the stack, and (2) the remaining program operations that haven't executed yet are "unwound" (collected into the error's unwound field rather than being executed). This allows programs to both examine what went wrong and potentially resume execution using the saved operations.

2 3 "four" * + 
[[actual "four"]
 [asked [number?]]
 [handled 🔳]
 [reason "type mismatch"]
 [type error]
 [unwound [* +]]]
"four" 3 2

Notice the unwound field contains the rest of the program that remained when the error occurred.

We can fix the problem and continue, but only if we can stop the unwinding before our entire program is unwound. We can do that using the word 🩹, which creates a self-recovering program. It takes two component programs: p and r. p is run and if it results in an error, the unwinding is limited to p and then r is run. When r runs, the error item is on the top of stack. If there is no error, r does not run.

In the program below, we recover by discarding the error and the string "four", and replacing it with the number 4. Then trying the operations * + again.

2 3 "four" [* +] [🗑️ 🗑️ 4 * +] 🩹 ▶️ 
14

The problem with the usage of 🩹 above is that we had to specify the arithmetic words * + twice - once in p and again in r in case they failed the first time. Remember those operations are saved in the unwound field of the error, and we can access them and even ▶️ them. There is a word that does this for you: retry: it takes an error on the top of stack, and executes its unwound program.

2 3 "four" [* +] [[🗑️ 4] 🪄 retry] 🩹 ▶️
14

In the above program, after the error occurs, we discard the string underneath the error and replace it with the integer 4.

Sometimes you need to raise your own errors, you can do that with the word fail.

2
[odd?]
["ok"]
[[[type error] [asked odd?] [reason "expected odd number"]]
 association fail]
⚖️ ▶️
3 4 +
[[asked odd?]
 [handled 🔳]
 [reason "expected odd number"]
 [trace [fail ▶️]]
 [type error]
 [unwound [3 4 +]]]

Sometimes you want to handle some errors but not others. There's no error type matching like you'd find with java's catch. You have to recover, examine the error, and if it's one you don't want to handle, re-activate it with fail.

5.16. Your own words

You're not stuck with just the vocabulary in the starting environment. You can add your own vocabulary!

5.16.1. Concepts

One hard rule of kcats: "an environment cannot change its own dictionary".

What does this mean for being able to define new behavior? If we can't alter the current dictionary to add or modify words, what can we do? Answer: you can run your program inside another environment, with a different dictionary.

This has some important security benefits that most languages do not have.

  • There are no global redefinition attacks that communities like Javascript and Python have suffered through many times. (Where a compromised library replaces a common function with an attack payload, and the user's own code trips over it later when calling the common function)
  • It's possible to completely disable further modifications, for use cases where security takes precedence over expressivity. This is not possible in other dynamic languages.

Let's look at how kcats accomplishes this. The word dictionary retrieves the current dictionary and places it on the stack. Then we can treat it like any other data and alter it. Then we can create a new environment and evaluate that environment, and retrieve its stack.

Doing this with low level constructs looks like this:

;; Fetch the current dictionary
dictionary
;; Create a new set of words
[[square [👥 *]]] draft
;; apply the change to the dictionary
[words] 🔀 update
;; The program to run with the new dictionary
[9 square]
;; Create an environment and evaluate it
[program dictionary] label
environment evaluate
;; make the inner stack the new stack
[stack] 🔍 restore
;; note the current dictionary is not altered
dictionary [words square] 🔍
🔳 81

Obviously this is quite cumbersome, so there are higher level words to do all of this for you, like let.

5.16.2. New Words

The most straightforward and common change you can make to the dictionary, is to add a word that wasn't in there before, and use it in some limited scope after which it is no longer accessible.

The word let is handy for small bits of code where you don't want to repeat yourself:

[[square [👥 *]]]
[9 square 8 square +]
let ▶️
145

Let's break this down. The word let takes two arguments, a list of new words paired with their definitions, and a program to run that uses those words. We define a new word square to mean [👥 *], and then we create a program that runs [9 square 8 square +] inside an environment with the new word square defined. Then finally execute that program to get 145.

The list of new words can even refer to another word from the same list:

[[square [👥 *]]
 [fourth [square square]]]
[3 fourth]
let ▶️
81

Words you're defining can refer to themselves - recursive functions are great!

[[factorial [[🔀 positive?] 🛡️
             [🗑️ [*] •🛡️ ▶️ [dec] 🪄 factorial]
             when ▶️]]]
[9 1 factorial •🗑️]
let ▶️
362880

5.16.3. Altering meanings of existing words

Creating new words is relatively safe - presumably no one is using those words, so giving them meaning doesn't cause any confusion. It's when you start changing an existing meaning that things get a bit complicated.

As we have seen, words perform actions, and those actions are specified by other words. So a word foo can use the word bar as part of its execution. So let's say I change the meaning of the word bar. Does that mean I changed the meaning of foo as well (because bar is part of the meaning of foo)?

The answer is it depends.

In kcats, by default, the answer is no. When you are changing the meaning, it's for you own immediate use of the word. Let's go over some examples.

Let's say we want to alter the meaning of a rather important word that's used all over the place in the standard library: 🗑️. "When I say 🗑️ I want to just insert the number 5."

[[🗑️ [5]]]
["a" "b" "c" 🗑️]
let ▶️ 
5 "c" "b" "a"

Ok, straightforward enough, right? But what happens if we call another word that uses 🗑️, like times? First let's look at the definition of times, to see that it really does call 🗑️:

[times] definition
[[1️⃣ [positive?] 🛡️ [🗑️ dec [2️⃣ 👥 🪄] 🪄 times ▶️] [🗑️] ⚖️ ▶️]
 🎒]

It calls swap quite a bit! Now let's alter the meaning of swap and call times.

[[🗑️ [5]]]
[["hi"] 3 times ▶️ 🗑️]
let ▶️
[[asked [execute]]
 [handled 🔳]
 [reason "word is not defined"]
 [type error]
 [unwound [execute]]]
[[[dictionary [[modules [#b64 "DRRitpxIz3S3HZ-keSV_EEMSscTJkFRzVwGI-4TTu4s"]]
               [words 261_entries]]]
  [program [["hi"] 3 times 🗑️]]
  [stack 🔳]]
 capture evaluate [stack] 🔍 restore]

Notice that times still works as expected! Even though it calls swap internally, it didn't insert any 5's. Only our own swap did that.

You can even refer to the old behavior of a word when defining new behavior:

[[swap [5 swap]]]
[["hi"] 3 times swap]
let 
"hi" 5 "hi" "hi"

Here we redefine swap to mean "insert 5, and then do whatever swap did before". This works even though swap is an axiom word:

[swap] definition
builtin

There are two other words related to let that are more flexible for times when you need to do a bit more complex alterations of existing meaning. One of the primary use cases is adding a method. You've got a word that behaves differently depending on its argument and you want to add a new behavior.

[[hash [[type [foo] 🍫 =]
        [drop "foo" hash]
        addmethod]]]
[[[foo myfoo]] association hash] revise
"foo" hash =
yes

What have we done here? We're taking the word hash, which is just a decide:

[first] definition
[📤 •🗑️]

See how it behaves differently for a byte array by calling hashbytes and by default it calls encode and then tries hash again? So now we've added a new logic branch there:

[[hash [[type [foo] 🍫 =]
        [drop "foo" hash]
        addmethod]]]
[[hash] definition] revise
[[[[type [foo] 🍫 =] [drop "foo" hash]]
  [[bytes?] [hashbytes]]
  [[yes] [encode hash]]]
 decide]

When the type is 'foo', we use the hash of the string "foo".

5.16.4. Sharing code

5.16.4.1. Overview

Sometimes you want to add some vocabulary to your environment that someone else wrote. We call those modules. A module makes some changes to the standard dictionary, usually by adding new words (as you saw already with let). The difference with a module is that a set of changes are bound up together and given an alias. You can load a module and then use it by using some words in it, we'll see how in a bit.

5.16.4.2. Contents

The contents of a module is a program whose input will be the current environment's dictionary, and it should output a new, modified dictionary. Typically, the module will add some new words to the dictionary, but it can also alter or delete them. Here's some example modules:

This one adds two words, square and cube - it gives two definitions in a list and join s them with the existing dictionary, merging them together.

[[square [[definition [👥 *]]]]
 [cube [[definition [👥 square *]]]]]
join

This one changes the definition of an existing word: it adds the action •🗑️ to the end of the definition of the word foo.

[foo definition] [[•🗑️] join] update
5.16.4.3. Inscribing a module vs using a module

When you inscribe a module, you make it available to be used, but the module's changes remain invisible until called upon with using.

;; inscribe a new module that we explicitly pass in
[mymodule] 🍫
[[[plus2 [[definition [2 +]]]]] join] inscribe
;; leaves the hash on the stack but let's drop it and pretend we want
;; to use the module later
drop
;; If we try to use plus2 it won't work yet:
;; 
;; 3 plus2
;; 
;; we have to declare we're using mymodule:
[mymodule] [3 plus2] using
5

using takes a list of modules so you can use more than one at a time. It only applies to the given program.

5.16.4.4. Quick and dirty words with let

Sometimes you want to define some words and use them right away and not care about making a module for use elsewhere. There's a word that combines inscribe and using for you, it's called let:

[[plus2 [2 +]]]
[3 plus2] let
5

However there are some downsides to using let. You should not call let in a tight loop, because each time it's called it's going to re-define the module, which is a relatively expensive operation. Instead you should call inscribe outside the loop, and use resolve. The difference between using and resolve becomes clear when we look at how using is defined:

dictionary [using definition] 🔍
[resolve execute]

using modifies a program by resolving all words to their modules, and then executes the program. If you intend to use a module's words in a tight loop, you can resolve the program and then later pass that program to the looping construct:

[mymodule] 🍫 [[plus2 [2 +]]] draft inscribe drop
1 10 1 range 
[[actual [[definition [2 +]]]]
 [asked [program]]
 [handled yes]
 [reason "type mismatch"]
 [type error]
 [unwound [update [[dictionary_redacted #b64 "M4G/6zwsAHWojAeJtU/Zrg/qYfE2QrnF3l5Bh9pqja4="]]
           🍫 evert first •🗑️ dictmerge drop 1 10 1 range]]]
[[definition [2 +]]] [plus2] dictionary_redacted #b64 "M4G/6zwsAHWojAeJtU/Zrg/qYfE2QrnF3l5Bh9pqja4="
5.16.4.5. Aliases

Modules are identified by their cryptographic hash 4. That lets you be sure that the code you meant to run, is actually what you are running. However since hashes look like this /1vRbfFezlcTCUfQCjC1FKukWLoOAeBuvxNXUDbFKSk=, kcats can keep track of human-readable module nicknames for you, and they're called aliases. For example you might call /1vRbfFezlcTCUfQCjC1FKukWLoOAeBuvxNXUDbFKSk= by the name customers.

When you refer to a module, you can refer to it either by its hash or its alias. Aliases are not universal names, they're local to your program. So you can use whatever aliases you want, without worrying about them colliding with other people's aliases.

5.16.4.6. Security rules

There are some safeguards in place to ensure that words mean what you expect them to mean, and that modules you load can't change the meaning of words outside of where you intended.

  • Aliases are immutable for the lifetime of your program - once you alias a name to hash, you cannot change it. You can only create new aliases. Trying to modify an alias to a new hash, is an error.
  • You cannot alter words in the core dictionary (the words of the default environment before any modules are loaded).
  • You cannot truly delete words either, you can only shadow them. If you delete a word from the dictionary in a module definition, what happens is the runtime makes a copy of that word, whose definition is to return the same error as if the word isn't defined. This allows you to block a program from accessing some words, but it is not a global or permanent change.

5.17. Generators

5.17.1. Overview

Sometimes in programming, having the concept of an indefinite sequence is handy. You have part of your program producing data, and another consuming it, but the producer doesn't know how much the consumer will actually need. A producer might calculate a huge number of items at great expense, only for the consumer to only need a tiny fraction of them. Generators allow the consumer to tell the producer when to produce, but the producer still retains all the logic of how that's done.

In kcats there's no special sauce for generators, we can implement them as a pattern with just the standard words we've already seen.

5.17.2. Example

Let's say you want to create the fibonacci sequence. Let's see how we can code that without worrying about how many items in the sequence we'll eventually need.

A generator consists of two things: state, and a program. Each time we want to generate an item, we run the program. The program should produce a new item and update the state. We just put however many state items we need on the stack, and then a program that can work with those items.

1 0 [[+] •🛡️ ▶️ 🔀 👥]

So here we start with 1 0. That's the starting state. Normally we'd start fibonacci with 1 1 but this isn't the actual first two numbers in the sequence, it's starting values we use to calculate them. Then we have a program that takes two numbers as input and leaves one new number. Let's just ▶️ that program and see the result:

1 0 [[+] •🛡️ ▶️ 🔀 👥] ▶️
1 1 1

We can see the 0 is now 1 and there's an extra 1 on the stack. Remember the generator must do two things, produce a new item and update the state. It updated the state from 0 1 to 1 1, and produced the first item, 1.

This gets us one number, but not the whole fibonacci sequence. Let's look at the word generator. It creates a recurrence - a program that when you run it, may leave another copy of itself on the stack ready to be run again.

You can ▶️ this recurrence and get the next item in the fibonacci sequence, and beneath that you get another copy of it ready to be run again, when you've done what you need to with the first item it gave you.

[#b64 "zubPuf7fwUx1W6i8RJqAE8DR43dHFfnjx1xulAZ0D_U"]
[fibonacci generator ▶️]
;use
stage 🔀 ;; lm env
[[stack] [📸] •🐋 assign] 🪄 ;; capture the stack at runtime
using ;; set up the resolver 
evaluate ;; execute the program in the inner environment
[stack] 🔍 restore
[[asked [fibonacci]]
 [handled 🔳]
 [reason "word is not defined"]
 [trace 🔳]
 [type error]
 [unwound [fibonacci generator ▶️]]]

Notice here that the only difference from before is that the program is sandwiched between the fibonacci number we produced, and the state.

Let's keep going and execute again! But wait, before we do that we need to do something with item we just produced, to get it out of the way. For now we'll just 🗑️ it. We've seen it and we want to see what's next.

1 0 [[+] •🛡️ ▶️ 🔀 👥] generator ▶️
🗑️ ▶️ 
1 [[[+] •🛡️ ▶️ 🔀 👥] 🔳 🔳 [🔀] 🪆 ▶️] 1 2

Ok, so the 2nd item is 1 and we can see the state is updated - instead of 1 1 we have 1 2.

One more time:

1 0 [[+] •🛡️ ▶️ 🔀 👥] generator ▶️
🗑️ ▶️ 
🗑️ ▶️ 
2 [[[+] •🛡️ ▶️ 🔀 👥] 🔳 🔳 [🔀] 🪆 ▶️] 2 3

Ok we can see that we can get items one at a time by calling generate, but this is not very useful. What we really want is to get the first 20 numbers in the fibonacci sequence, and collect them into a list. We can do exactly that:

1 0 [[+] •🛡️ ▶️ 🔀 👥] generator
20 taker
collect
[1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765]
6765 10946

There's the fibonacci sequence! And the state is still there beneath in case we want to use it again.

So what is happening here? We're building up generators by wrapping one in another. Starting with the last, we have collect which will repeatedly call ▶️ generator inside it. It keeps going and collecting the generated items in a list, until the inner generator returns 🔳. Then it stops and returns what it collected.

Then inside collect we have a generator 20 taker - what that does is keeps its own state of how many items we want it to take. It counts down as it generates items inside it, passing them up to collect and when it hits zero, it returns 🔳 (even if the generator below it would have produced something, taker won't even ask). That will signal collect to stop.

We have other handy generators we can stack up. Let's say for whatever reason we want to know what are the first 20 odd fibonacci numbers? Well, we have keep:

1 0 [[+] •🛡️ ▶️ 🔀 👥] generator ;; our original generator
[odd?] keep ;; a generator that keeps calling the one
            ;; below it until it gets something that
            ;; passes the predicate we specified
20 taker ;; another generator that calls generate 20 times
collect  ;; collects all the generated items into a list
[1 1 3 5 13 21 55 89 233 377 987 1597 4181 6765 17711 28657 75025 121393 317811 514229]
514229 832040

There it is, the first 20 odd fibonacci numbers!

Let's say instead we wanted to know the prime factors that make up each of the first 20 fibonacci numbers. We can do that with each:

1 0 [[+] •🛡️ ▶️ 🔀 👥] generator ;; our original generator

[🔳 🔀 2
 [[sqrt] 🪄 >=] 🛡️ 
 [🗑️ mod zero?] 🛡️ 
 [🗑️ 🗑️ under ;; c-d i c-d r
  [📮] •🪄 ;; c-d i new-r
  / 2] ;; dividend new-r
 [🗑️ inc] ;; c-d++ i r
 ⚖️ ⏳ ▶️
 🗑️ 📮] each

20 taker ;; another generator that calls generate 20 times
collect  ;; collects all the generated items into a list
[[1] [1] [2] [3] [5] [2 2 2] [13] [3 7] [2 17] [5 11]
 [89] [2 2 2 2 3 3] [233] [13 29]
 [2 5 61] [3 7 47] [1597] [2 2 2 17 19] [37 113] [3 5 11 41]]
6765 10946

There we have it. We can see that [2 2 2] is what makes up 8, etc.

Other included generators are:

dropper
Inverse of taker - drops the first n items of the sequence and returns the rest.
joiner
Joins items together
integers
all the numbers starting with 0

reduce will consume what a generator produces. You provide a program that takes 2 arguments, and reduce will generate all the items, and pass to your program: the result so far and the next item generated, and repeat that until there are no items left:

[integers
 1 dropper ;; drop 0 so we start with 1
 10 taker
 [3 *] each
 [+] reduce]
shield
135

5.17.3. Compatibility

Let's say you go to the trouble of making a beautiful stack of transformations and you want to re-use it, but you don't have a generator, you have a list! Our transformation stack needs a generator! How are we supposed to use it? Never fear, there is a simple way to adapt transformations to work on anything that works with the word take. You can use the word liberator to adapt a list to a generator.

5.17.4. Capturing items from the stack

One pitfall with generators is that sometimes you want to transform generated items and use some item from the stack to help do it. The problem with this is that generators can be arbitrarily deep and you won't know exactly how deep that item is.

The solution is to capture the items you want.

Let's look at a simple example. Let's say we want to generate every multiple of n (where n is some number on the stack). We can already generate every integer, we just need to multiply each one by n. A naive solution would be to just use each, but it doesn't work:

3 integers generator
10 taker
[*] each
;1 dropper ;; drop 0
;10 taker
collect
[0 8 14 18 20 20 18 14 8 0]
9 3

This doesn't work because n and the last integer we generated aren't next to each other on the stack, there's a bunch of generator machinery in between. We could try to guess exactly how deep the machinery is, but then our generators aren't composable anymore - we couldn't move that call to each somewhere else in the generator stack, without having to change the program. What we really should do is create our program for each first, before we start stacking up generators, and bind n:

3 
[*] bind
[integers] dip ;; insert the integers generator below the each program
each
1 dropper
10 taker collect
[3 6 9 12 15 18 21 24 27 30]
[[positive?] [dec [generate] dive] [[]] if] 0 [[[positive?] [[generate drop]
                                                             dip dec]
                                                while [generate swap]
                                                dip float]
                                               bail]
0 [generate [[3 *]
             bail]
   shielddown]
[inc 👥]
10

What exactly is this doing? We're taking values from the current stack, and prepending them to a program, so that later when the program executes, it'll find that value on the top of the stack. Put another way, we're binding the value of the first argument to the program now, rather than letting it take a value from the top of the stack later.

5.18. Coordination and Input/Output

5.18.1. Basics

In kcats, both coordination and input/output are done with pipes. See the definition for pipe.

Let's take a common example of coordination. Your program has to do several very long and intensive calculations but doesn't want to make the user wait to do other things. The way that's done in kcats is by creating multiple environments, and have them communicate with each other using pipes. You can send any item through a pipe that you could put onto the stack, including other pipes. You can clone a pipe to give access to it to more than one environment.

There are two main operations a pipe supports: put and take. You either put an item in, or take an item out. Either one of those operations may block, if the pipe is either full (when putting) or empty (when taking). Your environment would have to wait for some other environment to take something out so there's space to put, or put something in so that there's something to take out.

All pipes share the put and take operations but they can differ in other ways.

Note that put and take can also be used on plain lists. put adds to the end, and take removes the first item. Neither will ever block when used on a list. Another slight difference is what happens when you've reached the end of the content (either the list is empty or the pipe has, for example, hit the end of file condition): a take from an empty list will just return nothing, but a take from a pipe that is at EOF will result in an error.

5.18.2. Input/output

Let's look at how we do I/O using files as an example - let's say we want to write the word foo to a file called bar:

[[file "bar"]] pipe-in ;; create the pipe to the given file "foo"
"foo" encode ;; we have to convert string to bytes first, using the word
            ;; =encode=.
put ;; finally, put the bytes into the pipe, and they are written to
    ;; the file
[[to [[file "bar"]]]
 [type tunnel]
 [values [[type bytes]]]]

Note the representation of the pipe shows where it leads (the to field), and what types of items it can carry (the values field).

Neither put nor take consume the pipe from the stack, for convenience, as most of the time you'll want to use it again.

Let's look at reading from a file:

[[file "bar"]] pipe-out
take string
"Hello World!" [[type tunnel]
                [values [[type bytes]]]
                [to [[file "bar"]]]]

Note that the amount of bytes you'll get from a file on each take, is limited. You will only get the entire contents if the file is small. We'll want to repeatedly take until there's nothing left, and put all the taken parts together.

Here's how we do it:

  • turn the pipe that provides chunks of a file into a generator, with [take].
  • Assemble the chunks with reduce. It requires a program to say how to combine the chunks. We want to join them, so the program is [join].

We can also use the word file-out as a shortcut to get a pipe given a file's name.

"bar" file-out [take] join reduce string
"Hello World!" [take] [[type tunnel]
                       [values [[type bytes]]]
                       [to [[file "bar"]]]]

Finally there's a convenient alias for [take] [join] reduce string, it's called slurp:

dictionary [slurp] 🔍
[[definition [[take] [join] reduce string [drop drop]
              dip]]
 [spec [[pipe] [item]]]]

It actually drops the generator for you as well, since we know it's already been fully read from. So you can do this:

"bar" file-out slurp
"Hello World!"

5.18.3. Coordination of simultaneous programs

5.18.3.1. Basics

The way kcats handles parallel processing (aka multithreading) is by allowing you to create multiple environments, each with their own programs, that run simultaneously.

Often you need the environments to communicate with each other, and not just with the outside world. That's done with channels. Channels let you send items from one environment to another.

5.18.3.2. Channels

Channels are a type of pipe. They are different from other pipes, like files or network sockets, in that while files and sockets only deal with bytes, channels can pass any kind of item (numbers, strings, words, lists, and even other pipes). If you can put an item on the stack, you can also pass it through a channel.

  1. Handoff

    This is the most common type of channel. Think of it as a pipe with no length or capacity. It's more like a hole in a wall, than an actual pipe. You can pass items through the hole, but only if someone is already on the other side waiting to take it. If your program tries to pass an item through the handoff, but no other program running simultaneously is already waiting to take it, your program will block. That means it stops and waits. The same goes for receiving items - if your program tries to receive but no one is sending yet, your program will wait until someone sends. Handoffs support multiple senders and receivers through the same "hole" - imagine multiple people standing on either side of the wall, holding items they need to pass through, and others empty handed waiting to receive. Only one person can receive per send - it's a direct handoff and not a broadcast.

    You pass items from one environment to another by giving both environments a copy of the handoff pipe. One environment calls put (with an item) and the other calls take. If they're doing that at the same time, the item moves from one environment to the other and both the put and take complete at the same time.

    You can pass as many items through the handoff as you want.

  2. Buffered

    This channel works similarly to a Handoff, but instead of having no capacity, it has a fixed capacity that you specify when you create it. Let's say you give it a capacity of 10. That means the sender can put 10 items in, even when no one is receiving yet. On the 11th item, sending will block just like a handoff would.

    If a receiver takes an item out, that frees up space for one more item that the sender can put in without blocking.

  3. Signaling when you're done

    It's common for a program to keep taking from a pipe until there's nothing left to take, and then stop. How does it know there's nothing left (in other words, nothing more will ever arrive, no matter how long it waits)? With channels there's 2 ways to know:

    • One of the senders sends the item nothing. All the standard library words that pull from a channel repeatedly, will stop when they receive nothing.
    • All the senders copies of the channel are dropped.

    How is a sender dropped? That can happen several ways:

    • The channel item is dropped from the stack with the word drop or its variants, or otherwise consumed.
    • The entire environment is dropped (as often happens when its program is finished)
    • A program receives on this copy of the channel. By default, channels are one direction only. As soon as you send, the receiver is dropped, and vice versa. Bidirectional channels (where the same program can send and receive on the same channel) are possible but currently not supported because they take a bit more care to use properly.

    When writing a program that needs to know about the end of a stream, you must take care to not leave extra copies of channels lying around. Any extra channel copy that you don't drop when finished, will keep the stream open forever and will result in a deadlock. A deadlock is when some program is waiting for a condition that can never occur. In this case, a receiver is waiting for the end of the stream, and it will never get it. The primary symptom of a deadlock is a program that appears to be stuck or doing nothing, doesn't produce the intended result, doesn't produce an error, and doesn't exit.

    Note that it's also possible for receivers to signal they have everything they want, by dropping their copy of the channel. After all receivers are dropped, all senders will get errors when they try to send.

5.18.3.3. Multiple environments

Parallel processing is done with multiple environments running simultaneously, but how exactly do we set that up? And how do we enable them to communicate with each other? How do we create two environments with a copy of the same channel, so one can send and the other receive?

One handy way is to use the word spawn. It takes a program and creates a new environment, such that its program is the given program and the stack is a copy of the current environment's stack. That way, you end up with two environments, with different programs but the same stack.

Once the new environment is created, it doesn't actually do anything, it's just another item inside the current environment. To make it actually run the program, we need to animate it. At that point it disappears from this environment and goes off on its own.

handoff [5 put] spawn animate take
5 [[type out]
   [handoff 0]]

Here we create a handoff, and then create a new environment that inherits that handoff in its stack. That new environment will put the number 5 into the channel. Then we call take on the other copy of the channel to receive the number 5.

Note that when you animate a new environment, you can't debug it using the same debugging tools we saw earlier. You actually lose all contact with it, except for whatever pipe communication you program it to do. Debugging simultaneous programs is a bit more involved and this will be covered later.

5.18.3.4. Example

We'll show how parallel programming (multithreading) works in kcats, with an example. Let's say we have a database of books, and we have 3 librarians checking in returned books. Let's say for simplicity that all the books in the database were checked out and have now been returned and are ready for checkin. We want to update the database so that each book has a last-checkin property with the timestamp the librarian processed it, and checked-in-by with the librarian's name. We want all the books to be processed exactly once, with 3 librarians working in parallel.

We'll create chunks of our program and put it all together later. Here's one chunk. Let's read in our database and parse it into a list:

"examples/books.kcats" file-out slurp read

So that's our database of unprocessed books. Now let's feed our database into a pipe, in a separate environment. We could also have used a buffered channel here.

[[put] step] spawn animate
drop ;; this copy of the library as the task to feed it is already running

Now we can create two channels - one to carry unstamped books (the input for the librarian), and one to carry stamped books (the librarian's output). Then we create another pipe that gives the current time.

handoff ;; a pipe for unprocessed books
handoff ;; a pipe to collect completed work
timestamps ;; pipe for the librarians to stamp their books with

Now we specify what each librarian should do. This is the meat of the program. Note the use of the word siphon here. It takes a generator and a pipe, and keeps generating items and putting them into the pipe, until the end of the stream is reached (generator produces nothing).

;; the spawn prog should see: n ts po pi (librarian-name, timestamps, output-pipe, input-pipe)
;; the book each prog should see: book n ts po pi
[[[[take •🗑️ ;; get a timestamp and then drop the ts pipe
    pair [checked-in-by last-checkin] swap zip] dip join]
  binddown binddown  ;; bind n, ts.
  [[take]] dip ;; make a generator out of pi
  each ;; book, do the stamping
  float siphon] ;; from the each generator to the output pipe 
 spawn animate] ;; launch a separate environment
[binddown binddown binddown] shielddown ;; bind the 2-4th args
;; now that we've already captured the pipes to the workers we
;; can drop pi and ts
•🗑️ dropdeep

Now that we've specified what each librarian should do, we specify the librarians. In this case, all we need is their name.

["Darlene" "Thomas" "Ethel"] ;; the names of the librarians
swap map ;; for each librarian do the above program

drop ;; the resulting list, we only care about launching envs

Finally we can just collect all the results from the output channel, and drop the other items we don't need anymore, leaving just the results.

[take] collect ;; the results
;; drop generator, leaving only results
sink drop drop

Here's the whole program:

handoff ;; a pipe for unprocessed books
"examples/books.kcats" file-out slurp read ;; read the library

;; feed all the unprocessed books into a pipe, in a separate task
[[put] step] spawn animate
drop ;; this copy of the library as the task to feed it is already running
handoff ;; a pipe to collect completed work
timestamps ;; pipe for the librarians to stamp their books with

;; the spawn prog should see: n ts po pi (librarian-name, timestamps, output-pipe, input-pipe)
;; the book each prog should see: book n ts po pi
[[[[swap take •🗑️ ;; get a timestamp and then drop the ts pipe
    pair [checked-in-by last-checkin] swap zip] dip join]
  binddown binddown  ;; bind n, ts.
  [[take]] dip ;; make a generator out of pi
  each ;; book, do the stamping
  float siphon] ;; from the each generator to the output pipe 
 spawn animate] ;; launch a separate environment
[binddown binddown binddown] shielddown ;; bind the 2-4th args
;; now that we've already captured the pipes to the workers we
;; can drop pi and ts
•🗑️ dropdeep

["Darlene" "Thomas" "Ethel"] ;; the names of the librarians
swap map ;; for each librarian do the above program

drop ;; the resulting list, we only care about launching envs
[take] collect ;; the results
;; drop generator, leaving only results
sink drop drop
[[[checked-in-by "Darlene"]
  [last-checkin 1698764958940]
  [author-first "George"]
  [author-last "Orwell"]
  [title "1984"]
  [year 1949]
  [subjects [government dystopia surveillance totalitarianism freedom]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958945]
  [author-first "Aldous"]
  [author-last "Huxley"]
  [title "Brave New World"]
  [year 1932]
  [subjects [society technology dystopia happiness drugs]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958950]
  [author-first "F. Scott"]
  [author-last "Fitzgerald"]
  [title "The Great Gatsby"]
  [year 1925]
  [subjects [wealth love obsession american-dream tragedy]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958950]
  [author-first "J.D."]
  [author-last "Salinger"]
  [title "The Catcher in the Rye"]
  [year 1951]
  [subjects [adolescence alienation innocence society adulthood]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958950]
  [author-first "Jane"]
  [author-last "Austen"]
  [title "Pride and Prejudice"]
  [year 1813]
  [subjects [love marriage society class reputation]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958950]
  [author-first "Mary"]
  [author-last "Shelley"]
  [title "Frankenstein"]
  [year 1818]
  [subjects [creation science responsibility monster humanity]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958950]
  [author-first "John"]
  [author-last "Steinbeck"]
  [title "Of Mice and Men"]
  [year 1937]
  [subjects [friendship dream loneliness society tragedy]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958950]
  [author-first "Ernest"]
  [author-last "Hemingway"]
  [title "The Old Man and the Sea"]
  [year 1952]
  [subjects [endurance nature old-age fisherman sea]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958950]
  [author-first "Harper"]
  [author-last "Lee"]
  [title "To Kill a Mockingbird"]
  [year 1960]
  [subjects [racism innocence morality law childhood]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958950]
  [author-first "J.R.R."]
  [author-last "Tolkien"]
  [title "The Lord of the Rings"]
  [year 1954]
  [subjects [adventure elf dwarf hobbit ring journey magic evil]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958951]
  [author-first "Joseph"]
  [author-last "Conrad"]
  [title "Heart of Darkness"]
  [year 1899]
  [subjects [colonization africa journey morality darkness europeans]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958951]
  [author-first "Leo"]
  [author-last "Tolstoy"]
  [title "War and Peace"]
  [year 1869]
  [subjects [war peace society history love aristocracy]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958951]
  [author-first "Homer"]
  [title "The Odyssey"]
  [year -800]
  [subjects [journey odyssey homecoming gods heroism adventure]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958951]
  [author-first "Charlotte"]
  [author-last "Bronte"]
  [title "Jane Eyre"]
  [year 1847]
  [subjects [love morality society class womanhood independence]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958951]
  [author-first "Mark"]
  [author-last "Twain"]
  [title "Adventures of Huckleberry Finn"]
  [year 1884]
  [subjects [adventure racism slavery morality friendship river]]]
 [[checked-in-by "Darlene"]
  [last-checkin 1698764958951]
  [author-first "Ray"]
  [author-last "Bradbury"]
  [title "Fahrenheit 451"]
  [year 1953]
  [subjects [censorship knowledge books society dystopia future]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958951]
  [author-first "Charles"]
  [author-last "Dickens"]
  [title "A Tale of Two Cities"]
  [year 1859]
  [subjects [revolution love sacrifice resurrection society history]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958951]
  [author-first "William"]
  [author-last "Golding"]
  [title "Lord of the Flies"]
  [year 1954]
  [subjects [society civilization savagery childhood morality island]]]
 [[checked-in-by "Thomas"]
  [last-checkin 1698764958951]
  [author-first "Miguel de"]
  [author-last "Cervantes"]
  [title "Don Quixote"]
  [year 1605]
  [subjects [adventure idealism reality knight insanity literature]]]
 [[checked-in-by "Ethel"]
  [last-checkin 1698764958951]
  [author-first "H.G."]
  [author-last "Wells"]
  [title "The War of the Worlds"]
  [year 1898]
  [subjects [invasion aliens society technology war humanity]]]]

6. More Examples

6.1. Query a flat file database

["examples/books.kcats" file-out slurp read
 [[subjects] 🔍
  set [dystopia] 🍫 contains?]
 filter] 
shield
[[[author-first "George"]
  [author-last "Orwell"]
  [title "1984"]
  [year 1949]
  [subjects [government dystopia surveillance totalitarianism freedom]]]
 [[author-first "Aldous"]
  [author-last "Huxley"]
  [title "Brave New World"]
  [year 1932]
  [subjects [society technology dystopia happiness drugs]]]
 [[author-first "Ray"]
  [author-last "Bradbury"]
  [title "Fahrenheit 451"]
  [year 1953]
  [subjects [censorship knowledge books society dystopia future]]]]

6.2. Factorial

6.2.1. Recursive with recur

10
[1 <=]
[]
[👥 dec]
[execute *]
recur
3628800

6.2.2. Using range

10
inc [1 1] dip 1 range 
[*] step
3628800

6.2.3. Plain loop

10 👥 
yes [dec 👥 [*] dip 👥 1 >] loop
drop
3628800

6.3. Jensen's Device

https://rosettacode.org/wiki/Jensen%27s_Device

100 [0] [[1.0 swap /] dip +] primrec
5.187377517639621

6.4. Fibonacci

[1 0 [[+] shielddown swap 👥] ;; fibonacci generator
 20 taker ;; another generator that calls generate 20 times
 collect] ;; collects all the generated items into a list
 shield
[1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765]

6.5. Prime factors

36023425442111112
;12
[] 🔀 2 ;;  current-divisor input result

[🔀 sqrt <=] 🛡️ 
[🗑️ mod zero?] 🛡️  
[🗑️ 🗑️ dump 👥 ;; c-d c-d i r
;; c-d i c-d r
 [📮] •🪄  ;; c-d i new-r
 / 2] ;; dividend new-r
[🗑️ inc] ;; c-d++ i r
⚖️
⏳ ▶️  

🗑️ 📮

Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
Size of Item: 72 bytes
Size of Sized: 64 bytes
Size of Associative: 40 bytes
Size of Environment: 128 bytes
Size of List: 64 bytes
[2 36023425442111112 🔳]
[2 18011712721055556 [2]]
[2 9005856360527778 [2 2]]
[3 4502928180263889 [2 2 2]]
[17 1500976060087963 [2 2 2 3]]
[29 88292709416939 [2 2 2 3 17]]
1400012

[] 🔀 2 ;;  current-divisor input result

[[sqrt] 🪄 >=] 🛡️ 
[🗑️ mod zero?] 🛡️  
[🗑️ 🗑️ dump 👥 ;; c-d c-d i r
;; c-d i c-d r
 [📮] •🪄  ;; c-d i new-r
 / 2] ;; dividend new-r
[🗑️ inc] ;; c-d++ i r
⚖️
⏳ ▶️  

🗑️ 📮

Loading module (should be empty?): Program([])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
Loading module (should be empty?): Program([Frame { snippet: [🔀, 🗑️], next_idx: 2 }])
[2 1400012 🔳]
[2 700006 [2]]
[2 2 350003]

Prime factor generator

1231231231231231
2
[[[[[mod zero? not]
    [[sqrt] dip >=]] [execute] every?]  
  [inc] 
  while

  [mod zero?]
  [[/] shield dropdeep swap 👥]
  [drop [] swap]
  if]
 bail]
collect •🗑️
[89 3271 5683 744203]

6.6. bidirectional comms from a socket

["" [string join] ;; each group of bytes that come out of the tunnel,
                  ;; convert to string and join to whatever we already
                  ;; collectd
 [[type ip-host]
  [address "localhost"]
  [port 9988]] association ;; description of where to connect to (an ip port)
 tunnel ;; make a bidirectional tunnel
 "foo! bar!" put ;; send this string
 collect] ;; receive

6.7. Write string to a file

[[[file "/tmp/foo"]] pipe-in
 "blah" put
 close]

6.8. Search the dictionary

dictionary lingo ;; put the dictionary of all words onto the stack as key value pairs
[second ;; the value of one of those pairs
 [spec] 🔍 ;; look up the spec field
 first ;; the input part of the spec
 [number number] =] ;; is it taking two number inputs?
🧲 ▶️  ;; filter the dictionary using the above criteria
[first] 🚜 ▶️ ;; of what remains, just keep the key (which is the word itself)
[log - min / + mod * quot exp within? max rem]

6.9. Copy data from one file to another

  [[file "/tmp/myout"]] pipe-out 
  [[string] each [🔗] fold] assemble
;  [[file "/tmp/myin4"]] pipe-in 🔀 
;  [📮] 🪜 ▶️
"foo bar!\n"
"/tmp/foo" "/tmp/bar"
pair [[] [file] float assign] map ;; make file descriptors for both
take pipe-out
swap 🍫 pipe-in
[[unwound [[type] 🍫 = [[[[file "/tmp/foo"]] [[[file "/tmp/bar"]]]]] 🍫 evert first [[first second] [first first]] 🍫 branch [[[[count 1 =] [[first [type] 🍫 =] [first second] [first first] if] [[]] if] [[file "/tmp/foo"]] [[[file "/tmp/bar"]]]]] 🍫 evert first swap drop [[[[association] 🍫]]] 🍫 swap [[]] 🍫 or [[[[nothing?] shield] dip swap [or] shielddown] [] [[take swap [[execute] shielddown] dip swap] dip or] [execute] recur] execute swap drop swap drop [file] 🍫 = [[[[file "/tmp/foo"]] [[[file "/tmp/bar"]]]]] 🍫 evert first [[[[file "/tmp/foo"]] [[[file "/tmp/bar"]]]]] 🍫 evert first [[value file-out] [[[[type [ip-host] 🍫 =] [👥 [port] 🔍 [[address] 🔍] dip serversocket]] [[list?] [+kcats.pipe/->filled]]] decide]] 🍫 branch swap 🍫 pipe-in]] [type error] [reason "type mismatch"] [actual [[file "/tmp/foo"]]] [asked [list]] [handled yes]] [[[file "/tmp/bar"]]]

6.10. List the steps of program execution

[0 [1 2 3] [+] step] ;; the program to trace

[program] swap put wrap environment ;; create a starting env

;; now create a generator of environment states for each step of execution
[[[program] 🔍] ;; if the program is not empty
 [eval-step 👥] ;; step 
 [[]] ;; otherwise emit nothing to stop the consumption
 if]

;; consume the generator
collect
[[[stack [0]] [program [[1 2 3] [+] step]]]
 [[stack [[1 2 3] 0]] [program [[+] step]]]
 [[stack [[+] [1 2 3] 0]] [program [step]]]
 [[stack [[+] 1 0]] [program [execute [2 3] [+] step]]]
 [[stack [1 0]] [program [+ [2 3] [+] step]]]
 [[stack [1]] [program [[2 3] [+] step]]]
 [[stack [[2 3] 1]] [program [[+] step]]]
 [[stack [[+] [2 3] 1]] [program [step]]]
 [[stack [[+] 2 1]] [program [execute [3] [+] step]]]
 [[stack [2 1]] [program [+ [3] [+] step]]]
 [[stack [3]] [program [[3] [+] step]]]
 [[stack [[3] 3]] [program [[+] step]]]
 [[stack [[+] [3] 3]] [program [step]]]
 [[stack [[+] 3 3]] [program [execute [] [+] step]]]
 [[stack [3 3]] [program [+ [] [+] step]]]
 [[stack [6]] [program [[] [+] step]]]
 [[stack [[] 6]] [program [[+] step]]]
 [[stack [[+] [] 6]] [program [step]]]
 [[stack [6]] [program []]]]
[[[program] 🔍] [eval-step 👥] [[]] if] [[program []] [stack [6]]]

We could ensure the stack/program are printed in the same order each time

[0 [1 2 3] [+] step] ;; the program to debug

[program] swap put wrap environment ;; create a starting env

;; now create a generator of environment states for each step of execution
[[[program] 🔍] [eval-step 👥] [[]] if]

;; print with the fields sorted the same way for each step
[
 [[stack [[+] 3 3]] [program [execute [] [+] step]]]
 [[stack [3 3]] [program [+ [] [+] step]]]
 [[stack [6]] [program [[] [+] step]]]
 [[stack [[] 6]] [program [[+] step]]]
 [[stack [[+] [] 6]] [program [step]]]
 [[stack [6]] [program []]]]
[[[program] 🔍] [eval-step 👥] [[]] if] [[program []] [stack [6]]]

7. Contributing

7.1. Issue reporting

Instead of opening a github issue, add a TODO subheading to the Issues heading. Commit the change and submit it as a pull request. In the branch where that issue is being fixed, it will be changed to INPROGRESS. When the issue is fixed, the heading will be removed. (If you disagree that it's been fixed, submit a PR that reverts the commit to remove it).

You can edit this file right on github, in your own fork of the project, if you prefer.

Why do things this weird way? I don't want to rely on github, nice as it is.

Please do report design improvements you'd like to see - for example, inconsistencies in how words expect stack arguments to be, ways to make the standard library easier to work with, etc.

8. FAQ

8.1. How do I keep track of what should be on the stack, as I write programs?

There's tooling planned that will help show what should be on the stack at any point in a given program. However until that exists, you can use comments to annotate your program line by line, and show what is on the stack at each step.

See 6.5 example.

This is admittedly low tech, but it isn't as tedious as you might first expect. You only have to pay attention to the stack items actually touched by the code you're writing, which usually is rather small - if you need to annotate a line with more than 5 or 6 items you are probably doing something wrong. The solution could be to use an association or list to hold multiple properties of the same conceptual object, in one stack item.

8.2. Why is there no word 'no' in kcats, but there is 'yes'?

The logic rules are that empty containers are 'negative', and every other value is 'affirmative'. So the word no, by those rules would be affirmative, which would be very surprising! So the word no is not used in the language. To convey a logical negative, use empty list [].

8.3. I got an error, how do I diagnose the problem?

The first thing to do is check the contents of the error. The unwound field will show you the instruction that failed along with the remaining program.

If that doesn't tell you enough about what's wrong (and it often doesn't), there are several more tools at your disposal:

  • dump - this word prints the stack to stdout, if you add it to your program at strategic places, you can see if the stack looks the way you would expect at that point in the execution.
  • built-in debugger - this allows you to step through program execution.
  • tracing - see 6.10

8.4. kcats panics and exits, why?

This is due to a known unimplemented feature in the interpreter, or an unknown bug. Please see 9, and if you don't see it there, please add a new one and submit a pull request. Even for unimplemented features it's good to let us know you need that feature so we can prioritize it.

The goal is for kcats to never panic.

9. Issues

9.1. DONE Build without using emacs interactively

Users should not be required to know emacs to build the project, only have it installed. The build should be accessible from bash without having to use emacs interactively.

9.2. DONE Remove platform interop from lexicon

That was only there as a cheat when there was only the prototype implementation. The platforms are different and their function names don't belong in the lexicon.

I'm not even sure there should be platform interop at all - it doesn't appear to be possible in the rust impl anyway.

So far what I've done is have some lower level words actually in the dictionary but marked them like `++lookup`. I haven't decided what to do about this yet. Lower level words probably should just be first class citizens and I just need to think of better names. Right now the low level (single-depth) lookup is `++lookup` and the user-facing `lookup` does the arbitrary depth. In this case, the user-facing name probably needs to change to reflect what it does (something like `drill` or `extract`), and then the low level can just be `lookup`.

That means for all the i/o and crypto interactions, there needs to be low-level words. I'm not sure yet how to prevent namespace pollution, as one of the design choices is

9.3. DONE 'unassign' doesn't take a keylist, only a single key

Should change to match assign and lookup, accept a list instead of a single bare word.

9.4. DONE More support for nested/related envs

Debuggers, spawning, ingesting etc

9.5. TODO Graphical environment browser/editor

It would be nice to have a graphical display of all the environments in an application, and be able to

  • Drill into the environment and read the stack/program/dictionary
  • Pause/resume execution
  • Apply debugging (breakpoint, step etc)
  • View pipes and what/where they connect to (draw lines if they connect somewhere else in the app)
  • Manually put things into pipes or take them out
  • Create new envs
  • Persist changes
  • Revert changes

9.6. TODO Code distribution method

Let's say we write an app or library, how do we distribute it?

This ties in with durability - where do we store things in general, and not just libraries? kcats does support the filesystem but I would like that to be for compatibility only. The "native" kcats way of storing and retrieving things should be via hash keys. There may also be a fact database, probably with sparse tables (aka eavt format).

It brings up the question of what should "come with" the language. I am thinking maybe there's a "barebones" version of the language with no library management or anything. Then on top of that, build some durability and networking to distribute code and other data. Then the question is, what do we need to support in the base language? Seems like there needs to be database/network functionality there, but unused? Maybe make it a feature flag?

Let's explore the various options

9.6.1. Durability

It's tempting to want the flexibility of EAV (where there's basically just one big db table with 3 columns and every attribute is a row).

However this may be a little hasty. Perhaps what we're really after here is custom tables - the idea being that each user's db schema might be different depending on what data is important to them.

We've basically got a database schema consensus problem. Maybe Alice has a table CATS with columns SIZE COLOR AGE and Bob has a table CATS with columns HEIGHT COAT-COLOR AGE. How do they share data? The two tables are not really compatible without a specialized conversion tool and even then some data would be missing. So Alice and Bob ideally should agree on what a CATS schema is, otherwise they can't really share CAT facts. The advantage of EAV might be that even if they had different schemas they could stlil perhaps meaningfully talk about AGE and possibly even COLOR (with a bit of intervention, or even another fact that equates COLOR and COAT-COLOR in CATS).

The drawback of EAV is of course that it would perform rather terribly as the database grows. I can't say for sure how many facts could potentially be stored here, but here are some constraints:

  • Assume individual data only (no facebooks that store millions of people's data)
  • Assume popularity of the app (users may try to cram every fact they "know" into this db)
  • Assume there's some kind of garbage collection - Alice may collect weather observations or predictions constantly but doesn't need to keep old data. Maybe facts have a TTL? Not sure how that could be determined automatically.

    It's hard to estimate how large the db might get, but I suspect a lower bound of supporting 1M words is safe. As for upper bound, it's more difficult to say, but I would think the hardware limits of mobile devices would come into play. As of 2023 I think a db size on the order of 10gb would be approaching the device's capability limits, so maybe 100M words or so. I think it would be difficult to get an EAV database to perform well at that size, especially on mobile. Note datomic can handle that size so it's theoretically within reach.

    It may be possible to pick a standard db now (sqlite maybe) and not worry too much about performance. As long as the facts are portable to another db (which shouldn't be that hard), the issue can be revisited when it becomes an issue.

    Even using sqlite though, just building proper queries may be difficult. It may be possible to skirt that problem too and just do a minimal query to get a dataset that fits easily in memory and then post-process the rest. Let's say the query is "List all predictors (people who made predictions) and their accuracy", you could get all the unique predictor ids in a query, then one by one get all their predictions, then get all the relevant observations and compare them. Slow but not the type of query that will be done often, and possibly indexable.

9.6.1.1. Possible dynamic sql db

One possible design is to just use plain old sql (sqlite?) and create normal tables. However the table names would be namespaced, possibly with some sort of hash. That way, one person's "Customer" schema could be in the same database as another person's without interfering.

So for example, the kcats language might need to keep track of library dependencies. There could be a table dependencies-01234abcd with columns name, version, hash, blob etc. Anything else wanting to use the same schema could refer to it by hash. It would be possible to have foriegn keys too.

One thing we want to avoid is having kcats users writing sql query strings, that is not the idiomatic way of dealing with i/o. What should happen is there's a query word that takes a program and db descriptor of where the db is, and returns a pipe (where results come out). The program is a "query equivalent" and would need to be translated to sql and post-processed. This is very much nontrivial and a naive implementation probably wouldn't perform well but we will try it anyway. For example instead of writing

insert into Customers (name, age) values ("Bob", 25); 
select * from Customers where name="Bob";

you'd write something like

customers [[name "Bob"] [age 25]] put

customers [[name] lookup "Bob" =] filter

and And then the translation would see we're selecting from customers, then there's a filter. The filter might not translate to sql so it will either just select all, or if it sees a certain format for the predicate it can translate to a where clause. This is going to be complex and bug prone but hopefully can be done in a way that the worst case is poor performance and then iterate to get better speed.

I suppose content distribution might need to be done alongside this.

9.7. DONE Clean up all the vector conversion

I've been calling vec a lot, sometimes just so the list will print out with square braces. I now have a repr function that could do this, so using vec for that purpose is no longer needed.

However, I can't get rid of all of them- for example, calling conj on a vector vs list adds at different ends of the list so they are not interchangeable in that respect. It may be dangerous to leave any lists lying around if they might get conjed onto expecting it to go on the end.

9.8. DONE org-babel-execute for kcats

9.9. INPROGRESS At least one example for each word in lexicon

10 0.5 *
5
"foo" encode
#b64 "Zm9v"
[[a b] [c [[d e]]]] [c d] 5 assign
[[c [[d 5]]] [a b]]
[[a b] [c []]] [c] [[d 5]] association assign
[[c [[d 5]]] [a b]]
[[a b] [c [[d e]]]] [1 1 0 1] 5 assign
[[a b] [c [[d 5]]]]
[[a b] [c [[d e]]]] [1 0] 5 assign
4 3 [>] shield [wrap [wrap] dip] dip sink branch 
4
yes 4 2 branch
[[asked [program]] [reason "type mismatch"] [type error] [unwound [branch]]] 2 4 yes
5
[1 2 "oh fudge"]
[[+]
 []
 recover]
map
[[[type error] [reason "word is not defined"] [asked [handle]] [unwound []]] [[unwound []] [asked [handle]] [reason "word is not defined"] [type error]] [[asked [number]] [type error] [reason "type mismatch"] [unwound [+]]]] 5
5 1 [+] [] recover
[[unwound []] [asked [handle]] [reason "word is not defined"] [type error]] 1 5
1 type
number
5.01 5 0.1 swap [- abs] dip <
yes

9.10. INPROGRESS Prime number sieve example

2000 clone 2 swap range ;; all the numbers up to n

[sqrt 2] dip  ;; start counter at 2, stop at sqrt of n
[sink =] ;; stop loop when the counter hits sqrt n
[[drop drop] dip]  ;; drop the original args, just leaving the primes
[[[[=] 
   [swap mod positive?]]
  [execute] any?] 
 filter ;; keep the counter but no multiples of it 
 [inc] dip] ;; increment counter
[execute]
recur
[[asked [consume]]
 [handled yes]
 [reason "not enough items on stack"]
 [type error]
 [unwound [sqrt 2 [[]] 🍫 [sink =]
           [[drop drop]
            dip]
           [[[[=] [swap mod positive?]]
             [execute] any?]
            filter [inc] dip]
           [execute] recur]]]

Here's a mimic of the python version, WIP:

;; num
10
[[[] [yes put]] dip times] shield ; a n
2 ;; p a n
[swapdown clone * > ] ;; while test
[[wrap lookup] ; if test - fetch by index
 [
 swapdown ;; p n a
 clone ; p
 clone * ; p^2 p n a
 ;; range wants p, n+1, p^2 
 sink ;; p n p^2
 [inc] dip ;; p n+1 p^2
 [range] shield ;; r p n+1 p^2 a
 [dec sink drop] dipdown ;; r p a n
 swapdown ;; r a p
 [ ;; i r a p
  wrap ;;swapdown ;; [i] a r p
  [[]] update ;; set to false: a r p
  swap ;; r a p
 ]
 step ;; a p
 swap 
 ] ; do the for loop
 [] ; else do nothing
 if
 inc ;; p++
]
while 
[[actual 2]
 [asked [sized]]
 [handled yes]
 [reason "type mismatch"]
 [type error]
 [unwound [update swap [8 10]
           [wrap [[]] update swap] step swap inc [swapdown clone * >] shield [[wrap lookup]
                                                                              [swapdown clone clone * sink [inc] dip [range] shield [dec sink drop] dipdown swapdown
                                                                               [wrap [[]] update swap] step swap]
                                                                              [] if inc [swapdown clone * >] shield]
           loop]]]
[[]] [6] 2 [yes yes yes yes [] yes yes yes yes yes]
10
[yes yes yes yes yes yes yes yes yes yes]
[] [[yes] 15 times] inject
2 swap ;; p a
[clone clone *] dip swap ;; p^2 a p
[[[📏] shield] dip swap [<] shielddown] ;; b p^2 a p  
[[wrap [drop []] update] shield ;; do the update 
 float drop sink [+] shielddown swapdown] ;; 
;while
[[wrap [drop []]
  update]
 shield float drop sink [+] shielddown swapdown]
[[[📏] shield]
 dip swap [<] shielddown]
4 [yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes]
2

How do we write this code? Generally, how do we decide what order things go on the stack?

It looks like the array of bools is the main piece of data here, that is used throughout the algorithm. The other commonly used variable is p, the one that's incremented. I think probably p should remain on top. The outermost loop needs to know when to stop, and that needs to compare to num. That can go on the bottom.

The inner loop uses i. That should probably replace p on top when in use. So it should be [p a] and later [i a p].

Now that lingo exists, maybe should also write let for variables (where the values are evaluated before updating the dictionary)? Also these aren't actually "variables" because you can't change the value, without an inner let.

Actually this is probably best implemented in two parts:

  • a word that takes a set of bindings and evaluates the values, leaving a map of word to value
  • a word that takes the map above and inserts it into the dictionary. I think lingo does this already.

let's try to write the former here. I think we need map-values type of thing here, which requires treating a map as a list.


[[[a [+ 5 6]]
  [b [- 100 8]]]
 [a b +]
 let] 

ok round 2 here, let's just do the updating loop first: expect p, a, n, and modify a in place such that all multples of p are flipped to [].

10
clone yes swap repetition
2

first though let's just produce the array of indices to mark from p and n.

10 2
clone sink range
[2 4 6 8]

now given that and a, mark all those indices

1000 clone yes swap repetition 
2
swapdown
[clone * >]
[[•🗑️ wrap lookup]
 [[clone sink range rest] shield sink 
  [[wrap [] assign] step] dipdown]
 when
 inc]
while
drop drop
indexed
[second] filter [first] map
rest rest
[2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71
 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163 167 173
 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263 269 271 277 281
 283 293 307 311 313 317 331 337 347 349 353 359 367 373 379 383 389 397 401 409
 419 421 431 433 439 443 449 457 461 463 467 479 487 491 499 503 509 521 523 541
 547 557 563 569 571 577 587 593 599 601 607 613 617 619 631 641 643 647 653 659
 661 673 677 683 691 701 709 719 727 733 739 743 751 757 761 769 773 787 797 809
 811 821 823 827 829 839 853 857 859 863 877 881 883 887 907 911 919 929 937 941
 947 953 967 971 977 983 991 997]

Now take an array of bools and filter by index

[yes [] yes] [📏] shield [0] dip 1 range swap zip
[second] filter [first] map
[0 2]

this is too inefficient (copying the whole array due to the shield

[
wrap [] assign] shield

Need to avoid using shield, as it causes copying of the large array.

[1 2 3] [0] 100 [pair] shield [assign] dip 🍫
100 [0] [100 2 3]

9.11. TODO FAQ entry about composability

Kcats is difficult to use in the same mindset as other languages. The amount of complexity you can fit into a single function or subroutine, before it gets too difficult to reason about, is quite a bit smaller. However kcats is designed to be used this way. The kcats way is building small combinators, and then using those combinators to build what you need.

Kcats has some distinct advantages here:

  • The overhead of putting pieces together is basically zero, since function composition is the default.
  • Combinators are very easy to test independently.

Kcats makes it easy to test combinators independent of the context where you will use it. In purely functional code, this is straightforward: you just place some sample data on the stack and then the combinator you want to test, then run it.

With i/o it's not quite as straightforward. However kcats pipes are designed to be easily swappable for regular values. Let's say you have some code that expects to read from a file but you want to test it without having to maintain state in your local filesystem:

"bar" file-out
take string
"Hello World!" [[type tunnel]
                [values [[type bytes]]]
                [to [[file "bar"]]]]

Swap the file-out pipe for a plain list of byte arrays, they have the same interface, and gives the same result:

[#b64 "SGVsbG8gV29ybGQh"]
take string
"Hello World!" []

and of course you can build that byte array out of strings, if you prefer:

["Hello World!"] [encode] map
take string

"Hello World!" []

This technique works for any use case where you're using a pipe for its usual take/put/step interface, and timing is not critical.

9.12. TODO Inconsistent Type System   ai_lint

The type system shows several inconsistencies:

Some functions use specific types (number, string) while others use generic types (item a, item b) Some functions define both input and output types, while others only have input types Some functions have * as their output type without explanation The relationship between "traits" and types isn't clearly defined

9.13. TODO Contradictory Security Model for Dictionary Modifications   ai_lint

The documentation makes a strong statement:

One hard rule of kcats: "an environment cannot alter its own dictionary"

But later contradicts this:

Kcats does support both temporarily and permanently monkey patching. See the words define for permanent and lingo for temporary.

This presents a fundamental inconsistency in how the language handles security and side effects.

9.14. TODO AI model training from this documentation

While it's possible to train an AI model to write kcats using this file unmodified, it's better to break it down into Q/A flashcards. ChatGPT is pretty good at doing that, so if we need to rerun that procedure, this is the prompt:

= KCATS FLASH-CARD GENERATOR =============================================

You are kcats-FCG, a tool that converts prose facts in the kcats manual into instruction–answer pairs for fine-tuning.

INPUT

  1. This context block (the part you are reading now).
  2. The full kcats documentation in Org-mode or Markdown.

TASK A. Parse the doc; ignore code blocks (`#+BEGINSRC`, triple-fence, etc.).

B. Scan the prose (ignore code fences).

Identify every distinct piece of instructional knowledge that helps a reader understand kcats, including (but not limited to) • purpose and goals • design (high & low) • conventions • limitations • behaviour (semantics, runtime, I/O) • techniques & idioms • standard-lib usage.

A “fact” F may be • a single sentence or • a tight group of consecutive sentences/headings that together express one idea. Treat “idea-level” granularity as canonical; do not discard meaning that is split across sentences.

Extract every such fact; do not skip ideas just because they are implicit, short, or follow immediately from the previous sentence or heading.

Skip narrative, jokes, anecdotes.

C. For each fact F generate 3 Q/A variants:

Q1 (direct) : Ask for the fact explicitly. Q2 (boolean) : A true/false or yes/no question whose correct answer is an affirmation of F. Q3 (wh-word) : “Why / How / Which / When …” style paraphrase.

Keep each question ≤ 120 characters; answers ≤ 200 characters. Do not invent extra facts; keep wording faithful but concise.

D. Emit one JSON object per (question, answer) pair, newline-delimited:

{"prompt": "<question>", "answer": "<answer>"}

– No additional fields. – Escape double quotes and backslashes. – End output with a final newline, no wrapper, no markdown fence.

OUTPUT Pure JSONL: many lines, each a single compact object as above.

EXAMPLE Fact : “Kcats treats the empty list as false; any other value is true.” Lines : {"prompt":"What value is considered false in kcats?","answer":"The empty list (🔳)."} {"prompt":"True or false: kcats has dedicated boolean literals.","answer":"False. It uses the empty list as false and any other value as true."} {"prompt":"How does kcats represent truthiness?","answer":"Everything except the empty list is treated as true."}

NOTES • Keep Unicode (emoji) tokens intact. • Do NOT include runnable examples or stack traces; that corpus is built elsewhere. • Deterministic: if the same doc is given twice, produce the same cards in the same order (e.g. fixed template order Q1,Q2,Q3).

END OF CONTEXT

example output

{"prompt":"What is an item in kcats?","answer":"A unit of information that can be a number, string, byte array, word, character, or list."}
{"prompt":"True or false: In kcats an item may be a byte array.","answer":"True.  Items include numbers, strings, byte arrays, words, characters, and lists."}
{"prompt":"Which kinds of values can make up a kcats item?","answer":"Numbers, strings, byte arrays, words, characters, and lists."}
{"prompt":"In kcats, what is a list?","answer":"An item that contains other items in order and is written inside square brackets."}
{"prompt":"Yes or no: A kcats list keeps its elements ordered.","answer":"Yes.  A list is an ordered container of items."}
{"prompt":"How are lists written in kcats and what do they hold?","answer":"They use square brackets and hold other items in a defined order."}
{"prompt":"What is a program in kcats terminology?","answer":"A list of instructions intended to be executed by a machine."}
{"prompt":"True or false: A program in kcats is just a list of instructions.","answer":"True.  Programs are instruction lists meant for execution."}
{"prompt":"Which term denotes an executable instruction list in kcats?","answer":"Program."}
{"prompt":"What is the stack in kcats?","answer":"A first-in, first-out list where a program stores all data it manipulates."}
{"prompt":"True or false: The kcats stack holds every value a program works with.","answer":"True.  All data is kept on the stack instead of in variables."}
{"prompt":"Why does kcats rely on a first-in, first-out list called the stack?","answer":"Because it stores and supplies all values the program needs while executing."}
{"prompt":"What is a word in kcats?","answer":"An action that consumes items from the top of the stack, produces new items, and can be named with text, symbols, or emoji."}
{"prompt":"True or false: Words in kcats can be emoji like 🔀.","answer":"True.  Word names may be plain text, symbols, or emoji."}
{"prompt":"How do words interact with the stack in kcats?","answer":"They take items from the top, perform an operation, and push resulting items back."}
{"prompt":"What is an axiom word?","answer":"A word that is not defined in terms of other words."}
{"prompt":"Yes or no: An axiom word depends on existing word definitions.","answer":"No.  Axiom words stand alone without relying on other words."}
{"prompt":"Which kind of word lacks a definition made from other words in kcats?","answer":"An axiom word."}
{"prompt":"What is a definition in kcats?","answer":"A description of what a word should do, given either in the base language or as another program."}
{"prompt":"True or false: A kcats definition describes a word's behavior.","answer":"True.  Definitions state how a word should act."}
{"prompt":"How is a word's behavior specified in kcats?","answer":"Through its definition, written in the base language or as a program."}
{"prompt":"What is a dictionary in kcats?","answer":"The collection of available words together with their definitions."}
{"prompt":"Yes or no: The dictionary stores word definitions.","answer":"Yes.  It maps every word to its definition."}
{"prompt":"Which structure holds all words and their definitions in kcats?","answer":"The dictionary."}
{"prompt":"What does the environment include in kcats?","answer":"The complete state of a running program: its stack, current program, and dictionary."}
{"prompt":"True or false: The kcats environment contains the program's stack.","answer":"True.  The environment bundles stack, program, and dictionary."}
{"prompt":"Which elements make up a kcats program environment?","answer":"Stack, program, and dictionary."}
{"prompt":"What is a pipe in kcats?","answer":"A conduit for items to move between environments or to the outside world and back."}
{"prompt":"True or false: Pipes let kcats environments exchange items.","answer":"True.  A pipe carries data between environments or external systems."}
{"prompt":"How does kcats enable communication between environments?","answer":"Through pipes that transport items from one place to another."}
{"prompt":"Instead of variables, what does kcats use to track values?","answer":"A stack; all needed values are pushed to and popped from it."}
{"prompt":"Yes or no: Kcats relies on traditional variables for data.","answer":"No.  It keeps data on a stack rather than in variables."}
{"prompt":"How does kcats keep track of values during execution?","answer":"By pushing them onto a stack instead of storing them in variables or parameters."}
{"prompt":"When printing the stack, which element appears first?","answer":"The top of the stack, i.e., the last item pushed."}
{"prompt":"True or false: Kcats prints the bottom of the stack first.","answer":"False.  It prints from the top, so the last item in appears first."}
{"prompt":"Why does the last pushed item print first in kcats?","answer":"Because printed output starts at the stack's top (last in, first out)."}
{"prompt":"How are lists denoted in kcats source code?","answer":"With square brackets, e.g., [1 2 3]."}
{"prompt":"Yes or no: Kcats uses parentheses to mark lists.","answer":"No.  It uses square brackets."}
{"prompt":"Which brackets indicate a list literal in kcats?","answer":"Square brackets."}
{"prompt":"When is a kcats program considered finished during execution visualization?","answer":"When nothing remains to the right of the \"|\" separator; the leftover stack is the result."}
{"prompt":"True or false: If code remains right of the |, the kcats program has finished.","answer":"False.  The program ends only when no code remains right of the |."}
{"prompt":"How does the | separator show program completion in kcats?","answer":"Once no instructions remain on the right side, execution is done and the remaining stack is the result."}
{"prompt":"Does kcats enforce a specific code format?","answer":"No.  You can format code any way you like; the language ignores whitespace layout."}
{"prompt":"True or false: Kcats is strict about code indentation.","answer":"False.  It does not care about formatting."}
{"prompt":"Why can kcats users format code arbitrarily?","answer":"Because kcats does not depend on whitespace or layout for syntax."}
{"prompt":"How can you access standard library docs in kcats?","answer":"By querying the documentation from within the language itself during interactive use."}
{"prompt":"Yes or no: Kcats lets you query its standard library docs via code.","answer":"Yes.  The language can fetch its own documentation interactively."}
{"prompt":"Which feature lets kcats programmers explore library docs interactively?","answer":"Built-in queries that retrieve documentation from inside the language."}
{"prompt":"Is kcats suitable for huge datasets or high-performance number crunching?","answer":"No.  It is aimed at personal automation, not heavy data or computation loads."}
{"prompt":"True or false: Kcats targets high-performance numerical tasks.","answer":"False.  It prioritizes simplicity over raw performance."}
{"prompt":"Why is kcats not recommended for massive data processing?","answer":"Its design favors expressivity and simplicity, so it lacks the performance needed for huge datasets or heavy number crunching."}

{"prompt":"What is an item in kcats?","answer":"A unit of information such as a number, string, byte array, word, character, or list."}
{"prompt":"True or false: In kcats, numbers and strings are both considered items.","answer":"True. Numbers, strings, byte arrays, words, characters, and lists are all items."}
{"prompt":"Which kinds of values does kcats group under the term \"item\"?","answer":"Numbers, strings, byte arrays, words, characters, and lists."}
{"prompt":"How does kcats define a list?","answer":"A list is an item that contains other items in order and is delimited by square brackets."}
{"prompt":"Yes or no: A list is delimited by square braces in kcats.","answer":"Yes. Square brackets mark the start and end of a list."}
{"prompt":"What structure holds ordered items and is shown with square brackets in kcats?","answer":"A list."}
{"prompt":"What is a program in kcats terminology?","answer":"A list of instructions intended to be carried out by a machine."}
{"prompt":"True or false: A kcats program is just a list of instructions.","answer":"True. In kcats, a program is defined as such a list."}
{"prompt":"How does kcats describe the thing executed by a machine?","answer":"As a program — a list of instructions."}
{"prompt":"In kcats, what is the stack?","answer":"A list with a first-in, first-out interface where the program stores all its data."}
{"prompt":"True or false: The stack in kcats follows first-in, first-out order.","answer":"True. The stack is defined that way in kcats."}
{"prompt":"Where does a kcats program store all the data it needs?","answer":"In the stack."}
{"prompt":"What does a word do in kcats?","answer":"It performs an action, usually consuming items from the top of the stack and producing new ones."}
{"prompt":"Yes or no: A word usually consumes items from the top of the stack.","answer":"Yes. Words operate on the topmost stack items."}
{"prompt":"How are operations carried out on the stack in kcats?","answer":"Through words that take items from the stack's top and push results back."}
{"prompt":"How can kcats words be represented?","answer":"As English words, math symbols, or emoji; their names are not part of the language syntax."}
{"prompt":"True or false: Emoji can be used as word names in kcats.","answer":"True. Word names may include emoji, symbols, or text."}
{"prompt":"Which naming flexibility does kcats allow for its words?","answer":"Words may be named with text, symbols, or emoji; the name itself is just a label."}
{"prompt":"What is an axiom word in kcats?","answer":"A word that is not defined in terms of other words."}
{"prompt":"True or false: An axiom word is not defined in terms of other words.","answer":"True. That is the defining property of an axiom word."}
{"prompt":"Which kind of word lacks a definition built from other words?","answer":"An axiom word."}
{"prompt":"In kcats, what is a definition?","answer":"A description of what a word should do, written in the base language or as a program."}
{"prompt":"Yes or no: A definition describes what a word is supposed to do.","answer":"Yes. That is exactly its purpose in kcats."}
{"prompt":"How does kcats describe the intended behavior of a word?","answer":"With a definition that explains what the word should do."}
{"prompt":"What is a dictionary in kcats?","answer":"A set containing available words and their definitions."}
{"prompt":"True or false: A kcats dictionary maps words to their definitions.","answer":"True. That mapping is the dictionary's role."}
{"prompt":"Which component holds the available words and their meanings in kcats?","answer":"The dictionary."}
{"prompt":"What does the term environment mean in kcats?","answer":"The complete state of an executing program, including stack, program, and dictionary."}
{"prompt":"Yes or no: The kcats environment includes the current stack.","answer":"Yes. The stack is part of the environment."}
{"prompt":"Which structure represents the full state of a running kcats program?","answer":"The environment, encompassing stack, program, and dictionary."}
{"prompt":"What is a pipe in kcats?","answer":"A conduit for sending items between environments or to external destinations."}
{"prompt":"True or false: Pipes allow items to move between environments or to the outside world.","answer":"True. That is their purpose."}
{"prompt":"How does kcats describe the conduit used to transfer items between contexts?","answer":"As a pipe."}
{"prompt":"How does kcats keep track of values instead of using variables?","answer":"By pushing values onto a stack instead of using variables and parameters."}
{"prompt":"Yes or no: Kcats relies on a stack rather than variables and parameters.","answer":"Yes. All values are managed via the stack."}
{"prompt":"Which data structure replaces traditional variables in kcats?","answer":"The stack."}
{"prompt":"Where do words operate on items in kcats?","answer":"On the top of the stack, modifying those items."}
{"prompt":"True or false: A word may remove or add items only at the stack's top.","answer":"True. Words manipulate the top stack items."}
{"prompt":"How do words manipulate data in kcats?","answer":"They act on the topmost stack items, possibly removing, reordering, or adding values."}
{"prompt":"Conceptually, what are words in relation to the stack?","answer":"Functions that take the current stack and produce a new stack."}
{"prompt":"True or false: Each word maps one stack state to another.","answer":"True. Words are stack-to-stack functions."}
{"prompt":"Why can words be thought of as stack functions?","answer":"Because each word consumes a stack and returns an updated stack."}
{"prompt":"Which characters denote lists in kcats?","answer":"Square brackets [] denote lists."}
{"prompt":"Yes or no: Square brackets are used to mark lists in kcats.","answer":"Yes. Lists start and end with square brackets."}
{"prompt":"How does kcats mark a list in source code?","answer":"By surrounding its items with square brackets."}
{"prompt":"What happens when kcats encounters a list literal during execution?","answer":"The entire list is pushed onto the stack as one item."}
{"prompt":"True or false: A list is pushed onto the stack as one item.","answer":"True. Lists remain single units on the stack."}
{"prompt":"How is a literal list handled by the kcats interpreter?","answer":"It is placed on the stack as a single unit."}
{"prompt":"When a kcats program finishes, where is the result found?","answer":"Whatever remains on the stack is the program's result."}
{"prompt":"Yes or no: The leftover stack after execution holds the program's result.","answer":"Yes. The remaining stack constitutes the result."}
{"prompt":"Which part of the environment holds the final value after program completion?","answer":"The stack."}
{"prompt":"Does kcats impose formatting rules on code?","answer":"No. You can format code however you like; kcats ignores formatting."}
{"prompt":"True or false: Kcats ignores whitespace and formatting.","answer":"True. Code layout is insignificant to the language."}
{"prompt":"Why can developers format kcats code any way they like?","answer":"Because kcats does not care about code formatting or whitespace."}
{"prompt":"What kinds of tasks is kcats not suitable for?","answer":"Processing huge datasets or high-performance number crunching."}
{"prompt":"True or false: Kcats is currently unsuitable for high performance number crunching.","answer":"True. The language prioritizes simplicity over raw speed."}
{"prompt":"Which workload type does kcats deliberately avoid optimizing for?","answer":"High-performance number crunching or very large data processing."}

You are given multiple “stdlib entries” written in the kcats (sexp-like) language. Each entry looks like

[<function-name or glyph> [[spec …] [doc "…"] [definition […]] ;; present for user-defined words [examples [[[<prog1>] [<equivalent1>] "comment1"] [[<prog2>] [<equivalent2>] "comment2"] …]] ] ] ]

Tasks ===

  1. Produce training rows in JSON-Lines. Each line is a JSON object with the fields

    { "prompt": "<prompt shown to the model during training>", "answer": "<the desired answer>" }

  2. Emit two kinds of rows, all in the same stream:

C. Definition-request rows Only for entries that have a `[definition …]` field (i.e. they are not axiom words). Produce

input = "Write <glyph> in kcats." output = <definition> (emit the s-expression exactly)

D. Reverse-lookup rows Also only for non-axiom words. Re-phrase the doc as an instruction, answer with the definition. Example pattern:

input = "Create a recursive programme that “runs predicate, exits when false, otherwise body then combinator”." output = <definition>

  1. Keep the original ordering of lines for determinism: all rows of type A first (in the order the examples appear), then B, then C, then D.
  2. Escape any inner quotation marks so the resulting JSON is valid. Do not pretty-print; one compact JSON object per line.
  3. Do not wrap the whole output in an array, header, or explanation—only the JSONL rows.

Example =====

Input you will receive (shortened):

[👥 [[spec …] [doc "The twins denote copying the ToS."] [examples [[[1 2 3 👥] [1 2 3 3] "Clone integer"] [[🔳 👥] [🔳 🔳] "Clone Nothing"]]]]]

Expected JSONL you must emit:

{"input":"[1 2 3 👥]","output":"[1 2 3 3]"} {"input":"[🔳 👥]","output":"[🔳 🔳]"} {"input":"What does 👥 do?","output":"The twins denote copying the ToS."}

(There is no definition for 👥, so no rows of type C or D.)

That’s it—apply these rules mechanically for every entry you are given and output the corresponding JSON-Lines.

10. Acknowledgements

  • Kcats is inspired by Joy, created by Manfred Von Thun.
  • Tons of implementation ideas taken from Thun, a dialect of Joy written in python by Simon Forman
  • Many ideas taken from Clojure by Rich Hickey, and other lisp dialects.

11. Roadmap Notes

11.1. Higher level persistence abstraction

I wonder whether kcats should have any notion of files and sockets at all. Sort of like java doesn't have any notion of memory addresses or malloc/free - it operates at a higher level and handles mem management for you. Maybe kcats handles persistence for you. This may be a sort of chicken/egg problem where I need a network protocol to help w persistence and I want that protocol to include kcats as a language. Can they be bootstrapped as a single unit? Seems possible but not easy. Persistence might involve having another party store data for you, which might involve identity (to limit access) and money (to incentivize someone to keep your data for later). That might be a bit of a reach for a programming standard lib to handle.

And then there's the question of interop with other programs, how would they communicate if kcats doesn't know what a file or socket is? Maybe it can know what a file/socket is but you don't need to use it except as interop (like clojure's java interop or java's jni).

So what would this look like?

Instead of telling the program where to persist, you just want it persisted and you get a sort of claim check (maybe the hash of the data?). Then to get it back later, you present the claim check. Persistence is a best-effort deal (you can't be 100% sure no disaster could wipe it out). So maybe also include some optional params to indicate:

  • how long until you might need this again
  • how long you can wait between requesting it and getting it
  • how disaster-proof it needs to be
  • how much you're willing to pay to store it

Maybe we can even put messaging under this model - after all, sending someone a message is in fact making a copy of data you have. You don't necessarily want to retrieve it later though.

Computing might be better thought of as a worldwide resource - you might not be able to trust someone else to do a computation for you (yet, unless it's a specific type where you can verify without doing the full computation yourself) but you can trust them with storage (given enough redundancy - they can't steal your data because it's encrypted).

11.2. Object construction, caching

Often we create objects similar to java construction, where the input and output are informationally equivalent (you can reconstruct the output from the input anytime you want, and sometimes vice versa).

It might be nice if kcats didn't force you as a user to do this type of operation and just let you use the original data.

For example, lets say you have [[file "/tmp/foo"]]. That's an association of file (a type) to a string. Really what that means is we're referring to a file on disk. In java we'd construct a File object with new File("/tmp/foo"). It'd be nice if everywhere in kcats you never needed a File object and could use the original descriptor instead (or a pipe you've already created, if state matters). On the jvm platform obviously somewhere a File object would get created but that should be hidden from view. How would that work?

I thought of a word like derive that caches these things? Maybe it would keep a cache of previously derived things and just return the answer if asked again (like memoized function in clojure and could even be implemented that way). It would also have a mapping of how to derive one thing from another. eg [[file "foo"]] and create a pipe-in to write to it. You'd first need an inputstream to the file (as inputstream is what the pipe protocol is actually using).

The thing is, inputstreams are not values. They're stateful, pointers to places on disk. So we probably can't cache them nor need to.

derive would be more for things like crypto keys created from a seed.

For pipes, we need to go from a descriptor, to some platform specific object, to a pipe. How do we keep platform specific code isolated? I'm hesitant to make public abstractions for anything but pipes. I don't want a file word that creates file objects from descriptors, kcats users should never see that. The only solution I can think of is to just leave the platform-specific code where it is, and have some kind of switching mechanism like clj/cljs has.

Footnotes:

1

Inspired by Alan Kay's quote "Lisp isn't a language, it's a building material".

2

Function names can and do have symbols, eg + for addtion, but it's just a name and it's not part of the language syntax. You can change the name to something else very easily. There are some symbols currently in the language that are implementation detail leaks that will be removed as soon as a suitable replacement is found (for example, byte array literals are base64 encoded strings with a prefix of #b64, which comes from EDN syntax, which is what kcats is currently using for serialization).

3

Why is it done this way instead of just giving a program and its expected result? Some expected results don't have literal representations, so we can't directly input them, we have to calculate them from the input.

4

A cryptographic hash is a function that converts an arbitrary block of data into a fixed-size string of bytes. The function is designed to make it computationally infeasible to find an input that gives a particular output. This property, along with the fact that even a small change in the original data produces a significantly different hash, makes cryptographic hashes essential for content integrity checks. By comparing the computed hash of the content at two points in time, one can verify that the content has not been altered, providing a simple yet powerful tool for ensuring data integrity and security.

Author: Skyrod Vactai

Created: 2025-07-23 Wed 01:40

Validate