Lisp languages should be great for embedding into games to implement game logic, instead of more straightforward solutions such as embedding Lua. I've always wanted to explore this for my iOS side-project game, which now uses an event-based system inspired by ReactiveX, implemented in Lua. This works pretty well but is very hard to debug.
The main thing holding me back to try to replace it with something Lisp-like is that I have zero experience in Lisp programming. I know some FP theory (in fact the Lua-based event system lends some concepts from it), but when I try to reason about common subproblems I really have no idea how to map them to something like Lisp. For example how to efficiently implement a message bus with observers that can register themselves, using state-based functions to execute when certain events are received, scheduling events, async/await to suspend processing until some event is received, etc.
Does anyone have pointers to books or websites to learn 'real-world' LISP programming? In other words not yet another introduction that shows the same old forced examples that just show you how to evaluate an expression or use higher-order functions to solve artificial toy problems, but resources that teach you how to be productive in a Lisp language?
> but when I try to reason about common subproblems I really have no idea how to map them to something like Lisp. For example how to efficiently implement a message bus with observers that can register for certain events.
I think that's a badly posed question. The main benefits of using Lisp are its introspection and interactivity. You can use the REPL to inspect your message bus at runtime, you can modify parts of it as it runs, you can interactively debug errors as they happen, but none of that is directly related to "how to implement a message bus".
I think that is more of an ADS question that it is a Lisp question; an efficient message bus, no matter whether you implement it in Lisp or in C or in Haskell, needs some synchronization primitives from the programming language core that are likely backed by OS primitives, as well as some data structures that need to be cleverly designed to be fast. Neither of these is language-specific - only means of implementing and using those might be language-specific.
In other words: Lisp makes it possible to be more productive by means of dramatically reducing the time of feedback loop and by means of providing in-depth language introspection, but it doesn't solve the problem of actually needing to figure out how to design programs and data structures. That's why e.g. SICP is not a Scheme tutorial, despite using Scheme throughout the whole book.
This is a very classic Lisp answer in that "productivity" is considered in the pure abstract, and that the specific question of how to do a specific thing - the actual productive result - is uninteresting. Who would expect the classic book on Scheme to teach you how to write Scheme programs?
SICP isn't a book on Scheme. It is not a Scheme tutorial; it uses Scheme to teach programming. One can learn Scheme by following the book and inferring everything that is done in there, but the topic of the book isn't teaching Scheme, it's teaching programming.
Regarding productivity, I don't consider it abstract. I feel a concrete efficiency boost from the two things I mentioned, which are interactive and incremental development with a very short feedback loop (no time wasted while waiting for the toolchain to do its work) and the ability to introspect and debug the live system (since I do not need to bother with external debugger and tooling).
The original post complains of not knowing how to solve their particular problem in Lisp, and gives an example:
> how to efficiently implement a message bus with observers
You say:
> I think that's a badly posed question
But it isn't! It's the specific thing that person is trying to do and is having trouble with, the act of translating structures and designs they understand from other languages into the Lisp language.
This is the barrier to Lisp adoption. You can cite iterative development and live debugging all you want, but they only benefit after you've actually written a program, which is the barrier the potential users are struggling at.
> but they only benefit after you've actually written a program, which is the barrier the potential users are struggling at
No, you miss the point. There is little benefit in interactive and incremental development once a program is written. The very benefit that this such incrementality provides is noticeable during the process of writing the program, not after it; when a program is written, it's written, and from some point it doesn't really matter if you've done it by mutating an image-based programming language or by linking together compiled C objects.
Programs are never finished. Most work is incremental modification and debugging of existing systems. The "getting started" bit of going from a blank editor to something that starts to be useful is a small part of the process. But it's the very beginning that is a surprisingly hard barrier to cross. The starting to write a program in the first place. Which is where the OP has got stuck. And is dismissed as irrelevant?
Maybe I phrased that incorrectly indeed. It's not so much that I want to find a 'better way to implement a message bus', but more 'if I want to re-implement the message bus I already have, how can I use the features of a Lisp-like languate to improve the implementation'.
For the logic in my iOS game I started out with a straighforward imperative system based on objects and explicit state, which quickly devolved in a big mess of if-then-else-but-only-if spaghetti code that was impossible to maintain. I refactored that to a reactive system based on what I call an 'event graph' where each node performs some kind of filtering or processing. For example nodes like 'filter on event type', 'take first 4 events', 'complete when event X received', etc. Game logic is implemented by branching off (subscribing) to nodes, synchronizing stuff by means of attaching a temporary subgraph that waits for a 'completion event' etc. Like I said it's quite similar to ReactiveX but simpler.
The basic concepts behind this are sound, but my feeling is that I need way too much code in to implement something like this in Lua, compared to what I would expect in something like Lisp, which seems more naturally suited for this kind of task. I just find it hard to get started trying to map the solution I already have to a Lisp language, because most of the 'introduction to Lisp'-like resources are so focused on artificial examples that are mostly interesting from a computation-theoretic point of view.
In terms of the basic tools available to you, Lisp and Lua are remarkably similar. In both you will rely heavily on closures to get the job you mentioned done. Lua offers the table as its primary data structure, though this can also be used as an array. Lisp offers the list, though this can also be used as a table (associative container, whatever).
Where Lisp and Lua differ is that Lisps are infinitely more malleable as languages. You can more-or-less invent new syntax to assist in your problem domain. For example, if you find that constructing your 'event nodes' requires boilerplate in Lua, you can write Lisp macros that make things clear again. Or you might write a little macro DSL to make constructing your graphs easier.
I enjoy Lua, but I constantly pine for the flexibility of a Lisp when Lua's clunky syntax annoys me. I find Fennel is a good compromise between the two - especially if you already have Lua code that you want to port over.
I liked the book Practical Common Lisp (PCL) very much, you can read it online for free on the authors website: http://www.gigamonkeys.com/book/
I suggest using portacle (a preconfigured emacs) as development environment during that course as this is the easiest way to get cracking in CL imho.
Although Janet's core library is different from CL, I think once you grok LISP, it's not hard to switch to another dialect:
After working through ~half of PCL I abandoned it in favour of a learning-by-doing approach in ClojureCLR as I work mostly in .NET or Mono atm.
If you’re interested in Common Lisp, Practical Common Lisp is a good overview of the language [1]. Since you’ve done game programming, here’s a simple snake game I wrote a while back; you might find it useful to peruse the code [2]. You can find a couple of bigger libraries I wrote on my github if you’re interested in seeing more substantial code.
for libraries, see https://github.com/CodyReichert/awesome-cl (and see the lparallel or lfarm libraries, channels, cl-gserver, the built-in SBCL threading functions, etc)
Also note that CL is not purely functional, you can throw in FP or OOP or iterative code.
The real power of Lisp becomes apparent when you develop programs top-down with very general data structures with their own accessors and a liberal use of higher-order functions.
I still think that Winston & Horn's classic Lisp (3rd ed.) conveys these ideas very well. SICP is also very good at it, but it's based on Scheme.
Learn GNU Guile, it's a dialect of Scheme. AFAIK, Scheme is specifically intended for use in teaching programming. I spent a late night reading the manual and hacking out some toy programs, and the rest is history.
There is Nu which runs on top of the Objective-C runtime: https://programming-nu.github.io/
Relative easy to read code and has actually been used with some iOS games.
The main thing holding me back to try to replace it with something Lisp-like is that I have zero experience in Lisp programming. I know some FP theory (in fact the Lua-based event system lends some concepts from it), but when I try to reason about common subproblems I really have no idea how to map them to something like Lisp. For example how to efficiently implement a message bus with observers that can register themselves, using state-based functions to execute when certain events are received, scheduling events, async/await to suspend processing until some event is received, etc.
Does anyone have pointers to books or websites to learn 'real-world' LISP programming? In other words not yet another introduction that shows the same old forced examples that just show you how to evaluate an expression or use higher-order functions to solve artificial toy problems, but resources that teach you how to be productive in a Lisp language?