When I worked at Instagram, most of the codebase was in Python but occasionally I’d see these bindings into C++. I never really understood how that worked and just took it for granted since some other infra team had set that up.
In this conversation with Roberto Ierusalimschy (the creator of Lua), he finally clarified how this cross-language mechanism works for me. It was relevant since Lua is a scripting language purpose built to be embedded in other languages.
The programming language is famously used in World of Warcraft and Roblox where you have all the high performance rendering code written in something like C/C++. The parts of the game that change frequently though are written in Lua and called from that lower level language.
He also had some interesting recommendations on programming languages (aside from Lua) that he’d recommend people learn to expand their minds.
Check out the episode wherever you get your podcasts: YouTube, Spotify, Apple Podcasts.
Timestamps
08:35 - Comparing Lua with Python
13:04 - Top book recommendation on language design
14:20 - How JIT works and why it is hard
23:21 - Compiling Python and interpreting C
30:31 - How cross language calls work
36:58 - Lua unique design decisions
51:17 - Predictions for AIs impact on languages
01:00:10 - Top 3 languages to learn to become a better engineer
01:03:21 - Advice for his younger self
Transcript
00:43 — What sets Lua apart
Ryan:
[00:43] In 2023, Lua was the second highest programming language by growth according to contributors on GitHub. And I saw that it was also used in famous games like World of Warcraft and Roblox. And so I wanted to ask you what kind of programming language is Lua and what sets it apart?
Roberto:
[01:07] Lua is a language that usually is used together with another language like C or C++, and then you have an architecture where Lua takes care of the more dynamic part of your program, things that change more frequently, things that are not so resource intensive, the hard parts you have written in C or C++. So it’s typically what is called a scripting language. There is some confusion, I think, between scripting languages and dynamic languages.
[01:45] And people just, I think, considered a dynamic language to be more or less the same thing as a scripting language. And they are not exactly the same thing. So, for instance, a big difference between Lua and several other scripting languages is that for scripting you have two typical uses. You can have the app that I call who has the main loop. So you can have the program written in C and calling Lua, or you have your program written in Lua calling C.
[02:23] And for several scripting languages, actually, you don’t have to, but it’s much easier to use if you have your program written in the scripting language, and the C language is only for your libraries. Lua is a very good fit. When you want the reverse, you want the program written in C and then it calls Lua from time to time. So the point is that Lua is very good at this, that you have the main loop, the program written in C, and then it calls Lua for some tasks or for whatever you want to do.
[03:09] And for instance, in games, this is very important to keep the frame ratio of the game. So, for instance, you have the loop. The loop keeps the rhythm of the game and keeps everything. And then at every frame it calls Lua. Lua updates all characters, all images, everything in the game, and then it returns to C to do the renderization, etc. So it is a very good fit. But the main point is that people sometimes call this the embedding versus extending.
[03:46] So you can extend the scripting language with C, or you can embed the scripting language into C. And so several languages are very good only for extending, while Lua, it’s really good for both embedding and extending.
Ryan:
[04:05] Why is Lua good for embedding and Python? It’s almost the opposite, because Lua, since the beginning, we thought about Lua as a library.
Roberto:
[04:11] Since the beginning, Lua was written as a library. So I think that’s the main difference. Lua has a standalone program that, I mean, you can call Lua in your console, in your command line, but that program is just a client of the official client of the library, following the same API that any other program can use. So this idea of language as a library, I think it’s not very common. It’s something that is very particular in Lua.
Ryan:
[04:49] When you design a language as a library, what are the unique design decisions?
Roberto:
[04:56] I think that goes a lot of small and big details. For instance, your idea of what is your global space, or about scopes of variables, about exception handling. For instance, it’s very important that it’s very common in Lua that you raise an exception in Lua but you catch the exception in C. So all these things about how you do exception handling in the language. Another, it’s not a big deal, but it’s.
[05:31] For instance, most dynamic languages, that’s almost a definition of a dynamic language. They have an evaluation function that you can give a piece of code and then it executes that code. In Lua, we don’t, instead of eval, we have a function that we call load, that you give a piece of code and it returns a function, an internal function. Like it was a Lua function that when you call that function, then you execute the associated code. It doesn’t execute immediately.
[06:08] So you have this clear separation between compiling the Lua code that you have, and you can do that in C. So in C, you can get a piece of Lua code. You can compile it, check if it has errors, et cetera. Then you call that, for instance, a different piece, or you can even call several times. You can load functions from Lua into C. You keep them in the Lua space, and then you can call that same function again.
[06:42] And again, it’s not a big difference, of course, because if you have eval, you can evaluate a function declaration and then return that function. So you can, if you have eval, you can do load. If you have load, you can do eval. But I think load, it’s simpler to use for that kind of thing. That is, when you think about libraries, for instance, in Lua, it doesn’t have a global state. I think that’s, I forgot that.
[07:14] That’s a very important difference. When C starts, the first thing it has to do is create a new Lua state. And then everything you do, you do on that state. And that state is completely independent of everything else. So C can, for instance, create another Lua state, and both states are completely independent, completely. There is no communication between them. And so this again shows that these.
[07:51] And so, for instance, C can use a state, do a lot of stuff, and then it can close the state and all memory used by Lua is released. Everything that Lua was using is released. For instance, if C doesn’t need to use Lua anymore for a program, you release all resources used by Lua, and your program continues, and then later you can again create another state, etc. So there’s, as I said, several small or not so small decisions in the language.
[08:24] But all the time we think about the language as, is that good for embedding? Is that possible to do embedding and things like that.
08:35 — Comparing Lua with Python
Ryan:
[08:35] When you think of the programming community, generally everyone is quite familiar with Python, but not as familiar with Lua. I thought it might be interesting to compare the two languages. So if you compared Lua to Python, what are the pros and cons of each language design?
Roberto:
[08:54] Lua is a language that is intended to be used in this idea of scripting architecture. So Lua, for instance, it doesn’t thrive having a lot of different libraries. If you have something that, oh, I want to write a quick program for, something that Python is much better, it has all the libraries you can dream about. It has a lot of libraries built in already into the language. So the language is huge.
[09:27] I mean, the installation, it’s a huge download, etc. But it has exactly, oh, I need that. Oh, it’s there, I need that, it’s there. And Lua is almost the opposite, some very minimalistic language, because it is that you embed Lua into your program. It doesn’t use almost any resources. Most of the libraries, the important libraries that you will use, will be provided by the program itself, will be the commands like move a character or do some speech or things like that from the engine of the game, for instance, if you think about games.
[10:07] But whatever it is. So Lua, I think that’s the main big difference. They have very different goals.
Ryan:
[10:16] I saw some benchmarks, and I saw that Lua is much faster than Python. Why is that?
Roberto:
[10:23] Those benchmarks are not exact. But I think the main point is that Lua does have this focus on performance. Again, in the realm of scripting languages, it doesn’t want to compete with C or C++. But in the realm of dynamic languages, mostly now it’s a dynamic language, not a scripting language. We have some focus on performance. So I think this is the first difference. Python is exactly much more.
[10:56] I think it’s the decision that they make. Performance is not that important. It’s more important to be flexible, to be easy to do whatever you want to do. Then in Lua, sometimes we do not put some features because we think that there is no way to implement that efficiently. But also I think some part of the performance difference comes exactly because of the size of Lua. So most of the virtual machine fits in your cache, for instance.
[11:27] I think there are these two big things. One is that Lua doesn’t have so many features. It’s not so dynamic as Python. So in Python, there is a lot of interaction because everything can mean something else. In Lua, we are a little more conservative on that side. But I think also this thing of being small also makes it naturally faster.
Ryan:
[11:53] So, on the distinction between scripting languages and dynamic, or I guess it seems like dynamic language is a superset, scripting language is a subset of that. Am I understanding that JavaScript is a dynamic language, not scripting language from your perspective?
Roberto:
[12:10] Yes, exactly. Yes, exactly. Because the scripting language—the original scripting language was Bash, or the Unix shells—with this idea that it is a language that coordinates other stuff. So, for instance, it can be extending again, you can use. But this idea that you have two different languages for the shell is only useful because you have a lot of programs written in C that are controlled by shell. So the scripting language has this very strong idea that you have this idea of a dual-language architecture.
[12:50] So scripting is like the name scripting. It means you coordinate or you give a script to be executed by those other things.
13:04 — Top book recommendation on language design
Ryan:
[13:04] You gave a talk a while ago, and someone asked you a question about what books do you recommend for studying programming languages, and you said that you actually enjoyed studying the language design of other programming languages.
Roberto:
[13:18] I like reading books that describe the design of languages. I think the best books are those written by the author of a language about the design of that language.
Ryan:
[13:32] What book recommendation do you think is best on language design?
Roberto:
[13:37] JavaScript: The Good Parts, for instance. It’s kind of old now, but I think it’s a very interesting book. Although I always joke that JavaScript: The Good Parts is a very thin book compared with the official JavaScript books, like 110 of the language, but I think that book is really interesting because it discusses the language, the bad parts, and it focuses on the good parts, but then explains why it’s there, why it was made that way, etc.
[14:17] That is a book that I like.
14:20 — How JIT works and why it is hard
Ryan:
[14:20] When we were talking about Lua, it sounds like one of the things that sets Lua apart is the performance and how minimal it is. And when I was reading about Lua, I saw that there’s LuaJIT. How does LuaJIT work?
Roberto:
[14:34] LuaJIT? Well, the first thing, it’s a completely different project. It doesn’t have anything to do with us, but it’s an incredible piece of software that is a just-in-time compiler for Lua. And I think exactly one of the reasons it works so well on top of Lua is because of the simplicity of Lua. It’s a very regular language. I mean, it has very, as I said, it doesn’t have many exceptions or too many interactions or things like that.
[15:09] So it’s not that dynamic. I mean, everything can mean something completely different, so I think that gives a very good language for a JIT. But Mike Pall is the name of the guy that made the first LuaJIT. He’s still working on that. It’s unbelievable, his work.
Ryan:
[15:33] What makes writing a JIT difficult?
Roberto:
[15:36] The first thing for me, I don’t want to get involved with JIT, is because it’s machine dependent. It’s not very productive. I mean, you do a lot of work and only work on that architecture, and then, oh, I want to run on another architecture. But Paul, he created a kind of pseudo-assembler that he writes in this assembly, and then he translates that assembler to real machine code for the different machines.
[16:08] So trying to unify the different architectures. So there is a lot of work, but I don’t like—I think architecture I like to study, but I don’t like to work with it because I think it’s very unstable. Each new version, they change that or they change that. I mean, they change the ABI for something, and then now the stack has to be aligned in some very particular way. And of course, also to get that performance that he gets, he made what’s called a trace compiler.
[16:47] The idea of a trace compiler, instead of getting a function and compiling a function, it works like any JIT that tries to detect things that are executed frequently. And so it starts what’s called the trace. It gets recording everything that the code is doing, including function calls, et cetera, until it closes the loop, and then it compiles that loop, including function calls, et cetera, everything in line.
[17:23] It compiles that for that specific, for instance, so that number happens to be an integer. So it compiles it, the number will be an integer again. And there are a lot of checks just to check that everything is as assumed, and then it executes the loop, and then it’s very, very, very fast. But anything that is different, for instance, you call it again, but you call it frequently with integers. Suddenly you call it with a float, and then at some part of the code, it breaks the condition and you have to return to the interpreter part.
[18:05] And I think that’s one of the worst parts, is that you have to translate the state that it’s all compressed into register, et cetera. For instance, you don’t even have a call stack because you didn’t do the calls, you just inlined it. But then, oops, something went wrong. Now you have to continue interpreting. So you have to recreate this fact that you didn’t create originally, et cetera. So I mean, even to think about that, I have headaches.
Ryan:
[18:40] Standard Lua is portable, but to execute on a machine, it eventually gets converted into machine instructions. Where in the stack is it eventually translated to machine instructions?
Roberto:
[18:52] Lua is written in C, the interpreter is written in C. And then you compile that into machine code, the interpreter, the Lua code. Then when you get some, I mean the Lua code, the Lua interpreter code, when you get some Lua code, you do what we call a precompilation; that is, we translate to an internal language. Then the interpreter is just a big loop, a loop with a switch. That’s a very big simplification.
[19:27] But in general terms, a loop with a switch that gets instruction does a switch and sees all these instructions. The instructions are quite similar to a CPU move, but then instead of creating it, it executes that structure: move A to B. So it does A, move A to B, and executes that, and then again repeats, executes the next instruction. So this is how interpreter works. And basically, so we pre-compile Lua.
[20:03] There is a compilation, but we compile for this virtual machine, and then we have this main interpreter loop that people call that just fetches each instruction. A jump is just a jump. You have an array of bytecodes. A jump just goes to that. You have a counter that tells where you are in this array. You just update that counter. It goes to another position in that array where you’re going to.
[20:34] So it’s like you emulate the CPU in software. It’s written in C. The interpreter, this main loop, is written in C. So if you compile it in Linux, it will run in Linux on an x86 architecture. It will run on that architecture. If you compile that on a Mac, it will run on a Mac. It’s a C program.
Ryan:
[21:00] Got it? Okay, so. And that’s the part that’s not portable. And the C compiler handles that?
Roberto:
[21:05] Yes, yes, yes. All portability of Lua comes on top of portability of C. How does JIT compare to ahead-of-time compilation in terms of performance?
[21:17] Both ahead of time? Any kind of compilation can be 10 times faster than interpreting, or even 100 times faster than interpreting, depending on what you are doing. But 10 times is a very good figure between ahead of time and trace compilation. Then it depends a lot on what you are doing after. Trace compilation is particularly good for benchmarks because you’re repeating it more or less.
[21:55] They are very uniform. You are doing exactly the same thing again and again and again. So then trace compilers shine. I mean, this is the best they can do in real programs. That depends a lot, but I wouldn’t say there is a clear winner between the two. I think it depends a lot on the kind of program.
Ryan:
[22:20] Is it possible to write an ahead-of-time compiler for Lua?
Roberto:
[22:24] Yes, there are several. I mean, I don’t know if there is any production-quality one, but for research, et cetera, there are several. I have a student who wrote one, like a master’s thesis. A ridiculous simple Lua compiler, ahead-of-time Lua compiler or something like that. Exactly, because it was just—it gets these opcodes and expands into C code, and then you send that through to a C compiler, and then you have an ahead-of-time compiler for Lua, and it gave like three, five times boosting performance.
[23:13] Very, very. As the title said, it’s ridiculous. Simple compiler.
23:21 — Compiling Python and interpreting C
Ryan:
[23:21] I always hear there’s compiled languages and there’s dynamic or interpreted languages, I guess, but really that distinction is in the toolchain, not necessarily in the way that the symbols are laid out in the source code. I could write a compiler or a program that takes in Python code and then converts it into machine code. And then I would have compiled Python.
Roberto:
[23:46] And you can interpret C code too. You can write an interpreter for C. Yes, but the main difference is exactly what it’s easy to do. For instance, as I said, one of the hallmarks of interpreted languages is eval. So if you want to compile the language and keep eval, you have to have a compiler as a library of your runtime because you may want to compile things during execution. This is the hallmark of a dynamic language.
[24:21] So you can create code while running code. So that part is why, much more often than not, dynamic languages are interpreted, but you can compile. But then sometimes some compilers do not handle eval at all. They say, oh, you can compile as long as your program doesn’t have evals. So there are some restrictions. And as I said, you can interpret C code, I mean, but you’ll be extremely slow.
[25:00] It doesn’t.
Ryan:
[25:01] But I think one big difference, and when I look at static languages or compiled languages, they seem to have type systems in them, or type annotations. What is the role of a type system in the compilation process?
Roberto:
[25:18] Depends a lot on the type system. Several type systems, like in C, for instance, are written to be a very important part of the compilation process. So if you have the right type system, it’s much, much easier to write a compiler. A typical example is on the space for variables. Because if you know, oh, this is an integer, this is a float, this is a double, you know exactly how many bytes of memory you need to that. Otherwise, it’s a mess.
[25:54] More often than not, you have to keep everything in the heap, dynamically allocated, and so huge penalty in performance. And for instance, when you see, as I said, a trivial, you have an addition A plus B. If you know the type of A and the type of B at compile time, you know, oh, this plus is, I’m adding two integers, I just generate the machine code to add integers, and there everything is fine.
[26:27] You know, a dynamic language, like C, I don’t—I compile to a virtual machine, and the virtual machine is just a piece of C. But then how do I... At runtime, you have to check what is A. Oh, A is an integer, B is a float. Oh, I have to convert. Or B is a string. Or depending on the language you are, if you’re not doing addition, it can be doing concatenation or you can just call some function, and you have to do all that at runtime.
[26:57] So types can be very, very important to compile efficiently. But of course, you have to have a type system. There are several languages now, like TypeScript, that the types are kind of... You do not guarantee that everything has the types you set. So then it’s impossible to use them to compile because, oh, probably A will be an integer, but if it’s not, I mean, if I have an integer register to boot with, it must be a register or things will not work.
[27:40] But if you have the right type system, they are essential for good compilation.
Ryan:
[27:46] I know some programming languages have type inference. What if you ran type inference and you got an unambiguous set of types, and then you used that in compilation? Could you do that for Lua?
Roberto:
[28:01] No, I mean that’s not computable. But a lot of people try to do type inference for dynamic languages, and it’s a really hard problem, and it’s very difficult to do anything useful in terms of performance. What if sometimes you get? But you have to be very restricted. If you write your program in that specific way, use fall. So it’s like you don’t have the type, but you write the program thinking about types.
[28:38] And then if the program is in that very particular format, then you can do type inference and everything goes well. Otherwise, either type inference doesn’t work or, I mean, it works, but it infers very generic types for everything. And so you cannot take advantage of the types because of the dynamic nature of the language. This also happens with LuaJIT, for instance. You can get much, much better performance if you have a kind of type system in your mind.
[29:18] Usually we do that, and you follow those type rules even without—the compiler, the language does not impose them on you. But you assume, oh, I’m going to follow those discipline, I’m not going to use the same variable to store integers and strings, etc. And then you can have much better results.
30:31 — How cross language calls work
Ryan:
[30:31] Earlier, you mentioned that Lua can call C and C could call Lua. And you mentioned, we talked about the compiler and the interpreter and how do these pieces work together in the case where you’re chaining different programming languages.
Roberto:
[30:47] Yeah, the main trick, I mean, it’s not a trick. The technique for allowing C is C pointers, function pointers. There is a pointer to a function. This is part of the official C language. So when you start Lua, suppose you are in C, as I said, you create a library, you create a state, a Lua state, and then you can register, you send to Lua C pointers, function pointers, I mean, pointers to functions in your C library associated with names.
[31:28] Lua stores that in some data structure. Okay, so it says, oh, the function is associated with that pointer, etc. So when it’s doing the interpretation, oh, this instruction is call. Instruction, call. Let’s see what it’s calling. Well, it’s calling a. What is the value of A? Oh, it’s a pointer. And then it calls that pointer. And then we call the C function to do that. And when C wants to call Lua, I mean, a Lua function is just a data structure from the point of view of C.
[32:10] It’s just an array of bytecodes of opcodes, and then it calls the Lua interpreter function. Please interpret that function for me. So C can call Lua, interpret that function, and that function is running, and then can call a C function using a pointer. And you can have that in the stack at several levels. So we are running a Lua function that calls C and C calls Lua again and Lua calls C. You have recursive stuff in that, etc.
[32:43] And everything works, just works.
Ryan:
[32:48] In one of your talks on Lua, you mentioned that there’s security benefits to using a scripting language. What are those benefits, and how does it...
Roberto:
[32:56] protect the hardware. It’s not only hardware. Once I gave a talk here in Brazil, I was called to give a talk at a Python conference in Brazil. They called me for a talk. At the end of the talk I said, “Oh, let’s use it as a scripting language.” As I said, emphasizing that thing that you have this dual-language architecture. And I know of case of Lua being used with C, with C++, with Fortran, with several different languages.
[33:33] And then someone in the audience said, oh, we also use Lua for scripting Python programs. And what was the case? They have a huge Python program for financial stuff that has to do a lot of financial transactions, etc. And they wanted to do scripts, I mean, to have a command line where you could write instructions to be executed at runtime, for instance, for debugging or for inspecting, for whatever you need some kind of end-user programming.
[34:11] But they said exactly what I was talking about, spaces in Python. Basically, if you have a command line and you are going to run Python, that Python can do whatever it wants to your program. There is no way to protect it. So if you gave that to the user, the user could do whatever it wanted to do. I mean for the program. And so the program would break a lot of invariants of a lot of important stuff in the files it kept, etc.
[34:45] And so what they did, they gave the thing in Lua, because, as I said, Lua, you create a state. For instance, as I just mentioned, you can only call C functions. That is true in Python again, but in that particular data you can only call C functions if you give the C pointer to Lua. So have a very strict, and there is very little, as I said, there is a library. When you create a Lua state, it has no functions at all.
[35:18] There is nothing. I mean, you cannot open files. Everything there are, we call standard libraries. Usually you open a state and you register those standard libraries. In Lua, so you have the minimum, you have mathematical functions, etc. But you can open a state and register no functions at all. In Lua, it can only do things calling those specific functions you gave. So they used Lua to have this kind of control.
[35:49] Or you can write Lua code here, but that Lua code can only do very specific things in the Python program. You cannot call any Python function or build any kind of Python data structure, etc. So was this a nice example? In the case of hardware, it’s the same thing. You cannot just go in. For instance, there is a hard port that controls the speed of the fan. That keeps the temperature of the CPU.
[36:23] And if you put that very low, you can really burn the CPU. I mean, because the CPU gets very hot. And so you have a C. You cannot call C directly, you must call Lua. And only Lua has access to that. So if you can only program in Lua, then you check whether the numbers you are giving are precise or whatever. It has to check and then it calls.
Ryan:
[36:51] It’s kind of like a sandbox environment.
Roberto:
[36:53] Yes, exactly. Yes. Sandbox is the exact word for that.
36:58 — Lua unique design decisions
Ryan:
[36:58] I saw there was this, I guess, paper, maybe it was an article you wrote on the history of Lua with several other people. And I thought there were some interesting quotes in there I kind of wanted to ask you about. So one of them, it says that there’s this old joke that says that a camel is a horse designed by a committee.
Roberto:
[37:20] Yes, that’s not mine. That’s a real old joke.
Ryan:
[37:25] Does that mean you think the best programming languages are designed by as few people as possible?
Roberto:
[37:33] Yes, I do think, yeah. Because one thing that it’s easy to observe is this: If you have a committee writing a language, everyone wants to put something from themselves into the language. So you have your favorite mechanism, and you will fight whatever it takes to put your favorite mechanism into the language. And very few people you fight against putting anything in the language.
[38:11] I mean, some people say, oh no, this is too much, this is too complicated. But people fight much more fiercely for putting something they want in the language than against putting something in the language. So when you have a committee, the tendency is, “Let’s keep adding stuff, let’s keep adding stuff.” And people sometimes even do not understand, I mean, I’m even not sure if everybody there really knows the language well enough to be putting that.
[38:45] But that fits with other parts of the language. I didn’t even know the language has this other part. I mean, so things sometimes do not fit together very well, et cetera. So first, you have what’s called the conceptual integrity. It’s a name that gives that everything fits together, everything is designed with everything else in mind, etc. It’s much easier to. I’m not saying, I mean, Lua has several wrong stuff regarding because of.
[39:17] With time, etc. There you make mistakes, but it’s much easier to get things right if you have a small group of people that everybody knows what everybody else is thinking. I mean, you are deciding whether to put something in the language or not. It’s much easier to change your mind if you do not put something and then later you decide to add it to the language than to add something and then later you decide, oh, I shouldn’t have. That was not a good idea.
[39:53] So, when in doubt, the rule is always don’t put it. If you’re not very sure, don’t put it. We can always add that later and keep the door open to change your mind.
Ryan:
[40:08] I saw in the original Lua programming language that there was no Boolean type, and I’ve never seen that before. Why was there no Boolean in the original Lua?
Roberto:
[40:19] Well, C doesn’t have booleans. I mean the original C, and C90 up to C99, they use integers. If it’s 0, it’s false; if it’s different from 0, it’s true. And people lived quite well with that. Lisp and there are several languages that don’t have booleans. And actually, we only put booleans in Lua because we wanted false. True is completely useless in Lua. Nobody used true, and nobody’s a joke, but it’s too strong.
[40:57] But it’s not very useful. You can use almost any other value for true. But the problem is that in Lua, the special value nil is in a table. It is equivalent to the key not being present. You may think whether that was a good decision, but it’s very embedded in the design of the language. And it has this strong concept that a table, a key that is not present in a table, there is an associative array, it has a nil value, it’s completely indistinguishable whether it’s absent.
[41:43] I mean, absent means it has a nil value. Nil value means it is absent. And so sometimes you want to have a false value, but you want to know it is there in the table. And so we needed a false to have a false you can put in the table. And it’s completely different from being absent. You know, that is a key and the key has a false value. So we need a false value. And if you’re going to have a false, then we added true, true.
[42:16] It would make sense to have false without true. But with true, it’s almost useless.
Ryan:
[42:22] Why not use 0 as false if you’re okay with 1 as true?
Roberto:
[42:27] We could have used it, but among other things, that would be a big change because it didn’t exactly work that way. So we added that later in the language. So changing 0 to false would be a very big incompatibility. But it could be, for instance, in Python I think a lot of stuff are false. I mean, zero is false, empty list is false. And in JavaScript I think it’s even worse. I don’t recall exactly, but I mean this is kind of arbitrary.
[43:06] I mean, we could have, for instance, nil, false, zero, empty string. It just depends on the kind of test that you use. In dynamic languages, it’s very easy to.
Ryan:
[43:22] Python has the... Like you’re describing the truthy values and the falsy values, I guess they can be interpreted as false. But it’s more implicit, less explicit. And I know JavaScript is even further on that spectrum where you can do some really weird stuff that implicitly has some behavior. Do you think that design direction is good or bad?
Roberto:
[43:48] It’s like a small compact. It’s more easy to write stuff. But you always have this balance in any language. Almost anything, the more flexible you have, the less protection you have. In C, you have something like that people do not notice. But because C is typed, you can check whether an integer is true or false. You can check whether a float is true or false directly because, again, it checks whether the float is zero or different from zero.
[44:19] You can check whether a pointer is nil or different from null. Because in C, null is a magic zero. So it’s kind of, if you do not write any test, it has an implicit test like different from zero. And this zero can be an integer, can be a float, can be a pointer. So in C also you have this kind of flexibility. It’s just you have to be more explicit. Usually these are, avoids errors, but so this, you always have this balance between easy to write and easy to make mistakes.
Ryan:
[45:09] I saw that Lua is 1-indexed instead of 0-indexed, and I had never seen a programming language like that.
Roberto:
[45:18] In my experience, there were a lot of programming languages like that.
Ryan:
[45:23] So yeah, why is it 1-indexed?
Roberto:
[45:26] Because everything in the real world is 1-indexed. If you have a book, you have chapter 1, chapter 2. Nobody numbers chapters as 0, 1, 2, 3. Only programmers are the only, even mathematicians. If you get a math book, a book of mathematics, any sequence, it’s 1 and 2, 3. If you get a matrix, the first element is 1, 1 and 2, and then... Everything is written before. Only in programming language there is this idea of zero.
[46:03] And the funniest part is that, for instance, Fortran, now it’s a very old language, but to index it from 1. In Pascal, you can choose. I mean, you write an array, you can say this array is indexed from -5 to 5, and so it’s indexed from whatever value you want to whatever value you want. 0 became extremely popular because of C. C uses 0, and a lot of languages copied are kind of inspired by C. They have the same operators, the same syntax for expressions, the boolean operators, for instance.
[46:50] That is not standard mathematics. Almost all languages chose the same operators as they are written in C. And so a lot of languages copied C. And then zero index became very popular. And what is funny is that in C there is no indexing. In C, indexing is just an illusion because what you have is pointer arithmetic. And see, when you write A indexed by I, actually what you are saying is get the contents of A plus I.
[47:29] And so because in C, you don’t have indexing, you have these displacements or deltas, I don’t know, offsets. Then it must have indexed by zero because of that, because then the first element is the element that is at the original address. So C doesn’t have indexing. So when you say A is indexed by 0, it doesn’t index by 0 because it doesn’t index by anything. But it has this illusion of indexing.
[48:02] And then it’s easy to sync that in. And then a lot of languages that do not use pointer arithmetic do not have this restriction, do not have these semantics, copied C and kept the zero indexing as this thing. If you get a 12-year-old and try to explain zero indexing, I assure you it’s much, much easier for them to write. Oh, I have a list of the first element. I always joke that the first element is 0, and you write first, you for 1, but the first element 1 is not 1, is 0.
[48:45] So it has advantages, for instance, for some specific operators. Mathematically, for instance, when you want to do a circular buffer, zero is better. There are some small advantages, but it’s much, much more confusing. And as Lua has this idea of end-user programming, we always joke it’s much easier to make life easier for the non-programmers. And I’m sure that programmers can program whatever index they have to do because they’re supposedly professional. They can learn that, oh, indexing from then, to put on the end user the burden of using something completely different from them.
[49:34] Indexing by zero, what does it mean, the element index zero? They never saw the chapters in the book. Yes, the first chapter is at zero, the second chapter is at one. No, I’m not joking. When you try to explain that to non-programmers, it doesn’t need to be 12-year-olds. I mean, just get anyone that is not an Internet programmer. Think, “Oh, I have the absolute.” I mean, you are grown up, you are, I mean, okay, you are a student. If 0, I’m sure you can learn to program; if 1 or minus 1 or whatever it is, the base you have to use.
[50:06] I’m sure.
Ryan:
[50:16] But do you think you see a lot of bugs because maybe someone thinks it’s zero?
Roberto:
[50:22] Unfortunately, I see some bugs, but as I always say, those are the kind of bugs that just show that you didn’t do minimum testing, because that kind of bug is not a kind of... That kind of bug that, oh, if you try to... Anything you try to do in an array, if you start from zero or start from one, you have a bug. So the most simple test that you can imagine to test anything related to an array, to a list, et cetera, and if you mistake 0 for 1, you detect that.
[51:08] So if you have that kind of bug, it just shows that you didn’t test that code at all.
51:17 — Predictions for AIs impact on languages
Ryan:
[51:17] There was this talk that you gave on the cost of adding language features to Lua and how there’s a lot of hidden costs. And I think today, implementation costs of software are going down due to AI code generation or LLMs, and I was wondering if that changes your thinking on, I guess, the cost of adding features to a programming language.
Roberto:
[51:43] AI is something that we don’t really know what’s going to happen. If you go to an extreme, it’s completely feasible that we won’t have programming languages. And I mean, because if the AI is writing all your code and is checking all your code and doing everything, in a few years maybe we don’t need programming. AI can write machine code directly; it doesn’t need programming languages. It’s easier for them to compile or even generate the code directly.
[52:21] I mean, I don’t know what’s going to happen in a few years. So I think it’s very hard for me to talk about anything about AI because I have, and I think nobody has a clear idea what—I mean, people may have a clear idea what happens in one year or two years, but in five years, I mean, anyone that says, “Oh, that’s going to happen in five years,” is just guessing. So I think it’s hard to say anything.
[52:59] This is all because of AI. So keeping the minimum thing, I mean that still have differences there. Oh, why do you still use programming languages? Because you want the user or some human to be able to check the result of what the AI is doing, etc. So I still think that most of the things about programming languages still hold. For instance, the cost of complexity of you understanding. I mean AI create a code for you and then you really understanding what that code does.
[53:37] I mean it’s even worse. I mean it’s much more important that the language should be clear and has no hidden mechanisms. Because the AI may use a hidden mechanism, it doesn’t have the concept, oh, that will be difficult for a human to understand that what is really happening here is something it says something that, for instance, oh, I can use that. But I’m for sure it’s going to put a comment here because I’m sure that someone that reads that in one month is not going to understand.
[54:14] EA doesn’t have that kind of thinking, and so just use that trick or that thing. So I think if you think about this idea of AI generating code, I think it’s even more important for the language to be simple, to be understandable, for you to be able to really understand that the code that you are seeing does what you think it should do. You really understand what the code is doing.
[54:44] So this thing about simplicity, I think it’s very important, and I think the other parts from the book maybe facilitate documentation. I’m not sure if I mean this thing about conceptual integrity, for instance, to keep things, that makes sense, etc. I really think that one of the main costs is that it’s the burden you put on the user to learn one more thing about your language. So there’s the other cost of it.
[55:19] I said that implementation is the part that AI can really help you. It’s not a really important cost.
Ryan:
[55:32] If you think about a spectrum of simple to complex, what programming languages are the most simple ones that you think of, the easy to understand, less confusing side effects, and what programming languages are the most complex and have the most foot guns.
Roberto:
[55:50] This famous quote, also—I don’t know from whom—that the simplest possible, but not simpler than that. Because if you go to the extreme of simplicity, you could get, for instance, lambda calculus or Turing machines and say, “Oh, that’s really simple.” I mean, this is really simple, but it’s completely impossible to write any program in that. I mean, if you have really simple stuff, I think the extremes will be like that.
[56:22] But you don’t want to be there, but for the other side. I think C is a good example of a language that I think is really, really complex.
Ryan:
[56:33] A question that comes up a lot is, given how much AI has been progressing, would you still recommend people learn computer science today?
Roberto:
[56:43] If you really think about that, it’s really difficult to recommend people learn anything for a profession. We have no idea what AI will do in five years. If someone is entering university now, they’re going to graduate in four years, five years, or, at minimum, three, three and a half years, and nobody has any idea what the world, your profession, will be like in four years from now. So it’s really hard to, I mean, to say, “Oh, yes, you can still go into it,” because now people say, “Oh, no, the manual tasks, the AI, it’s very good, but you still need the whole architecture.”
[57:31] You needed a software engineer to do, to see the big picture, etc. That is now. But you have no idea that that will be true in four years or five years. So I think I like, I enjoy programming. I could do. I mean, some people say I use the AI for that. I mean, I program because I like that. But, basically, so if you choose something that you like, but really if you’re really thinking, how am I going to live with that?
[58:03] I have no idea. It’s really, I think it’s a really hard time to be choosing the profession.
Ryan:
[58:13] I mean, you’ve worked on Lua for such a long time. When you look back on it, what went well and what didn’t go well?
Roberto:
[58:22] We had this privilege of not having to satisfy clients. So you do not have, as I said, for instance, you can choose to add some feature to the language because you do not have a pressure. Oh, we need that feature, whatever it takes or et cetera. So we have this privilege of, oh, we are not sure whether to put that. We don’t put that. We can wait one year or so. I think that it’s much easier to do a good project, a beautiful project.
[58:55] So I’m not sure this is a recommendation to try to work without much pressure. People usually do not have this choice.
Ryan:
[59:07] What was your measure of success? If you’re working on a programming language, how did you know it was good?
Roberto:
[59:14] It’s more important to have, sometimes, good people that I recognize as would like the result than a lot of people that I have no idea who they are, etc.
So I think this metric of quantity that is the standard metric nowadays on the web has gotten much worse, that everything is just in the number of follows. I mean, it doesn’t care who is following you, just as many people as possible.
[59:48] So sometimes, for me, it’s much more important if someone that I really admire or think is, or says something good about Lua, then, oh, we have that many. But, of course, it’s good also to have some number of users and to be recognized. But I think it’s a balance between all of this.
01:00:10 — Top 3 languages to learn to become a better engineer
Ryan:
[01:00:10] Do you recommend people study other programming languages to get better at programming? And so the question is, what are the top three programming languages that you think every engineer should learn in 2026 to become better at programming? Haskell, I think, is an incredible language.
Roberto:
[01:00:28] It has everything you need to really learn about functional programming. I think one of the main benefits is that it’s much. There’s an old joke in the Haskell community that if you write a program in Haskell and in C, in C, you really spend one week to make it efficient, and then you spend one year to make it correct. In Haskell, you spend one week to make it correct, and then you may spend one year to make it efficient.
[01:01:15] Again, it’s not my joke, but it has some truths. Not always true, etc. But I think it gives the idea. I think this is maybe the most important lesson of Haskell. But Haskell has many other validities. This thing about type inference, for instance, that in the entire language, types are optional everywhere, and yet it can do type inference for everything. It can infer the types correctly, etc.
[01:01:51] See, or some old language. I mean, you can also have to learn some assembler, or I mean, to see. But assemblers now are becoming too much complex. It would be to learn, like, the Intel 8080 assembler, very old assembler of a very... But to have this exactly, this idea of what a machine does, how it does stuff at exactly the basic level, to have this understanding. Scheme is a language that I like very much because of this, exactly this economy of ideas.
[01:02:27] There are very few concepts, and it can do some amazing things with very few concepts. The way one language that this is very, very old. But I still think it was SNOBOL. I think one of the first languages to have pattern matching and this idea of doing, I mean, really strong patterns, et cetera. And I think it’s in... Sometimes it’s interesting to see old languages because, exactly, nowadays a lot of languages tend to, like I said, zero indexing that people, they copy a lot.
[01:03:06] They tend to be too uniform in some aspects. And it’s interesting to see some older languages that have some ideas that may maybe not even be good ideas, but it’s really interesting.
01:03:21 — Advice for his younger self
Ryan:
[01:03:21] Last question for you is, knowing everything you know now, if you could go back to when you had just started your career and give yourself some advice, what would you say?
Roberto:
[01:03:34] To know a lot of stuff, you really have to study a lot of stuff. And it takes time. I also joke about that. People say, oh, learn Lua in 30 minutes or learn Python in 5 minutes. And I always say that I wanted to learn programming in five years. But that’s really learn. Take your time. You have to learn that. And that is a lot of stuff. Stuff to learn. There is no magic there.
[01:04:07] You really have to learn. I mean, a lot of different things.
Ryan:
[01:04:12] Thank you so much for your time, Professor. I really appreciate it. It was a lot of fun.
Roberto:
[01:04:16] Thank you. It’s my pleasure.
