EPISODE 1760

[INTRODUCTION]

[0:00:00] ANNOUNCER: Python 3.13 was just released and brings fundamental changes to the language, including a new interactive interpreter, experimental support for running in a free-threaded mode, and a just-in-time compiler or JIT. There are also updates to the Python type system, module removals, and docstring improvements among many other changes. ?ukasz Langa is a CPython Developer in Residence at the Python Software Foundation. He joins the show with Sean Falconer to talk about maintaining Python, the evolution of the language, Python optimization, the most impactful changes in Python 3.13, and more. 

This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him. 

[INTERVIEW]

[0:00:57] SF: ?ukasz, welcome to the show. 

[0:00:59] ?L: Happy to be here. 

[0:01:00] SF: Yeah. Thanks for being here. You're the Cpython Developer in Residence for the Python Software Foundation. I want to start out by asking what does that role entail? 

[0:01:10] ?L: Right. Historically, Python has always been a language that is maintained by a community of volunteers. There have been companies that employed people who work on Python core. But, usually, that was not their main job. This recently shifted to a model that is more alike of having some corporations invest in their own teams but also other corporations sponsoring independent developers like myself to work on Python full-time. 

And in my particular case, the work is sponsored by Meta currently. Thanks to them. But it's work that is maybe less glamorous than you would expect. Because being paid for it means I'm essentially the janitor of Python. There's a lot of very kind of boring work that needs to be done. Or just a little bit of a frustrating work, like some bot doesn't work. There's a GitHub outage. Some database that we're using for, I don't know, contributor license agreements or so on is down. And someone needs to fix those things. I would be that person. 

And, magically, through the kind of wonder of being paid, those things that you would never want to touch as a volunteer just become tasks at work that you complete and you're happy with it. And everybody's happy that somebody else is working on them. That's essentially my job. 

[0:02:34] SF: Is the motivation for companies to essentially contribute to having a paid position where someone is doing this full-time is because they recognize that if you only have a community of volunteers, some of this kind of grungy work that needs to get done in order to support the language is probably not going to be the highest priority, something that someone's working. But, fundamentally, some of that stuff has to get done. 

[0:02:55] ?L: Right. A lot of those bigger corporations are very interested in Python these days. And they specifically have their own contributors that actually want to also be active in the development of Python. And when we don't have enough people to review their contributions or the automation and workflow is buggy, doesn't work for them, their changes are waiting. They're stuck just as everybody else's. 

Making sure that those basics of accepting external contributions work actually allows also the big corporations to be an active participant. And for them, the timing is more crucial than for other volunteers where sometimes developing a feature will take a year, or two, or five, and that's okay. Corporations usually think in quarters, right? Because that's how they report. That's how their shareholders want to know about how the company is doing and so on and so on. They're very rarely interested in five-year timelines. They usually want their changes to be discussed and implemented quickly. That does not mean that we prioritize any changes by people who sponsor the Python development. However, it just helps the entire pipeline. 

[0:04:12] SF: I mean, Python is a language. It's been around for quite a while. But I think that - and it's been a language I think that's maintained a lot of popularity throughout its entire life cycle. And if anything, it's growing in popularity. I'm curious around your thoughts of that, especially with everything that's happening in the AI space. But what do you think makes Python unique from other languages and have sort of sustaining power? 

[0:04:39] ?L: Well, the way Python syntax looks like and reads is very high level. If you just spend a little time with it, you're going to find that a lot of it reads sort of kind of like English, which is much simpler than many other languages that are very popular these days. For example, there's going to be a lot of people now interested in Rust because of performance, because of the claimed security and so on and so on. That is all perfectly valid and great reasons to know the language. But the learning curve there is pretty steep. 

In Python, it's essentially the opposite. It's very easy to start and it's very easy to actually try something out. Thanks to the very wide library of possible packages that we have on the Python package index, you can essentially do anything that you would. And a lot of those things used to be in web surfing, in web development. They are very popular today in education as well as data science. Now, machine learning, AI, and so on. 

[0:05:44] SF: In terms of like the history with Python, you've been working in the space for a long time, how has the language changed and evolved perhaps since its inception? 

[0:05:54] ?L: If you listen to what Guido van Rossum, the original creator of Python said about his initial plan for the language, he expected Python to be the glue language between C and what people actually want to do. You would write high-level orchestration code in Python. But it would call into lower-level instructions, functions, however you call that is implemented in a different programming language. 

And this is sort of true today, especially in the field of data science and so on. But what Guido van Rossum did not initially expect was how big programs written in Python are going to be. He famously said that he expected them to be all under 1,000 lines. Most of them under 500. And we know for a fact that there are multiple very known code bases of Python that are counted in millions of lines of code. During this 30-plus year maintenance cycle of Python, the ability to effectively write, read, and execute source bases, code bases that big has essentially been a major sort of concern for us. 

[0:07:06] SF: Mm-hmm. In terms of the C extensions, do you think that that is a big part of what led to the growth in popularity with data scientists? Because data scientists, maybe they're not day-to-day sort of software engineers. Working in a language like Python that's maybe easier to pick up and learn is a good place to start. But then because it's data science and you probably need to run NL models and do a lot of number crunching, you want something that's more performant and has the power of something like C. And then that bridge between Python and C works very naturally there. 

[0:07:40] ?L: Oh, absolutely. C extensions are, in data science, essentially the single feature that makes Python what it is and how big it is. And why efforts to replace Python with another data science language, like TensorFlow tried Swift and so on, ultimately failed. Because it is not just about our little interpreter and it's not about a single library that we're using. But if you look at NumPy and SciPy and you trace what actual dependencies and implementation of different algorithms and different parts of math they have, you will quickly discover that it's not just C. There's literally a bunch of code there written in Fortran and so on. 

There are essentially loads of existing kind of effort by scientists that is now accessible via Python to the wider community. Having to have this in a different language, for example, Rust, would require all of this to be reimplemented. And this is not only inconvenient but also creates a challenge of reproducible research. Right? If you're changing what you were depending on to actually calculate your scientific results in a paper, there's a non-zero chance that the results are going to be computed differently. And sometimes it's going to be very hard to find why that difference occurs. There's a tremendous legacy in science. And that legacy was now brought into Python through the magic of extensibility via C. 

[0:09:12] SF: Yeah. And given the transformation that Python has gone from the original vision of it being a few hundred lines of code essentially of these - essentially a script that's tying these stuff together to now code bases that are millions of lines of code driving and running large web applications and large applications, how has that impacted considerations around the language designs and the priorities that need to be placed perhaps on making Python a more performant language? 

[0:09:43] ?L: Right. With the growth of the language, which, to be clear, was not always data science, kind of where now the biggest growth is probably that, we had plenty of sort of killer applications for Python over the years. The first one was probably Zope, which was this web application server that now nobody even thinks about. But early on, it was very popular. It was very powerful. 

What people kind of need to realize is that there is no single user base of Python because of this. Because even if Python is now growing in data science, there's still going to be very many users that use it for web. There's going to be people writing FastAPI code, Django code, Flask code and so on. Other people who write it as an orchestration language for servers, and others that use it for desktop UI applications, and so on and so on. We cannot really always just make a single change that's is going to benefit the entire community. 

With that in mind, we usually think of improvements to the interpreter with that lens in mind. That even if something helps one group, it should not hinder another. And this is, for example, why type annotations which allow you, like TypeScript for JavaScript, better control over what your functions accept, what they return, and so on. 

Essentially, to have better understanding of what the shapes of the objects that are flowing around are, this is an optional feature for Python. This is something that we take very seriously. And I personally am a very big proponent of this because I've seen how much it helped big code bases in terms of understanding and onboarding new developers. But there is loads of users, including in education, where type annotations create additional challenges that are maybe not the best focus for people who just started out with programming kind of in general. You don't want to learn, you don't want to teach type annotations to people who are starting out, right? 

[0:11:51] SF: Mm-hmm. Given that you have sort of a breadth of personas of programming personas that are using Python in very different ways, does that hurt the evolution of the language in some ways where it's harder to really lean into like, "Hey we're going to serve data scientists like really, really well. And that's the thing we care about." You sort of dilute the focus in some ways. And does that hurt the evolutional language in some way? 

[0:12:17] ?L: Saying that it hurts is like a strong statement. But it does definitely impact us. However, Python, which some people might not realize, is actually a few years older than Java. It is a pretty old language right now. The way we survived quite a few - let's call them paradigm shifts in programming, and kind of changes in what constitutes a killer app, and where most of the focus is, we survived all those because the core of the language just makes sense. 

Focusing on a single feature might very well be worthwhile for today. But for the next slew of killer applications in 5 or 10 years, it might have essentially make us super outdated. Yes, it does hinder short-term progress. But I wouldn't necessarily say that it hurts us long term. So far so good. It worked for us pretty well.

[0:13:11] SF: Yeah. That makes sense. It's like if you kind of over-specialized, then maybe you can get sort of faster earlier traction. But you basically create a ceiling for yourself. Especially as the industry ebbs and flows, maybe people - if Python had to really leaned into, I don't know, desktop computing or something like that 30 years ago, then much of that world's gone away. A lot of the development now is happening on the web. And maybe it wouldn't have been set up for success there.

[0:13:36] ?L: Exactly right. 

[0:13:37] SF: Are there some tasks though that Python is not particularly well-suited for? 

[0:13:41] ?L: Oh, yeah. Absolutely. And why that is, is just partially by design and in large part because of the current status of how the implementation looks like and what currently people are doing. A few cases that are rather obvious is if you want to write sort of scaler code in Python that actually makes computations inside Python with just for loops, while loops, and function calls, you're going to quickly realize that this is much slower than in languages like C or Rust. 

The reason why is that all those functions, and loops, and so on, and objects that actually we interact with, they're extremely flexible. They're dynamic to the point where you can change a lot of things about them at runtime. Whereas a compiler language would essentially translate the code that you typed into very efficient but limited machine code that does exactly one thing at the same time. 

This particular performance hindrance that we have now has been an area of interest in Python for a long and long time. The flexibility that it gives us in debugging and in learning how Python works, because you can just literally ask an object like what methods, what fields are there on you, and so on and so on. All this value added is a big, big, big performance cost. And the way to address this is sort of well-researched by now. V8 in Chrome showed us that JavaScript, a language that is also dynamic and very much so can be sped up significantly with just-in-time compilation. 

And there is in fact a custom Python interpreter called PyPI which effectively employs this. Where you can have code that is going to be 8x, 10x, in some limited scenarios maybe even 50x faster than regular Python. However, PyPI is its own project. We're our own project where the C API as we said with C extensions is extremely important for us. There is our own effort to add a just-in-time compiler to Python. And kind of the initial scaffolding, the fundamentals of it are already there in Python 3.13 that we just released a few days ago. 

[0:15:59] SF: Yeah. And that is a perfect segue too what I wanted to talk about next, which is around Python 3.13. That update, there's a lot of new features. What do you think is the biggest impact to the Python developer community with the release of 3.13? 

[0:16:15] ?L: That's interesting because there's things that are impactful short-term and long-term. Long-term, we are now trying out a super large experiment in where Python should go in the future. The simpler experiment that we have now is that the JIT so far had the scaffolding of it the fundamental parts of the interpreter ready, which means that even though now Python has to have a little more computation to actually compile your code into machine code, it is no slower than it was before. 

That part, because we have an annual release cycle, has not yet implemented any optimizations on that machine code to make it actually faster. But it is already a little faster so that the additional cost of recording the traces and compiling them at runtime are not actually hindering any performance. You can have an experimental JIT in Python if you compile your interpreter in a special way. But it's not ready for end-user consumption yet. This is sort of a longer-term feature that we have really high hopes for. The JIT should be amazing but probably in 3.14 or maybe even versions beyond. 

And an even bigger experiment that is happening right now is about something that we call the global interpreter lock. Until very recently and still by default in Python 3.13, the Interpreter had a single lock that protected the interpreter from becoming corrupt when multiple threads of execution are trying to access the same data. We have very high-level collections, right? You have built-in dictionaries or HashMaps. You have lists, which are essentially are arrays, right? And so on and so on. 

You could imagine that accessing them and trying to mutate them from two different threads could essentially lead to one of the threads seeing that data structure to be in a partially incomplete state. Because, I don't know, the other thread is adding a new key to a dictionary. The observer is going to see that this process is happening but it's not complete yet. 

For this, all of this, we have a single global interpreter lock that simplifies the design a lot and allows even C extensions to rely on the fact that, if you hold this global lock, you can do whatever you want with Python objects because no other Python code is going to execute at the same time. However, the big price for this is that if your machine has multiple cores, which would mean you can actually run multiple threads very efficiently and truly in parallel, Python so far has not been able to do this, which very quickly is visible when you're serving web or actually doing computations in pure Python. 

We had workarounds for this. For example, waiting on socket operations would free the global lock to be used by another thread. But in the end, Python was always limited in how it can scale. Now we are in the middle of a process to actually figure out whether the community wants to get rid of this global interpreter lock. With a large investment from Meta, some Meta employees are writing Python enhancement proposal for the removal of the global interpreter lock and implementing that for python 3.13. We can now actually test how that works. 

This is very much dependent on the community and the libraries to be ready for a world where now you have free threading and everything can happen in your code. This is experimental. This, again, requires custom build for Python so far. Python 3.13 is not where you're going to want to use the GIL-free, the free-threaded version of Python just yet. 

But let me just tell you that the maintainers of frameworks and libraries are very interested in this. We were sort of worried. Like, "Will they be? Will they not be?" Because it was a gigantic effort to replace the global lock with very many fine-grain locks and not lose performance terribly. But it turns out they are. When both the scientists and the maintainers of web frameworks are already pretty invested in this new world. They're already making their extensions available for the variant of Python that is free-threaded and so on. This looks pretty promising. And maybe in 3.14 or the version after we are going to have Python that is free-threaded in the default build. Anybody will be able to test it on their own computers and see whether the additional support for multi-core computation helps them and so on and so on. These are long-term changes.

[0:21:09] SF: Yeah. If you get rid of the global lock, then how does the language help solve the problem around essentially a thread looking at a data structure when it's in an intermittent state? 

[0:21:21] ?L: Right. For the internals of Python, you replace a single lock with many smaller locks. Or, where possible, you are deferring computation to be essentially serialized on a helper thread. Or you use a slew of algorithms that are lock-free. All of those techniques are in fact employed right now. And we're still working on using them more. Since now, in Python 3.13, because of the annual release cycle, you cannot just redesign the internals of the interpreter in 12 months, we had to disable a bunch of optimizations that the regular version of Python has just for kind of correctness. Right? And we hope to reenable them in Python 3.14. But this will require smart usage of the sort of three kinds of techniques that I described. And that's for internal kind of, I don't know, internal code of Python itself. 

And for the users, this turns out that is also a little bit of an issue. Because there were plenty of cases in Python in the past where not having true free-threading caused kind of implicit serialization of what things happened in your user code. Now, when those things can be truly parallel, you might see surprising results in code that just happened to work fine before. It was always incorrect formally, but it just happened to just run on a runtime that never triggered the bugs. Now they will be triggered. Definitely a big transition on both ends. 

[0:22:57] SF: And in this future state, will both configurations be supported where essentially you have a global lock but you can make the choice of turning it off? Or will it be basically removed and these other mechanisms will be in place to control locking on the thread if needed? 

[0:23:12] ?L: Right. This is where the experiment part comes in. So far, the bet and the entire effort about this is that the community is going to accept this new design and the new complexities involved because the scalability is going to be a clear win. In which case, at some point, this interpreter would become the default but it would still allow you to reenable the global lock for different sort of - for a various number of reasons. 

The main reason would be that you would have a c extension that did not declare that it's safe to use with free-threading. What you have to understand is even though the global interpreter lock in Python was a facility that was used for Python to protect its own internal data structures, many C extensions use the fact that it's there to protect their own internal C state as well. They could know that there's only going to be one thread that writes and reads to some structure that they keep somewhere in memory. Code that has nothing to do with python. Now those things are sort of unsafe to do actually in parallel if there's no global lock. C extensions have to declare safety. They have to declare support for free-threaded Python. 

Until that happens, we will have the global interpreter lock kind of standby and ready even in the free-threaded version so that it can be re-enabled in such cases where you import a module like this. Or you want to import it yourself to see the difference in behavior. Maybe report a bug and so on and so on. But long-term, if this experiment works out, if the community accepts this new way of programming, we will remove the global interpreter lock entirely from Python. But it is very long-term. Currently, the Python Enhancement Proposal, the PEP document that describes this change is called making the global interpreter lock optional, right? It doesn't say removing it entirely. Because it's going to be a longer-term process to actually get rid of it, which you would like because it would make the implementation simpler than having both options at the same time. But it's going to take a few years.

[0:25:18] SF: How does Python which each new version handle backwards compatibility? And specifically, with Python 3.13, is there any challenges that someone might face with upgrading to it? 

[0:25:30] ?L: Right. Historically, Python has been sort of rather fast-paced in terms of change. But that was a long time ago, like in the days of Python 2.1, 2.2, 2.3, 2.4, and so on and so on. Then we had this sort of weird state for quite a while where Python 2.7 was the last version of python on the generation of Python 2. And then the core development moved on to Python 3. That was first Python 3.0, 1, 2, and so on and so on. And it took us a long time to convince people to migrate their code since, unfortunately, the effort required to migrate from Python 2 to Python 3 was rather large. 

The core team at the time sort of under - essentially, the effort required was much higher than we anticipated. That was a lesson learned. We're not going to you know make that same mistake again. But only after like Python 3.5, with Python 3.6, we saw significant adoption of Python 3. A lot of users of Python from that age, they have this very comfortable mindset of like, "Python 2 was awesome. It never changed." But that was only because Python 2.7 never changed. All of core development was moved to Python 3 even at that time. 

Now, we are developing python 3.14. 3.13 was just released now. It will receive bug fixes from now on. We have an annual release cycle, which means, every October, you can predictably expect that there's going to be a new Python version. And that means a lot of more incremental changes. There's not going to be any Python 4. We're never going to do like a big rift of this is the old version. This is the new version. You have to do a massive migration. 

Definitely something that allows people to make more incremental adoption of newer versions of Python. And that also allows the library and framework authors to keep themselves up to date with the newest version of Python. That being said, we are still developing some changes that require backwards incompatibility. We're trying to limit those. But we are making some movements, for example, with performance being a driver of some changes in Python. 

In this case, there is a policy that we adopt where a change that is going to be disruptive is not going to happen from one version to the next. It's going to raise a pending deprecation, then a dictation and so on. And it's going to take three releases until that particular feature actually breaks. That gives us sort of enough time if you are keeping tabs on Python development to make the transition very smoothly. Because nothing is going to break on day one when you're trying to upgrade. 

Our users sometimes tend to be more conservative with upgrading. In which case, they're going to see some level of breakage. We try to limit it. But with an interpreter language, it is unavoidable that some details of the standard library and the Interpreter itself are going to change. And your code is just trying to be executed by this new interpreter. You cannot compile it with an old version and run it on a new box. Essentially, if you're running on new Python, you have to be compatible with new Python. 

[0:28:53] SF: Why is it that an interpreted language has a constraint more than essentially a type language that's compiled? 

[0:29:00] ?L: A compiler like a C++ compiler or a Rust compiler can more easily include behaviors of previous versions of C++ or Rust. In Rust, you can switch the editions of the language in the compiler. In C++, you can define which particular version of C++ you want to target. And the compilers actually have a lot of if, elses about how those behaviors should actually be compiled and what code should they end up with. 

For source code in those languages, you can have variants of if diffs where you can actually check for a particular environment for a particular set of configurations, you're going to actually only kind of do a subset of the code that you have in a file and either, I don't know, a C++ for a header and so on. Python does not work like this. You're going to have the new interpreter just execute your Python code. There are no switches to behave like Python 3.13, or 3.10, or 3.8. 

In theory, you could have them. But they would not be very distinguishable from just running Python 3.8 instead of Python 3.13. And the important distinction there is that if you had a toggle like that, that would still be a runtime toggle. All your dependencies, all your other libraries would need to understand that as well. Currently, the Interpreter language does not try to be multiple versions at once. It just implements a current version. If you need an old one, you're going to have to execute an old one. 

[0:30:35] SF: You mentioned some of the focus around some of the improvements in 3.13 is around improving performance. And you talked about a couple of those features. Are there additional features that are in this update that are focused on performance improvements?

[0:30:48] ?L: Right. This particular update is very much looking into big bets in terms of performance. I doubt like it'll be measurably faster on the default build that is not using the JIT and that it's not free-threaded and so on since those efforts are sort of multi-year. This was actually a thing that we needed to understand how we're going to approach since the annual release cycle is important for very many reasons. I'm not entirely objective here since I was the release manager that actually proposed this change back in Python 3.9. But we do see that, thanks to this, Fedora, for example, ships the newest version of Python in their own release that is essentially the same month as Python 3.13 and so on. The users can get the version of python that is latest very quickly after it's out. 

More importantly, Red Hat is able to test the newer version of Python with us as it is being developed, which catches a lot of issues and allows the community to also adapt their libraries faster. To the point where before Python 3.13 was even finally released, we had some release candidates and so on, the binary packages for NumPy were already waiting on the package index for Python 3.13 to get released. This sort of adoption is very cool to see. It is allowing also our users, if they choose to, to switch sooner than they could before. 

But what that also means is large-scale changes like the JIT and like the removal of the global lock, those will require more effort than a single release. What to do in that case? Well, we release a state that is usable. But it might not be immediately kind of visible in benchmarks why you would want to use this feature just yet. because all the effort was already spent on those, you might expect that there might not be very kind of visible changes performance-wise otherwise in Python 3.13. 

Python 3.11 and 3.12 saw a lot of good changes with making the interpreter make opcodes specializations. Not quite a just-in-time compiler, but a building block that we're definitely going to use in the JIT. Where if you run through a loop and make computations onto objects but you know that the loop has a counter that is only numbers, now Python will observe this and will notice at some point that, "Oh. Actually, those things are always two numbers." We'll specialize the opcode from a very generic add operation, for example, to an add operation that operates on integers in particular. That makes them faster and so on. Python performance from 3.10 to newer versions is definitely going to be better. You're going to see this. But 3.13 has focused on very big bets in that area. And we are in the middle of the development there. For that particular reason, all your benchmarks might vary for now. 

[0:33:55] SF: One of the things you mentioned earlier was that, typically, Python developers seem to be a little bit conservative with their upgrades of the language. What about the popular libraries and frameworks, how quickly do they typically pick up these new languages? Things like TensorFlow or Django. Immediately, once this update becomes available, they're jumping on to upgrade and have that support built in?

[0:34:21] ?L: As I said about like NumPy, which is a super popular dependency on Python, that was essentially ready on day one. There are others that sometimes lag for months after a release of Python. The reason why, it's usually because the developers there are volunteers themselves. They're not being paid for their work. They just need to be either patient or actually support them in a way where they can either be paid and spend the time to upgrade their library to a newer version. Or you can just propose that change your yourself. Maybe they're going to include it and release a new version without much effort on their end. 

But even then, as you probably know, there's no free maintenance. Even if somebody actually gives you a change, you still have to review it, you still have to test it, you still have to actually do the entire release process which might be automated, might not be. But in any case, that sort of widely depends on your particular dependency graph. 

We see that currently the most popular version of Python in the wild is depending where you look, either Python 3.10 or Python 3.11, which are versions from like three or two years back, which is not terrible. If you look at most of C++ code is going to target versions of C++ that are 10 or 20 years old. We're fine. However, we could do better if we made essentially less changes that are visible to the users. But a lot of the changes that we are making are some of the things that were actively asked for by users for literally decades. You have to make this compromise somewhere where the amount of breakage should be minimal. The amount of adoption should be minimal so that just old code is just going to work. But at some point, you have to just bite the bullet and make a change that people asked for because it enables some larger gains and so on. 

You mentioned Django for example. As far as I can tell, Django is already supporting Python 3.13 right now as well. I'm not actually keeping tabs on which popular libraries would not just yet. We can just actually just see TensorFlow right now. Let's see. TensorFlow 2.18 released candidate one just today. And that version, let's see the download files for this, actually supports Python 3.12. At least not officially, they're not on Python 3.13 yet. We'll see how long that takes for them typically. 

But we have annual releases. And before that release is replaced with another, it is essentially every time that all the major libraries do upgrade. But whether this is within the month of our release or 10 months, that is impossible to tell. Even for the same library, it might differ between versions. 

[0:37:20] SF: How much of a role does community feedback play in terms of shaping any new version of the language? 

[0:37:27] ?L: Right. Kind of in general, a lot. Right? Because the entire idea for having the just-in-time compiler and allow new news cases for Python and the removal of the global lock which will, again, allow for doing things that now Python was not very well suited for. All of those things are user-driven. Users are complaining. Users are going to get a new capability of the interpreter. 

To be clear, the process is that changes that are of smaller scale are just driven by issues on the issue tracker. Essentially, anybody can file an issue. Like saying, "Something does not work for me." Or, "I wish something new worked that does not today." If those changes are small and uncontroversial, they're just a matter of having an implementation. Which, again, thanks to the magic of GitHub and pull requests, essentially, anybody can do. 

For things that require design work, usually we would redirect people first to the official forum of Python, which is on discuss.python.org. And then in the ideas section of that forum, we can discuss new features that are either syntactic or features of the standard library. And when there's consensus for that, typically, they would need something more than an issue on the issue tracker. And when that happens, what you need to do is you need to write a Python enhancement proposal, which is a kind of RFC document, which is then sort of developed for a while and discussed for a while. It has some expectations what you're going to include in this document, including what the motivation is and what the rejected alternatives were and so on. And how to teach this new feature and so on. 

And, finally, you will claim that the PEP is ready for inclusion. In which case, you're going to submit it to the steering council. Steering council is a body of five people elected every year by Python core developers that makes those high-level design decisions on what is included in the next version of Python. And, hopefully, your PEP, if the steering council likes it, is going to be accepted and then implemented in the next version of Python. 

[0:39:36] SF: Roughly, what percentage of those actually get accepted? 

[0:39:39] ?L: I don't really have a number off the top of my head. But what I know is a lot of them do get accepted. And the reason why is that ideas that are like very wild and kind of raise a lot of sort of concerns and so on are typically rejected before you actually go and write a lengthy PEP with a lot of effort given to examples. How the implementation would look like? Rejected alternatives, and so on and so on. If your idea is one of those ideas that maybe sounds good, but if you really look closer it's going to be terribly complicated and controversial, you will learn sooner than that either on the issue tracker or on the forum. Which is why we redirect the initial discussion there since a lot of the kind of larger-scale issues is covered then when you just can have an, essentially, open discussion about an idea. 

At the time where a PEP is introduced, which some core developers just find to be perfectly fine with having rejected PEPs, kind of are also documents that tell you why a change didn't happen. There are some features that come up every now and again. And people would really like to have them. Having a document saying like, "Yeah, we looked at this. And these were the pros. But these are the cons," and the steering council at some point decided not to accept it, that is still worthwhile effort since it will essentially inform future discussions that would otherwise waste a lot of contributor time that something is more tricky than it looks like from the get-go. But, yeah. I don't really know the percentage of them being rejected.

[0:41:20] SF: Yeah. Most sort of infusible ideas are going to be filtered out before they get there. 

[0:41:23] ?L: Right. 

[0:41:24] SF: How much are you paying attention to what's going on with other languages and sort of factoring that into decisions made about Python? 

[0:41:32] ?L: Usually, the extent to which we are looking at other languages is very closely tied to the current challenges that we are facing. For example, when async and await were added to the language, we were paying close attention to other languages that have them. C# was a great example. And we looked at other languages that also already had this functionality. And sometimes we see the opposite to happen. Where Rust was adding asynchronous programming, they were asking us questions about how we approach certain things. 

When I was working initially with Guido and Yuka on typing in Python on PEP 484, we were very closely looking at what TypeScript is doing and what other languages with incremental, gradual typing, how those languages feel like. At the time, Facebook's HHVM had this variant of PHP that also allowed for gradual typing. It was also one of the things that we looked very closely at. And we talked with people who use HHVM daily to see what the things that they depend on are. What the challenges are with what they came up with? Sort of, I don't know, traps that they avoided or they did not so that we could avoid them, and so on and so on. 

We're definitely not blind to how development of other languages looks like. Now, the JIT is sort of a very important kind of interest area. For example, the design for that was very closely based on a paper about how a person made automatically a JIT for Lua, right? Which is not quite as performant as the Lua JIT written manually. But allows for much better maintainability and allows for you to essentially just keep it running on more platforms faster. And what that particular technique, called copy and patch, how that works. And that allowed us to set this up for Python in the matter of the 12 months we had since the idea's inception. Yeah. We're looking at the issues. We're looking at the developments in other languages but mostly to the extent of where we are actually spending our time on Python at the moment.

[0:43:56] SF: Yeah. Absolutely. Awesome. That makes a ton of sense. Well, is there anything else you'd like to share? 

[0:44:02] ?L: Well, since you asked me about new features of Python 3.13 and I told you like, "Yeah, everything is long-term. Nothing exciting," let me tell you about something that is a little exciting in this particular version. We replaced the interactive shell, the REPL of Python, with a REPL based on what PyPI had, which is written in Python, which is also amazing to see. Because now from this bunch of C code that we had before, you have essentially a Python library that is our interactive shell now which allows for a lot of people to contribute to it that were scared of that before since a lot of our python users are not themselves C programmers. 

This particular shell has some features that you would probably expect Python to have by now but it did not. And it has them now. Things like multi-line editing, so you can actually edit a multi-line block in one go. That is now supported built-in by Python. We did have external interpreters that you could download for Python from PyPI. Pip install IPython or something. But those had to be installed indirectly. And when we are ourselves developing Python's new version, sometimes downloading packages for that version is just not going to work. And other users like teachers, and students, and some school environment might not even be allowed to download packages from the internet. In which case, the built-in interpreter is going to be very important for them. 

The multi-line editing, and color support, and allowing for you to copy and paste things in a more predictable way where it's not going to break indentation and so on. Which, in Python, indentation is actually significant in terms of semantics. Right? All of those things are much better now in this new REPL. It is a small feature in terms of how large Python is. But let me tell you that once you start using this, going back to Python 3.12 just feels like going back an entire era of software development. 

[0:46:06] SF: Yeah. That's also a lot of times someone's sort of entry point to Python as well, which it's great for sort of lowering the barrier to entry there. Helping people adopt the language even faster. 

[0:46:15] ?L: All right. 

[0:46:16] SF: Yeah. Well, ?ucas, thank you so much for being here. I really enjoyed it. And have a great day. And cheers.

[0:46:21] ?L: Yeah. I'm very glad to be here.

[END]