Python Memory Management & Performance Optimization Course

Welcome, Welcome, to Python Memory Management and tips. This is a pretty unique course. We're going to dive deep into the internals of Python to understand how it works to make your code more efficient and make it faster. I feel like Python memory management is a bit of a black art. We use Python all the time, but most people don't really understand how Python memory works. How are Python objects structured actually in memory in the runtime. What does the reference counting part do? What does the garbage collection part do? we've heard maybe even you can avoid using the garbage collector altogether. It's a bit of a black art. Well, maybe not this kind of black art. Maybe this kind of wizard. Maybe this is the kind of black art, the programmer kind. But if you really want to know how Python Memory works, this is a course for you. We're gonna have a balance of high level conversations about algorithms, we're gonna dig into the CPython source code and we're going to write a ton of code to explore these ideas and see how they work in practice.

So why should you care about memory management in Python, anyway? Isn't it just gonna work the way it's gonna work, and we just have to live with it? Well, you'll see there's actually a ton that we can do to work with it or even change how it works. Here's a simple little program. It's running Python running one of our applications, and it was fine, but then a little bit later, it wasn't fine. Now it's using 719 megabytes. That seems bad. Maybe we should do something different. We're gonna learn a whole bunch of amazing techniques that would allow us to keep this memory usage much lower and we're going to dig in So you understand exactly the trade-offs you might be making and where these costs come from, how Python is doing its best to get the most out of the memory that we're using and so on. Not only will you come away with an understanding, but we're going to dig in and get some design patterns and some tools and techniques that you can proactively use to make your program look more like the one on the left, and less like the one on the right.

Now, when you think about memory management and making our code more memory efficient or understanding how the garbage collector and reference counting works, you might think this is to make our code use less memory. To make the memory required smaller. But it also will have a side effect. It will make our code faster. Our code Python will have to do less garbage collection events, potentially is using less memory so less swap space, maybe better cache management, but also just some of the design patterns and some of the aspects of Python that are not frequently used but we're gonna bring into it will actually make our code quite a bit faster for some really interesting use cases. this is sort of a general performance thing with a concentrated focus on memory.

Let's talk about what we're gonna cover in this course. We're certainly going to talk about Python memory management. But there's actually three distinct and useful things we're gonna cover here. So, three separate chapters. We're going to talk about how Python works with variables and look behind the scenes in the CPython source code to see what's really going on when we work with things like strings or dictionaries or whatever. We'll get a much better understanding of the structures that Python itself has to work with which will be important for the rest of the discussion around memory management. We're also going to talk about allocation. People often think memory management, clean up. You know, garbage collection, reference kind of and so on. And yes, we'll talk about those things, but Python actually does really interesting things around allocation and a bunch of cool design patterns and data structures and techniques to make allocating memory much more efficient, which then lead into making reference counting more efficient as well as garbage collection. So Python has two ways to clean up memory. We're gonna talk about them both, when either of them come into action. We'll also write a lot of code to explore that. The data structures that you choose to represent your data and your application can dramatically vary in how much memory they use. We're gonna take the same data and look at it through the lens of storing it in a bunch of different types of data structures: arrays, lists, dictionaries, classes, even pandas data frames and NumPy arrays and see what the various trade-offs are around these different data structures. Once we get to talking about functions, you'll see that there are some really powerful and simple design patterns that can dramatically make our code faster and more memory efficient. So we're gonna look at some really cool ways to make our functions use a little less or a lot less of the memory that we're using. We're also gonna look at classes because storing data in classes is super important in Python and maybe gonna create a list of classes, a whole bunch of them and so on, and you can have a lot of data there. So we're going to talk about different techniques we can use when we're designing classes in Python to make it much more efficient both in memory and it turns out a nice little consequence in speed as well. Finally, we're going to do some detective work. Once we understand all of these things we're going to take a script and we're just going to run it through some cool tools. Try to understand from the outside what exactly is happening in our application. How Python is using memory. We're gonna create some interactive web views of this data. We're actually gonna create some graphs, all kinds of stuff. So we're gonna use some really neat tools to do the detective work, to understand how our program is working and where we should apply some of these techniques that we've learned previously that make it even better and faster. That's what we're gonna cover. It's gonna be a lot of fun. It's gonna be hands on, but also some high level conversations. I think you'll get a lot out of it, and I'm looking forward to sharing it with you.

What are the prerequisites to take this course as a student, what do you need to know? Honestly, the requirements are really, really simple. You just need to be familiar with the Python programming language. We obviously don't start from scratch. We're digging into the internals of CPython. We're talking about the trade-offs of different data structures that you should already know from Python. So, you should be familiar with dictionaries, lists, the ability to create classes, functions, and then the core stuff like list comprehension and loops and whatnot. If those things are all in place for you, then you're absolutely ready to take this course. If not, check out our Python for the absolute beginner course as a great foundation, and then you'll be ready to jump in here.

finally, let me introduce myself. Hey there, I'm Michael. You can find me over on Twitter, where I'm @mkennedy. You may know me from the podcasts "talk Python to me" or the podcast "Python bytes". I have done both of those. And I'm also the founder and one of the principal authors here at Talk Python Training, and I'm super excited that you're in my class. Thank you so much for taking it and I hope you really get a lot out of it, so, let's get started right now.

in this short chapter, we're going to talk about what you need to follow along with the course and to write code and play with the same tools that we're going to use. Now, right off the bat, Would it surprise you to know that you're gonna need Python for a Python memory management course? Of course you are. But I do want to point out that you're gonna need at least Python 3.6 if you want to run the code that we're using and if you want exactly what we're using in this course, we're using 3.8, specifically, 3.8.5. These memory behaviors can change slightly from version to version, but they should be pretty stable in Python 3 at this point. But the syntax, things like f-strings and whatnot, we are using those, and those require Python 3.6 and beyond. So the code will not run without 3.6 so make sure you have that right version. Now, how do you know if you have the right version? Well, you go to command prompt or your terminal and type Python3 -V in Mac os and on Linux, and hopefully you get something like 3.8.5 or higher. The screenshot when I took it was a little bit older, but you know, you get the idea. On Windows, you probably should type Python space, Dash capital V. Now one of various things can happen here, especially on windows. On windows, what you might get is, it might say, Python 3, or it might say Python 2. You might also have Python 3 but your path might not have it first. If nothing at all comes out or the Windows Store opens, the newer versions of Windows 10 will try to go to the Windows Store and install Python for you if you type it. And that will happen usually if you just type Python or Python 3 by itself. But with this -V, if it says nothing, that means you don't really have Python installed. It's just this store shim that really should say "you don't have Python installed", even with an argument. Nonetheless, that's what happens. So make sure that you get Python 3.6 or above ideally 3.8 or above on your Windows machine or on Linux or Mac Os. Finally, if you don't have Python installed, there's all these varying recommendations of how you install it and how that changes over time, What the trade offs are. Personally, I installed Python 3 on my Mac using Homebrew and I installed Python three on Windows using Chocolatey. So, those are both systems package management systems for the operating system that lets you automatically upgrade to the next version just by saying Brew - upgrade or Chocolatey upgrade Python Things like that, which is really, really cool. But there's a bunch of different ways and trade off, so I recommend you check out this Real Python article: realPython.com/installing-Python/ and see what they say now, they say they're going to keep this up to date to try to keep up with the times or what the options are for installing Python. So if you need to, have a look here.

If you want to follow along exactly what I'm doing, I'm going to be using PyCharm so you should use PyCharm too. It's a fantastic tool for writing Python code and exploring it. However, if you don't want to, you can use other editors as well. That's fine. So, if you're going to use PyCharm, I recommend you just get the latest version and use that. They offer two versions. They offer the PyCharm Community Edition and The PyCharm Pro Edition. For what we're doing this course the community addition, the free open source edition is totally fine. There's one step where I say "let's see the CPU profile graph in PyCharm" and we spend about one minute on that. That requires the Pro Edition, which you can just look at the picture, you don't have to be able to run it yourself. Other than that, you're gonna be able to do everything with the free, open source Pycharm Community Edition. So, there's no real reason to not use it other than you just want to use another editor. I think this is a great one and you'll see me using it throughout the course. The best way to get PyCharm is to go to the JetBrains toolbox and get that installed and then use that to install either PyCharm Pro or PyCharm Community or both, and this will keep it up to date and let you know when there's new versions and let you roll back to different versions. It's a much nicer way to work with PyCharm and the JetBrains applications rather than just installing it directly. If you don't want to use PyCharm, which is fine, I recommend that you use Visual Studio Code. That's the other really good editor these days. Those two have, like 85%, 90% of the market share at this point. From what I can tell the recent surveys and so on, this is a great one. Just be sure that you install the Python extension or it's not going to do much for you. And you get that by clicking on this little button here and then the Python extension is the most popular one. Looks like it's been downloaded 55 million times when I took this screenshot, and I know it's much higher than that right now. So, two editor options, PyCharm, Visual Studio Code, PyCharm community free version will totally work for this course.

You want to get the materials, the source code and whatnot data files for this course. You could pretty much type them in as you go, but if you want to play with it and tweak it, there's a lot of little ideas where you just make a small change and you see what happens in terms of the memory usage or the performance or whatever, so I definitely recommend that you get the source material, so go over here to github.com/talkPython/Python-memory-management-course. If you're a friend of Git and you like to work with it, just go clone this repo. If you don't really know about Git and your uncomfortable doing it, you don't have to clone it. Officially, you can just click the green button where it says "code" and download It as a zip file and roll with that, right? There's no major changes or whatever we expect other than bug fixes along the way. Make sure you get the source code and take it with you. In fact, I recommend you fork and star it, just so you have it for sure. Alright, that's it. Go through these steps. You should be all set up and ready to take this course. I'm excited to dive into Python memory management with you.

it's time to get into Python memory management properly and start writing some code and looking at how things work behind the scenes. We're going to start by focusing on variables. Now, you might think the place to start would be garbage collection or creating new memory or whatever, but to understand how all those things work, we're gonna start by talking about variables and how Python manages them, passes them around and so on. We're gonna start in a place that might surprise you for a Python course. We're gonna talk about C. Now, just to be clear, it's only a little bit of C. If you don't know the C programming language, don't worry, it's a few lines of code, I'll talk you through it. The reason we're gonna talk about C is when you talk about Python, people say "Oh, you just type Python and you run that". What they almost always mean is CPython, right? Python is actually the programming language implemented in C. And you can go, we'll talk about some places where you can see the source code, how things work in the actual implementation of the Python runtime. Some people call the thing that executes Python, a "Python interpreter". That's true for CPython. There are many other things that run Python that do so that is not interpreted. We've got PyPy, which is a JIT compiler, not an interpreter, you've got Cython and so on, so I prefer runtime. But if you want to say interpreter, same basic thing here. So what we're gonna do is we're going to talk a little bit about C and how specifically pointers and things like that in C surface when we actually get to the Python language, because even though there's other ways to run Python, the vast majority of people do so on CPython.

Now let's look at an example of some very, very simple code in C, C++ and first C++ it's a little bit complicated, cause even simple code can get pretty complicated. But here it is. We're gonna look at this function so this is not object oriented programming like C++ more on the C side, and what we're gonna do is we're gonna have two person structures or they could be classes whatever. But we're gonna have two pointers notice the person star, not person, but persons star. That means we're passing the address of where that thing is in memory rather than making a copy of its data and passing it along. We're gonna take these two person objects that we're referring to by address and then we're gonna ask the question "are they the same?" The way we do this in C is you go to the pointer, which is some place in memory. In this particular example, it was something like this, like 0x7ff whatever it doesn't matter. This the value of this thing is just a place out in memory. And here in that memory location on what's called the heap, we've got two pieces of information, right? Two of these pointers point to those places out in memory. That's why we're gonna use this arrow operator P1 goes to our dash greater than arrow, the id. So follow the pointer out and then find out there that memory location what the id is and see if they're the same. So this is how it works in C, and it raises a lot of questions that are maybe not even knowable here. For example, who owns these objects? Who is responsible for creating them and ensuring that they get cleaned up correctly? That answer could vary. It could be allocated in the stack of the thing that called this function. It could be some other part of the program created it, and when it shuts down or that part cleans itself up, it's supposed to delete those out of memory. How long will they stick around? I don't know. Until the other part, whoever owns it, decides that they're going to go away. What happens if we forget? What happens if somehow the programmer lost track of that piece of memory? Well, that's a memory leak, and it's just going to stay out there forever. There's nothing that's gonna fix it. That's bad. But what's worse, usually is if actually it gets cleaned up too soon. And the reason is you'll still have that p1 pointing to that memory address, But what is living out there is either gone or it's Some other piece or some partial piece of some other data, and that could be all sorts of problems if it gets cleaned up too soon. So what I want you to take away from this is just there's a lot of actual bookkeeping or accounting around things that were created, how long they live, whose job it is to clean them up, when they can do it, and they've got to make sure that they don't forget to do it, but not too soon. Okay? So all of these things are really tricky around C and pointers, and we'll see in Python that generally that's not what we have to do. A lot of things are done for us, but we want to understand what is done for us. How is it done for us, and so on.

Let's look at one more example before we move on to talk about how Python does the equivalent thing. So let's look at a slight variation. We've got the same SamePerson function. It's going to take two people and determine if they're the same. However, this time, instead of passing pointers, which are basically numbers to memory addresses where the real data lives, we're just gonna pass the id's. Remember before we're saying p1 arrow, id. Might as well just past the id that makes it a little easier to do. So we have our two pieces of data which are integers p1_id and p2_id. They don't point anywhere. They literally just have the value, right? This is the same thing that was the id before. So in C, we can pass the value of a thing or we can pass a pointer like we saw before and there's good reason for both. If you have a large data structure and you want to move it around without making copies because that would be slow, You would pass by reference or pass a pointer. If you wanted to, even more importantly, you want to make changes to it and have those changes reflected in other parts of the program, you need to pass the pointer, change the shared location, and then everyone will see those changes. As opposed to here if we change the id only this function would see it. But, C and many languages, c#, other languages, they have this distinction between passing sometimes just the value, like the integers, and sometimes like previously the address of the thing that you gotta follow as a pointer out there. So, I want to put these two up and contrast them for you, and then we're gonna dive into Python.

Alright. Time for some Python code. In a big question, does Python have pointers? Well, let's look at a function. The same, SamePerson function but written in Python. So here we have a function: def same_person, and we're passing a p1 and a p2, and we're using type annotations to indicate this is a person class. So, p1 is a person, p2 is a person indicating also the function returns a bool, true or false, on whether they're actually the same person. But notice p1.id == p2.id That's not that arrow thing, right? We don't have to treat it differently. And if you think about Python, you've probably never seen the star in the context of meaning this is a reference or a pointer to a thing you've never allocated memory, you probably never cleaned it up. There's a del key word, but that means something totally different that doesn't really directly affect memory. Are there pointers here? I don't see any pointers. Here's the thing. Let's try to print out p1 and p2 and see what we get. Well, the interpreter would just say "__main__" we have a person class object. So person is the class and we've created one of them. So object at this memory location. Hmm, at this memory location sounds a little bit like, well what we have the pointers, doesn't it? You could also use this function, the cool function in Python a built-in called "id" and say, where does this thing, basically, where does it live? Hey, and if those numbers are the same, they're sharing the location. If they're not, it's a not shared thing. And we can talk about that. We will talk about that as we go. But if you go out here and actually look in memory, we're gonna have two things out on the heap dynamically allocated, And these are going to be pointing to it. Well, p1 and p2 are pointing to it. Those id of them actually correspond to the address. This is the same situation as we had in C++ pointers. The language is hiding it from us. We don't have to worry about it, right? That's cool. But as you think about, you know, what is the lifetime of p1 and p2? Who was in charge of it? All those same questions I asked, Come up. Who owns these objects? Well, in Python, the answer is better because you don't have to worry about it. Like I said, you probably have never really thought about cleaning up memory by, like, going free or delete or whatever on some thing you've created. Because you can't really do that. But somebody has to, right? If these get created. So the question Who owns it it's kind of, it's interesting, It's kind of the community of all the things in the program, all the things that share that piece of data. Once they all stop paying attention to it, it goes away. It goes away for one and other reason. There's a couple ways in which it could go away and we'll talk about it. But the runtime itself kind of owns of these objects. You don't have to worry about that. How long were they stick around? Until everyone is done with them, maybe a little bit longer, depending on how they're linked together. But generally speaking, just until everyone is done with it and the runtime also knows who's paying attention to it, so you don't have to worry about the time. It is really nice. So in a sense we have pointers in Python, yes, but we don't have the syntax of pointers, lovely. Nor do we have all the stuff for in the memory management and the accounting of who owns what, when, and when it should be cleaned up. All those things are gone, which is beautiful, but we also need to understand how and when Python does those things on our behalf right? Does Python have pointers? I'm going to say yes, Python has pointers, but you don't syntactically have to worry about it.

Let's look at that alternative example that we saw in C++, But now in Python. Remember what we did is we passed over the id We said these are integers and we're just gonna compare them directly. And here's the same code in Python, really The only difference is where the return value goes and whether it goes int p1_id or p1_id int. Right, this is the same basic code. And if you print out p1_id or p2_id, it's gonna look just like the same thing that we had in C++, but it's not the same thing. In Python, these are different than what we had in C++. In C++ they're just allocated as the program runs technically on the stack. They're not passed by value, all sorts of stuff like that. So we're going to see that, actually, these also live out in memory, they're dynamically allocated, they have to be cleaned up just like our person object was. So these are pointing out here in just the same way. In some sense, does Python have pointers? Python has more pointers than almost every other language, right? Even things like numbers and integers or functions themselves or modules and source code all of those things are objects and pointers. Even when they operate as basic numbers, like we have here these are basic numbers that are allocated dynamically out in memory.

I'm going to reveal the hidden truth about Python variables to you now. So, if you've seen the movie The Matrix, one of the best science fiction movies of all time, Morpheus spoke to Neo when he was just first realizing, or was being told, that he was living in a simulation and Morpheus told him a little bit about that and said, "look, you have two choices here, a choice between two things. One. You can take the blue pill. Forget this happened and just go back to being blissfully unaware that you're actually in this weird, dystopian world. Or you can take the red pill and see the truth. You can see inside the simulation what's actually happening". And I feel a lot of this course is a little bit like that. When we work with Python, it generally just works, everything's smooth. Sometimes it uses more memory than we want. Well, that's the way it is. There's not a whole lot we can do about it, right? What I'm gonna show you in this course, is there are a lot of little techniques that combine together that will either help you build better data structures, better algorithms and how you use those data structures, or at least just understand why your program is using a lot of memory or it's a little bit slow or whatever. And in order to do that, we have to look beyond the Python syntax down into the CPython runtime. So in that regard, this course is much like taking the red pill. By the way, if you wanna watch this little segment, the link at the bottom is like a five minute video. It's great. So let's take the red pill. Here's some Python code. Age equals 42, so age in an integer. Yes, its value is 42. You can add it. You can divide it. You can treat it like, you know, numbers in a programming language. It couldn't be simpler, right? Well, not exactly. So if we look at what is actually happening inside of the CPython runtime, wouldn't we work with numbers like these integer types, They're called integer types or int in Python, but they're actually PyLong objects in the C level. You'll see there's a whole lot of stuff going on here. Now, this is just a very small part of a single function in a very large piece of code that you can actually click "Go here" and go to bit.ly/cPythonlong. This will take you to this line in the CPython source code, which I don't remember exactly how long it is, but it's hundreds of lines long. So the ability to create one of these things is not super simple. Now, for the number 42 it gets treated special. Small numbers get treated special, as we'll see later. If this was 1042 it would be closer to what's actually happening. The important thing, Is not which one of these functions around Python integers runs, but, just take this one as an example. So, PyLong_FromLong, which is a C++ or C long that's converted to this PyLong, what does it return? It returns a PyObject pointer. Everything in Python is a PyObject pointer. Strings, numbers, functions, source code, classes you create, everything. This is a common base class for everything in Python, okay? But specifically what it's creating is a PyLong object that's then being sort of down casted to this lower version. So you can look through. There's a bunch of stuff even within this function I removed just so it would fit on the screen. But it's allocating here. See this line that says _PyLong_New(1). And then it does a bunch of work to it, is this pointer that's allocated? That's the dynamic memory allocation out on the heap. It does a bunch of stuff to set its value, and then it returns it as a pointer, which the Python runtime just converts that to something that feels like a nice little clean integer like that one line we have above. But there's actually a ton going on. So this is the red pill world that we're going to explore what's happening behind the scenes, the algorithms that are running, the reasons that they're happening throughout this course.

Alright, we've taken the red pill, let's go explore. Over here, I'd like to enter that link bit.ly/cPythonlong and it'll redirect us over to this horrible url, which you can see now why I was using Bitly to shorten it. This is, in the Python organization, the CPython project in the longobject.c file, and this is the PyObject PyLong_FromLong, and you can just go through and you can see all the stuff that's happening that's going on here. So there's all the stuff I deleted we're still in the same function and still in the same function. When are we done? Goes for aways. There's quite a bit going on here. Let's go to the top. You can see that this is the long, arbitrary precision integer object implementation. And in this objects section, you'll see all different sorts of things, like Booleans, and then you've got dictionaries, here's part of the dictionary implementation. There's the allocation stuff. obmalloc. Here's the base PyObject. yup, PyObject pointers all over the place, Right? This thing. So this is the implementation for that and one thing that's interesting is, you would think just the base class would be pretty simple, but it's 2000 lines long, like the header file. The header file, if you go up, is already, I think, like 600 lines long. So there's a lot of stuff going on with every one of these objects, and throughout various places in the course, I'll be giving you the short your url's, which will take you to different locations in the source code. You don't have to go look at it, especially if all the code here makes your eyes water. That's fine. You don't have to look at it. But this is where you can go and see exactly what is happening and when or what kind of data is being stored and what structure and how the runtime is using that to manage the memory or allocate things and so on.

Let's real quickly look at the documentation from Python's docs around this id built-in function, and it says "id is a function that takes a single object". And if you look at the description, it says "it will return the identity of the object. This is gonna be a number which is guaranteed to be unique and constant during the objects Lifetime, however, two objects may with non-overlapping lifetimes may have the same id value". That's more likely than just random places in memory. As you'll see, there's patterns that have a tendency to reuse memory rather than allocate new memory. Nonetheless, this while the objects are around, what this number come back means this is the unique identity of the object. And if you get two objects and you ask from them and they're the same, they're literally the same object in the runtime. If those numbers are different, even if they would test for equality like two strings might be equal equal to each other but they're not the same location of memory, potentially, then the id would come back to be different. And, little special detail here at the end, if you happen to be in CPython, The CPython implementation detail is that this is the memory address, the address of the object in memory. So this literally is the base 10 version of "Where's This Thing in Memory?" Id is simple, but we're gonna be using it a couple of times throughout this course to ask questions like "are these actually the same thing?" or "where do they live in memory?" and stuff like that. So, here's the deal.

Now finally, let's write some code, as we're going to do for much of the rest of this course and explore these ideas. But before we actually start writing, I want to just show you how you can work with the code from GitHub and just get it started if that's somewhat new to you. So over here, we've got our code structure. Now, right now, these are all empty. I just have little placeholders, so Git will create the folders. Git only tracks files, not folders. So this is the trick to make it create them for me. What I want to do is just open up this entire project here in PyCharm. Now, the first thing I want to do in order to do that is create a virtual environment. We're not actually going to need the virtual environment for quite a while, but let's go ahead and just start out that way. So I'm gonna open a terminal here in this folder. I checked it out as a mem-course not its full name, but I'm gonna create a virtual environment. So, Python3 -m venv venv, and we want to activate it. On Windows, this would be the same. Sometimes you drop the three. This would be scripts, and you wouldn't do the dot. It always turns out that whenever you're creating a virtual environment almost always pip is out of date. That's super annoying, though we just want to make sure that we're going to upgrade pip. Alright, now we're ready. I'm gonna take this directory mem-course and drop it into PyCharm. And because of the way I put it over here, it's kind of annoying. You can't get back to its super easily. So if you right click up there, you can get a hold of the course. I'm gonna drop this on PyCharm. Now, on Windows and Linux, you can't do that trick. Just go file, open directory, and select it. Now, for some reason, it found the wrong virtual Environment, so I click here, add interpreter, pick an existing one, go to my project directory. Sometimes it gets it right. Sometimes it doesn't get it right. Perfect. So you can see it's running down here, and it looks like it's probably the right thing. So let's go down here and we're gonna just add a quick little function that we can, file that we can run just to make sure everything is working. I'm going to call all the things that are here that are meant to be executed directly "app_something" and then there's gonna be a bunch of libraries that they use but maybe don't get executed directly, and those will just be whatever their names are. So that's the convention. I'm gonna try to make it clear what you can run in here and what you can't. So I'm gonna call this "app_size". And then as a good standard practice, we want to create a main method we want to use the "__name__='__main__'" convention. I've added an extra personalized little live template to PyCharm so I don't have to type it. So you'll see me type "fmain", and if I hit Tab, it writes this for us. You could write that yourself, but it's super annoying to write it all the time So I'm gonna write this and we'll just print out. Right click, Run. Looks like we're using the right Python to run our little project. Cool. So now we've got our projects open. Once we get stuff in here, I'm gonna be deleting these, but that's how you open the code. Of course, you'll have the code already here, but then you could just go right click and run the various things that you want to explore.

Alright, let's answer some questions and you can see I've already threw in the questions up here just you don't have to watch me type like print. How big is this? How big is that? And the thing I want to explore is how much memory is used by certain things. So we get a sense for a number versus a string that has one character versus a string with 20 characters or a string with one character versus 20. Like how much is it? The character space versus all the underlying runtime infrastructure. That's gonna contribute to the memory use. So we're in luck. Python has a good way to answer this. We're gonna import sys. And over here we can print out things like sys.getsizeof a thing like the number four. So let's do that real quick and then run. How big is the number four? Well, in a language like C++ or C# or something like that where these are just allocated locally, you always have to talk about the size and the number is like a short or a long or something like that. But typically this would be 2, 4, or 8 bytes long. in Python, a number, a small number, is 28. If we had a little bit bigger number, it's still 28. Let me make a little bigger so it stays on the screen But if it's a lot bigger, we use a tiny bit more memory. So the size matters, but not so much. But there is some overhead. Remember, this is the PyObject pointer and all the things to know how many people are keeping track of it, where it was allocated, what type it is. All of these things are happening behind the scenes, and we just see the simple number 4, but Python is doing a bunch of work through that infrastructure that we talked about. Remember the red pill stuff? That's what's happening, that's why this a little bit bigger. Alright, what about this one? Let's print sys.getsizeof the letter "a". Well, these you feel like these might be similar, right? I mean, in most programming languages, that's 1, 2 or 4 bytes and this is 2, 4, 8 So maybe it's even smaller. Let's see. Nope. 50. It's bigger. So it turns out, strings have a lot going on, so there's a little bit going on there. And let's see if we have a big string like this. How much larger is it? 25. Well, 25 larger right? 75 now, and that's because we have 26 of these rather than 1. So you just multiply that by 26. We get our 26 with multiplied by 100. Look, it's about 100 bigger. Okay, that's what's going on here, right? So basically, there's this infrastructure to keep track of the string and do all the string things and then the extra data. And if we had Unicode characters, they might take up more than one byte per character. How about a list? Simple little empty list. How big is that? 56. Okay, that's not too super large. Now let's do something with like, 10 items in it. There we go. I put 10 in there. Let's see how much bigger it is. 136. Well, that's quite a bit bigger. Let's think about that for a second. Well, what does the list actually contain? The list doesn't contain the values the list contains basically what every variable in Python is It contains a pointer to a number out in memory. So there's somewhere out in memory a 1. And in here in the list, there's the go find the 1 the number over there and then here's another pointer out to the 2 wherever it is in memory. These smaller numbers are interesting where that actually is. But we've got a list, and it's got 10 of those in there. The lists generally don't allocate one slot at a time. They kind of grow in a doubling type of way, like, you've got 10, and then you add a few more so we're gonna allocate 20 and then 40 and and so on that kind of pattern so that you're not constantly allocating every time you're adding something and copying cause that's super slow. All right, well, that's how big that is. Sort of. We're going to see that this going to get interesting and let's actually do something else. So how much memory did this take like, How much does that line contribute? Well, it contributes, we saw that each number is 28 and then it's gonna allocate whatever the list needs to be. The list by itself is 56. So each one of them would have 280 bites, probably for all of those numbers, those 10 numbers because they're 28 each and then we're gonna have the 56 for the list. That's like 320 or something 330. And then there's also the pointers that are gonna be in the list as well that have to point out there, maybe the over allocation. So something's going on like this is not big enough, right? Just in the numbers alone, they should take 280 bytes. We're going to see what's going on in a minute but this is how much room this thing is taking. But let's try to force the issue by saying "What if there's a large piece of data right there and a large piece of data right here?" So I'm gonna do a little bit of work here. I'm going to come up with some data that we're gonna ask about, and I'm going to go from 1 to 11. I'm going to add in some stuff 10 times. The number of elements in the list should be the same. I'm going to come up with an item and the item is going to be a list that starts out with the number of whatever it is the loop. So, first time through this will be 1 second time It'll be 2, and so on, and Python has this funky little trick that we can do here. So if n is 7 and we can come over here, there's a list of Let's put, like 3 in there and we times n what we get is a like a multiple a list with that copied that many times. So here we get a single list with seven 3's instead of just one 3. We're going to do that here. i times i so first it will be 1, then 4, so on, times 10. And by the time that gets to be 10 that's gonna be 1000. So it'll be a list with 10 in it 1000 times. the last one that's out here is gonna be bigger than just, you know, the number 10. Absolutely. You're gonna put that item in there and let's just really quickly print out data just so you see what we got here. Notice there's a whole bunch of tens and a bunch of nines fewer and so on, right? It kind of grows geometrically. All right, so that looks like that did kind of like what I said it did. And let's just print sys.getsizeof, Data. 184. What do you think? I'm gonna think no. no, that's not right. But, this does give us a sense of what the base size is. So what are we answering or what information are we getting when we say getsizeof? What we're getting is it goes through and it says, "I'm gonna look at the actual data structure, the list". So, this one right here and let's see how much it's internally allocated, what are its fields? And if it's got a big buffer to store items it puts in it. How long is that buffer? But what it doesn't do is it doesn't traverse the object graph. It doesn't go "Okay, well, there's 10 things in here. Let me follow the reference from each one of those 10. See how big it is. And if it has references, follow their references" it doesn't do this traversal which is actually what you need to know about how much memory is used. But this getsizeof, its A start, you'll see that there's a better way that we can get going to actually answer this question more accurately.

To answer the question of how much memory do these things actually use in the way that I describe you traversing the object graph, We're going to use a cool foundational library: "psutil" and I said we didn't need the virtual environment right away. Well, I forgot about this one, so we need it right away. We're going to pip install that one. You can see PyCharm if I just click This will do that for me. I added this requirements file, now I put it in there. Alright, that's all happy, but what we're gonna do is we're going to write, or take this code over here that basically will do that traversal, so give me the size, and it'll just recursively go looking and say, "is that a dictionary? Does it have a __dict__? As in it's a class. Is it a thing that can be iterated"? and so on. And it's just going to recursively dive into those things. Okay? So we can use this to traverse that object graph over here, if we just import it. Looks like it will import right? But no, it doesn't. Even though you can be assured that is what I called it "size_util". And the reason is PyCharm is looking for something up in this folder called "size_util". So what I am going to do is I'm going over here and tell PyCharm "also look in this directory as a like a top level directory for imports" and the way you do that, is you say mark directory as sources root, and we're gonna need that for the others, potentially as well. Here we go. Oh look, now it works. So, what we can do is we can do the this and say "get full size of that" and let's do it for the "a" and let's do it for "26 a's". Now, those things don't have stuff they contain, so these numbers are likely to be the same. And they are "28 28 50 50 75 75", but where it gets interesting is where you have container objects. So if we just put the list in because the List is empty it's probably the same size. How big is the list? It's still 56. Alright, so those are the same size, but here's where it's going to get interesting, where we look at the size of something that points to or contains other things that are not empty, basically. Look at this one: the 10 items instead of being the 136, if you take into account all the stuff out there, so the other 280 bytes that you're gonna get from these, then that's how big it is, okay? And then here's the one where it was a really out of whack. Where It was really crazy in terms of, we know we allocated tons of memory and yet it said 184. Not likely. So let's go get the full size of this data list that we've created. There we go. It probably would be better, even if we go with this way little comma digit grouping there 31 basically 32 kilobytes for this beast that I created here this geometric growing list of lists. So if we're going to understand the amount of memory of these objects take, we gotta look at all the container objects, the things that they contain and recursively do that, right? If there's container objects that are contained, look at their things that are contained and so on. And that's what this little utility does, and we're gonna be using this throughout much of the course. Do not use this sys.getsizeof which is great for the relative size of the like, the core essence of the thing, but not the object graph that it relates to, and that's what this size utility is all about.

Let's talk about a cool little optimization that Python uses for some of the objects that are common and reused. Now you've heard me talk about the numbers, and seeing small numbers behave differently than others. And that's because Python applies what's called the flyweight design pattern to them. So let me just read what Wikipedia had to say about it: "in computer programming, flyweight is a software design pattern. A flyweight is an object that minimizes memory usage by sharing as much data as possible with other similar objects". So, for example, numbers that have the same value could literally be the same place in memory. Those are immutable. It is a way to use objects in large numbers when a simple, repeated representation would use an unacceptable amount of memory. So, like if you have the number 3 appear 1000 times like that's gonna be say, that would be 28k when really it could just be 28. So often some parts of the object state can be shared. It's a common practice to hold them in external data structures and pass them to objects temporarily when they're used. Okay, so that's the flyweight design pattern, and you'll see that Python uses that some of the time, especially around numbers.

Alright, let's do one more demo. Let's explore this idea of flyweight numbers. So, I'm gonna create a new little thing we can run here, app, remember that means you should run it directly, all right. And we'll do our fmain to just boom pop us into the right pattern. And I copied some text here that's just gonna help us a little bit. So this is gonna be an example showing number, which numbers are pre computed and reused. That is the flyweight pattern. And what I want to do is come up with a range of numbers from -1000 to 1000, and then we're just going to say for this 872 and that 872 are they the same? So a really simple way to do that is to have just two lists, each with -1000 to 1000 numbers in them, like -1000, -999 etcetera. It will say this is gonna be a list of range from -1000 to 1001. Annoying, but it doesn't include the end right in this range thing, and we'll do that for 2. We're going to do it twice. What we want to see is by doing it twice, even though the exact same values will be in there, will they be the same memory address? Will they literally be the same PyObject pointer thing in memory? Or will they just be the equivalent values? All right, and a way to do that, we'll just keep a track of reused, make a little set or something like that. Actually, let's make it a list so we can sort it. for number 1 number 2 in we're gonna use this cool thing called "zip", and if you have two lists, what it's going to do, if they will get out the way, is if you give it two lists, it will take an item from one and the other and put them together as a tuple. So if I say list one list two, I'll get -1000, -1000, -99, -99 each time through. But they're going to be the values out of the two lists, and we're gonna store them into n1 and n2. And just so you know what we're doing here we'll print n1 and n2 and that's not what I want to run, let's run it here. Here you go. You can see, It's just like lining them up side by side, but this one comes from list 1, and this one comes from list 2. So what we want to do is we want to ask, Are they equal? Of course they're gonna be equal. More importantly, are they the same place in memory? So we'll say if the id of n1 is equal to the id of n2 then we're gonna reuse, go to our reused and append doesn't matter which one just one of them to say "this number is reused". Then we're gonna print and print out "reused", something like this: "Found Reused, n1". All right, let's go and just print that and run that real quick and see what happens. Ok, this is pretty interesting. Look, it goes up to 256 and it starts at -5. It's weird, right? So the numbers -1000 up to -6, Those are not reused. Those were different allocations treated as totally different, unrelated things that just happen to have the same value. But from -5, to, scrolled the wrong thing, to 256, these are literally the same thing in memory. So there will only be one 244 PyLong object pointer in the runtime because it's always using it over and over this flyweight pattern. So we can also just be a little more clear to make sure we know what's happening. So we have a lowest equals the min of reused, and the highest is the max, and we just print out flyweight pattern from, put a little "f" in the front. There we go, lowest to highest. Alright, one more time. Just like we saw flyweight Pattern is from the numbers -5 up to 256. This is actually a super cool pattern that you can use in your own code. You saw, like, Python objects and data structures are fairly expensive, memory wise. So if you have the idea of like we've got immutable data that's reused in lots of places, you could come up with your own concept of a flyweight pattern here and optimize that. But, you know, that's not really the point here. The point is more to talk about the internals of what Python is doing around the numbers to prevent them from going totally crazy in terms of how much memory they consume because -5 to 256 those numbers appear all the time.

If you're a fan of this red pill type of thinking, you should definitely check out Anthony Shaw's CPython internals. I've had him on the podcast a couple times to talk about it, and he's finally written it up as a book. So this, like 350 pages talking about all the CPython source code, how you can figure out, you know navigate where in the whole structure, different things are, like where the object headers vs the objects themselves and digging into a whole bunch of interesting things the parsers and whatnot. So if you want to dive into not just the memory side of CPython, which is what we're gonna still spend a lot of time on, if you want to look at the whole thing, the parsers, the execution and everything, you know, check out Anthony's book. It's definitely good, recommend it.

When you think about Python memory management, you're very likely thinking about reference counting, garbage collection, how things get cleaned up. But memory doesn't start there. It starts by getting allocated, and it turns out that Python has a whole bunch of techniques and patterns and ideas around making allocation fast, efficient, doing things so there's not memory fragmentation, all of those kinds of things. Allocation in Python is actually way more interesting than you would think. So we're going to spend some time talking about the core algorithm, using pools and blocks and arenas, which maybe you've never heard of, but all of these things are really important in working with objects to prevent memory fragmentation and prevent them from blowing up in terms of taking too much memory and so on. So this will be really a fun and important prerequisite before we get to what most people think when they think of Python memory, and that's the cleaning up side of all that.

Let's do a little thought experiment. Imagine we have this one line of Python code, which we know whole tons of stuff is happening down in the runtime. But on the Python side, it's simple. We have a person object. We want to create them, past some initial data over to them, their name is Michael, and so on. Now let's imagine that to accomplish this operation we need 78 bytes of RAM from the operating system. What happens? How does that get into Python? Like, what part of memory do we get? How is it organized and so on? So a very simplistic and naive way to think about this would be all right, what we're gonna do is going to go down to the C-layer, and C-Layer is going to use the underlying operating system mechanism for getting itself some memory. It'll malloc it, right? So malloc is the C allocation side and then free, we would call free on the pointer to make that memory go away, Okay? So you might just think that the C runtime just goes to the operating system and says "give me 78 bytes", and the operating system says "super, we're gonna provide 78 bytes of virtual memory that we've allocated to that process", which then, boom into that we put the thing that we need, some object that has an id and the id wasn't explicitly set. But, let's say it's generated. The id is that in the name is Michael. Well, that seems straightforward enough. I mean, you have these layers, right? Python is running and then Python is running actually implemented in C and C is running on top of the operating system and the operating system is running on real RAM on top of hardware. So this seems like a reasonable thought process. But no, no, no. This is not what happens. There's a whole lot more going on. In fact, that's what this whole chapter is about, is talking about what happens along these steps. And it's not what we've described here. Let's try again. So at the base we, of course, still have RAM. We have hardware. That's where memory lives. We still have an operating system. Operating systems provide virtual memory to the various processes that are running so that one process can't reach in and grab the memory of another. For example, we saw that there's ways in the operating system to allocate stuff. So at C, there's an API called malloc that's gonna talk to the underlying operating system. This is what we had sort of envisioned the world to be before. But there's additional layers of really advanced stuff happening on top here. Above this, we have what's called Pythons allocator or the PyMem API and PyMalloc. So the C runtime doesn't just call malloc, it calls PyMalloc, which runs through a whole bunch of different strategies to do things more efficiently. We saw that in Python and CPython in particular, that every tiny little piece of data that you work with, everything, numbers, characters, strings, all the way up to lists and dictionaries and whatnot, these are all objects, and every one of them requires a special separate allocation, often very small bits of data, and that's why Python has this Pymalloc. But wait, there's more. If you're allocating something small, and by small, I mean sys.getsizeof, not my fancy reversal thing. So if you're allocating something that is in its own essence small, then Python is going to use something called the "Small Object Allocator", which goes through a whole bunch of patterns and techniques to optimize this further, and we're going to dig into that a bunch. So if you want to see all this happening, you can go to "bit.ly/pyobjectallocators", the link at the bottom, and actually the source code is ridiculously well documented. There's like paragraphs of stuff talking about all these things in here, but there's actually, in there, There's a picture, ASCII art-like picture that looks very much like this diagram that I drew for you with some details that I left out, but they're in the source code.

As you saw, in just the last video, Python has this Small Object allocator. Well, if small objects are treated differently than other objects, you want to know what objects are small and more of them are small than you would think, because most big objects are actually just lots of small ones. So let's take a look at one example. There's many we could talk about. So here we've got data, a list that contains 50 integers, 1 to 50, and in a lot of languages, the way this would work is you would have just one giant block of memory that would be, say, 8 times 50 long for each 8 byte number that's gonna go in there 4 times that right 200 bytes if they're regular integers and so on. And it's just one big thing. But that's not how it works in Python. When you put stuff together like this, you're gonna end up with the list that has, you know, a bunch of pointers as many pointers as there are in the actual list. Plus probably a few extra for that buffering so you don't reallocate like I talked about, but what you're actually gonna do is have a bunch of little objects that are being pointed to by the parts of the list. And if you have a class, you're gonna have fields in the class. The things that are in there are like strings and numbers and maybe other lists those are not part of that object in terms of how big it is, those are outside of there, pointed to by, pointed to by the variables within that data structure. So these objects that feel like they're big, often they're many small ones, and if that's the case, all of these little things get stuck into the algorithm applied to the Small Object Allocator, not a big object. Even though taken as a whole, they might use tons of memory. Most of the parts will probably still be effectively as far as Pythons concerned small objects.

Let's look at one of the red pills in CPython around object allocation. So this is "obmalloc.c", and if you look here, you can see here's the ASCII art part that I was telling you about before. This is when the second take that we did on what allocation looks like. We have physical RAM, we have virtual memory, then we have malloc, then we have PyMem, API, the allocator, but there's something really interesting at the bottom that we didn't talk about then and check that out. It says "a fast special purpose memory allocator for small blocks to be used on top of a general purpose malloc heavily based on prior art". And if you want to go check this bit of the source code, this is literally straight out of GitHub, just go to bit.ly/pyobjectallocators, and it will take you right to this line, and you can look through it. So what's the deal? To reduce the overhead for small objects, that is, objects that are less than 512 bytes in the "sys.getsizeof", not the whole traversal, but the small bit, Python sub-allocates larger blocks of memory, and then, as you need it, will free up, or give, that memory to different objects, which potentially could be reused once that object is cleaned up. Larger objects are just routed to the standard malloc. But for these smaller ones, which are most of them, the Small Object Allocator uses these three levels of abstraction. We've got arenas and pools and blocks. At the lowest level we've got this thing called a block, and then pools manage blocks and arenas manage pools. So we're gonna go through all of those. But there's this trifecta of ideas or algorithms that we're going to use to manage, remember, the small objects. And this little quote right here comes from an article about Python memory management by Artem Golubin, and he's done some fantastic research and writing around it. So I recommend that you check out his blog. There'll be a couple of articles that I think I refer to. Definitely, I've read as researching all the stuff for this course, so check out his article here. It has a lot of interesting analysis on what's happening and why it's being done.

Let's start at the lowest level where the actual objects are stored in memory. Instead of allocating as we saw at the very beginning of this chapter 17 bytes or 20 bytes or whatever you need exactly for a thing just randomly where you've got a gap in your memory, Python uses these things called blocks. Blocks are chunks of memory of a certain size, and each block is designated to hold objects of a certain size. So, for example, we might define a block that holds objects of 24 bytes, or around 24 bytes, let's say. The places where the objects go are 24 bytes and anything that's between 17 to 24 bytes is allocated into those 24 byte spaces. And sure, if you've only got 20 bytes you need and you stick it into a 24 byte spot, you're wasting, quote "wasting" 4 bytes, right? You could have packed it a little bit tighter. But this algorithm allows Python to create these sets of memory, that it's really easy to allocate stuff into once it's freed up, just un-assign it, but not give it back to the operating system, necessarily. Then when you want to allocate something new, maybe next time it's 22 bytes, you can use that same little spot and reuse it. Okay, so that's the idea of these blocks. And there's some rules. One of the rules is it only holds, each block only holds things that fit into its block. So if you've got one, that's the size for a 24 byte element, things of 17 to 24 bytes go in there, but they kind of waste the space if they don't totally fit and you can see they're broken into these different categories. So once a block is allocated, it's always dedicated to its size. Its either a bunch of stuff that fits into 24 byte pieces or 16 byte pieces and so on. So you can think of Python allocating these blocks, for of the different size of allocation it's going to do and then be able to just reuse that memory. It doesn't have to go back to the operating system, free up memory, ask for more memory, get that fragmented on in RAM and things like that. It can get a whole bunch of space for those pieces of those small objects that it needs and just works with it internally, and it's more efficient that way.

The next level up in this algorithm are the pools. Now pools, their job is to manage a bunch of blocks and a given pool always contains blocks of the same size. So if Python needs to allocate something that fits into a 16 byte blocks, it can just go to the pool that contains those and ask for it to allocate it, ask for one that has some free space and so on. The pool size is typically set to match the memory page so it maps well to RAM, it doesn't get fragmented, and Pythons ability to reuse this fixed, contiguous set of memory helps reduce fragmentation that would otherwise happen if we just went to the underlying C-layer and just said "give me the next free bit of memory that you have of this size". You can look at this a little bit here is the source code around the pools. It has the next free block. Right there is a pointer that you can always get you, and it also has the next pool and the previous pool. So it's kind of a doubly-linked list of pools that within there contain a bunch of these blocks.

Let's go back to the CPython source code and look at this idea of blocks, pools, and arenas. We haven't talked about arenas yet, but let's just look over there real quick. So I've come up with another Bitly URL for you, cause like, all of these are super long. So "bit.ly/cPythonpools", if you want to go there, and you might be surprised to see what you get when you come here. You don't get source code immediately. There is source code if you scroll enough, like here, there's some source code, but check this out. There's like a full on essay about what is happening around one of these pools. Okay, so if we go down a little bit further, it talks about how blocks are managed within the pools. So blocks in the pools are getting carved out as needed. The free block points to the start of a linked list of the free block, so you can always just go there and start allocating into that one. So pretty awesome, right? There's a bunch of stuff about, like things that they're doing that might be a little bit obscure, or what's unclear, and so why it's happening that way and so on. So when I was studying this I Was really surprised at the level of detail, describing how pools and blocks interact together. And if you want to see what the core developers and the people who worked on this and implement it and maintain it, what they say, Well, here it is, right in the source code on GitHub.

The last data structure or idea that we gotta cover when we think about how Python is working with small objects are arenas. And arenas are 256 kilobytes of memory, they're allocated on the heap and they manage 64 pools. You can see down here the data structure that defines them. It's quite similar to the pools. You've got doubly linked list. You've got the next free one. Things like that. These arenas, this is the top level thing. Arenas contain a bunch of pools. Arenas are always the same size. The pools are often the same size, the blocks that they contain, they might be different scale, they might be 8 byte objects, they might be 16 byte objects, though, you know, the second would only hold half as many as before, but that's the idea. We've got arenas that control the pools, the pools hold the blocks, and the blocks are where the objects actually go with 8 byte alignment.

Well, with all this abstract talk about blocks and pools and arenas can be a little bit hard to understand. So, let's just actually go and see what Python will tell us about all of this stuff and the Small Object Allocator. So, go here and create an "app_something", Remember? That means you can run it and I'll just call it "stats". Real simple, we'll just do our fmain magic once again, we're gonna need to use sys, so there's our sys, Perfect. And let's you put a little message to see what this is about. Great, so over here, we can just run "sys.", It's not great, kind of this semi-hidden, but you can still get it.. "debugmallocstats()", like so. Let's go ahead and run this, see what we get here. Notice it didn't come up in our auto-complete, Right? "_debug", no, not there. But, we tell PyCharm not to obscure it and not to tell us it's not there. Hey, We can see that it looks like it's okay. Alright, so let's look and see what we got out of this. Check this out. Run it once from scratch. If we get to the top, what do we got? We have this "small block threshold". Remember, things that are managed by the Small Object Allocator, they are how big? 512 bytes or less. And there are 32 different size classes. So, 16, 32, 48, 64, 80 and so on, is apparently what we got right now. There's how many pools of each size of those are allocated and how many blocks are in use within those pools? so like 5 pools, we've got 558 blocks used within those 5 pools, and we don't need to go allocate more until we've used up 72 more of them. And of course, all of that is only for 32 byte elements. All right, so let's scroll through this. This just gives you all the information about the blocks, their sizes, how many pools go with them and whatnot. And then here's our arenas. How many arenas have we allocated? 12. So that would be 12*256, Should be 3MB areas reclaimed. 3, The max we've had is 9. Currently, we have 9. And then it tells you that this much memory is being used right now. You can go through and you can see a lot about it here, Right? Then we come down here and it actually tells you what is allocated. So we have "PyCFunctionObjects", we have 56 bytes each. We have 5 of those, so that's cool and so on. Right. So You can see all the different objects: tuples, stack frames, floats, dictionaries and whatnot being used here. So you get a lot of operations. Let's go and make it do something. So I'll just come down here and say "make it do memory things". So I'm just gonna copy some code. What we're gonna do is we're gonna create a bunch of larger numbers, so that's going to make sure they get outside the flyweight pattern and they're gonna actually get allocated every time. And then we're gonna create a list, then we're gonna put a list which has the string repeated 100 times of whatever this large number is that we get here. So we're creating a bunch of objects. Basically, we're doing a bunch of things here. Alright, let's try this one more time and see if it did anything different this time. Scroll, scroll, scroll, scroll, where is my separator? It didn't flush quick enough, did it? Alright, so we gotta go through the first one, right? The first time we see 512, there we go. That's the second one here. So you would see that there's potentially more of these in use. Not a whole lot more. You can see, I think more frame objects were in use here and so on. Let's look one more time. Yeah, the data is still there. Make it a lot bigger. Here we go, it's slow. It's doing stuff. Okay, great. Now you can see more arenas were allocated. We had 62, the highwater mark was 34 and we're still there, were using 8.9MB of RAM and so on. Then we can see our list object and all these different things. Okay, we have more of these blocks in use than we had before. So the 64 one's, 1000, 700, and before we had only 138. So you can see as we do more work, It's consuming more memory, it's getting allocated into the different spaces, and so on. Not sure how much meaningful information you're going to get from, like, actionable stuff you can do here. But I definitely think it helps understand some of these basic ideas that we've been talking about and gives you a look at the current state of the system, and that's pretty cool.

Well, we've come to the end of the line. Not for our course, there's a lot left there. But for the life-cycle of Python objects. We're going to talk about cleaning up memory and destroying Python objects that we've been working with, but we don't need anymore. This is really what most people think about when they think about Python Memory Management. I think the stuff about allocation is not even on people's radar for the most part, but this is. This is when we're working with objects and we're working with data, how does Python make it go away without us having to do what C has to do? Where you have accounting and some part of the program is responsible for tracking when a thing can go away and it's got to make sure it doesn't do it too early, but not too late. So we're gonna talk about two different techniques in Python that, but let's not worry about that, but understanding it is still super important. So if we grab our red pill again and jump over into the CPython source code, you can just go to "bit.ly/cPythonobject" to go find this bit here. Remember, that we saw object.c already. This is object.h, the header file that defines what is one of these objects. There's 667 lines just in the header. This is a crazy thing. So I've only grabbed a very small bit of the definition of PyObject. So we're defining a new type, You can see at the bottom it's called "PyObject", we're gonna have a whole bunch of pointers to it, but the most interesting thing I want you to take away in this part is to do with this "ob_refcnt". So how Maney references, how Maney variables, pointers, remember, Python doesn't have pointers, but it kind of does so, references or pointers point back to this particular object? and it's a "Py_ssize_t", which is a weird type, basically, it's a way to get an unsigned number, an integer or a long or something like that, that matches the type of architecture you have. So like on a 64 bit processor it's probably 64 bit or 64 bit version of Python, a 64 bit number or a long, things like that. Think of it is just a number, an integer, and it just counts: 0, 1, 2, 3, 4, 5 as different variables point at this object, Python behind the scenes will increment this number, and as they stop pointing at it, cause either they go out of scope and get destroyed or they get assigned to another variable or to none or something like that, Python will take this down, and when that number reaches zero, the object is cleaned up. Boom, That's it. Remember, it's taken out of its block so that other things can be put into that hole into that block and reused. This is the primary way, this reference counting idea is the primary way that Python keeps track of things that you're working with and cleans them up for you. There's a secondary way, because reference counting does have some flaws. But when you think of Python Memory Management and things getting cleaned up, this should be the first thing that comes to mind.

Alright, we're gonna do some fun programming over here, and it's fun because you've got to be a little bit clever to work and understand reference counting in Python, because if you create a variable and you point it at a thing so you can ask questions about it like, "is it alive or not?", it's always gonna be alive cause you're giving at least one to that reference count. So we're gonna use some cool little utilities we're gonna write. So I'm gonna write one first called "memutil" and I'm going to drop some code in here because we don't need to do a whole lot here. So we're gonna use this "ctypes" thing, which actually allows us to get kind of a direct reference to PyObjects, that's the same one we talked about, and then give us the field reference count. Now I'm going to add that to the dictionary so it doesn't look broken. It's going to come back as a long, like I said, and then what we can do is if we get one of those id's, remember, id of thing, we can come down here and use this, say parse it from that address, and then we're going to get the reference count. Okay? We're gonna use that. Then we're gonna have our "app_refcount" That's the one we're gonna run. I'll go ahead and set it to run now and I'll just hit the hotkey, we don't need it that big, do we? There we go, and I'll do my fmain magic, so it's ready to run. Now we're going to need something interesting to work with. First of all, we can work with the garbage collector. We're not really talking about the garbage collector, and I only want to work with it to the extent where I say "let's not have it do anything". Let's just disable it for now so we don't think about it or worry about it because what's happening has nothing to do with the garbage collector. Later we're gonna enable it and focus just on its role in this whole world. We're gonna create some variable, call it "v1" and let's just give it the number 7 and then we'll have, I'll call it "oid" for object id. It's gonna be the id of v1. Now what we need to do is we need to grab this early so we have the id. This number knowing the memory address that the location given to us by id will not keep this thing alive. Right? So it could be potentially up for collection. And we could still use this to ask from our memutil here. So we'll say "import memutil" as well. We're gonna need that, but in order to actually track this really clearly, we need to do one more cool thing. So I'm gonna create an object, a class, and I'll call it "doomed". Why is it doomed? Because its only purpose in life is to get created and then get destroyed by the reference counting cleanup system. So let's go in here and find a class "doomed". Now, this is going to basically plug into the Python data model, the "dunder methods" to capture different lifetime events of this object. So we'll have an "__init__", and this is going to be when it gets created. Let's go ahead say that it can have some friends. I don't think we're gonna use this yet, but this "*friends" means this will come in as a list and then we'll just print. We'll come over here and say "at {id(self)}". So we created this doomed thing wherever it happens to be now. That's what happens when the thing comes to life. And then we're gonna have another one run when the thing gets deleted. Then we wanna have some string representation of it so it's easy for us to just print it out and see what's going on and we'll just do the id there, but we want to have a string Representation so we'll have a "__str__" and we'll return a "__repr__" as well we'll just make them be the same. We're just gonna paste some code so you can see what's happening. All right, so what we're gonna do is we're gonna say "there's a doomed object at this address and it has however many friends if there are friends, Otherwise it's not going to talk about his friends". Doesn't want feel bad. So this is gonna be are doomed object. We're going to create one of these over here in our code. Instead of creating 7, I'm gonna create a "doomed" and it's going to start with no friends. We'll come back, the friends is more important in the garbage collection side of things. But let's just run this and see what happens. Notice it created a doomed object at some address and then it deleted the doomed object at that address. And that delete was when this function returned, no more things pointed at it, and it went away. But we can be more interesting than that. Let's go see what else we can do. Let's start by first printing out how many things refer to it. So let's say "print", and then here we're gonna use our "memutil.refs(oid)". Now we run this. You might think it's zero right, cause we're kind of done with it, but it doesn't get cleaned up explicitly. Doesn't get cleaned up unless we explicitly do so until line 13. So it should still say "1". There we go. Step 1 ref count is 1. Now, let's go to step 2. Step 2 is we're gonna have another variable that is equal to v1. So remember the way this works is Python sees this and says "there's a new variable defined that is now pointing over there", and so we now have v1 and v2 pointing at our doomed object, and so now it's gonna have to increment that reference count. Let's try that. Sure enough, reference count is 2. Alright, let's do some more things. Let's go over here and do this again. Step 3 is we're gonna change where variable 2 points. We're going to tell it to point at none, but it could also be at 7. It doesn't matter. If it's just not pointing at the doomed thing anymore, that's going to decrement, take away one from the reference count. Can you see it went "1, 2, 1", okay? Pretty cool. And the final step, step 4, let's go and tell v1 it also no longer points at doomed. And at that point, nothing, not v1 not 2, nothing else should be pointing that v1 and so it should clean itself up. And let's do a "print('End of method')". Right? So we should actually see this go to zero. We should see it get cleaned up here, and this should return zero, and you should see all that before it's getting cleaned up naturally as part of the method return. Are you ready? Let's see what we got. Here goes. Beautiful. Okay, reference count is 1, then it's 2, then it's 1, and this line, you can't really make it happen all at once, but this line 19 is what is causing this right here. When we say "the last thing no longer points here" immediately, like on line 19, basically, this thing is getting destroyed, the memory is getting reclaimed. And then later we can say, Well, now how many things point at it? Nothing. But that's already been the case on line 19 which did the cleanup. And of course, that happened before the end of the method. And if we don't do this one, you'll see the cleanup doesn't come until after the end of the method. So that's pretty cool. And I want to emphasize this is not non-deterministic. Rather, I should say this is deterministic. It will always, always be the case that on line 19 this will get cleaned up. Run it again. You can see it definitely ran between step 3 and 4, the destructor, if you will, of our doomed object, right? And it said "hey, I've been destroyed, I'm gone, I've been deleted by memory management, either reference counting or the GC". This is really important. This is incredibly lightweight. All you have to do, all Python, rather, has to do to implement this is just to keep count of how many things point at it and when some variable changes assignment just increment or decrement that number. If that number ever hits zero, immediately take it out of the block and tell the block that spot is now available again, right? You don't even actually have to clean up the memory. So this is really, really efficient. In the deterministic part, a lot of languages use garbage collectors as the primary way of cleaning up their objects, you know .NET, Java, those types of things. And because of that, it's non-deterministic. When the garbage collector runs is based on the behavior the program has had over time. You can't say on line 19 this thing's gonna get cleaned up. You can say "well when the memory kind of gets full enough and the heuristic decides that section of memory is worth looking at again, then it'll get cleaned up", and that could be problematic for real-time things, like stock trading, that has to have no latencies of, like, 4 milliseconds or whatever it might turn out to be, right? So this deterministic aspect of reference counting is really nice, because it's going to behave the same way, memory wise, every single time.

Well, you heard me go on about the benefits of reference counting. It's fast, It's lightweight, it's deterministic and yet we saw there's this other thing called a GC that must be doing something, a garbage collector, and it wasn't involved in reference counting. So what's the deal? Where does reference counting fall down? Where does it break? Reference counting is excellent, but one thing it cannot deal with is cycles. You have one object you create and then that object refers to another and then some other part to another, which links back to itself, which might create some kind of cycle. You're gonna end up in a situation where the reference count can never go smaller than 1, so the object will never be freed, and it'll be leaked. Memory will be leaked and it won't be great. There's some interesting things you can do around that for performance, but first, let's just look at the problem. So let's start with some simple code. We have our person that we created, and we can add friends to them. So we're gonna create one person whose name is Michael. There they are. They're out here in memory. So person name is Michael. They have some friends, there are no friends in there yet. Create another person. Her name is Sarah. She's out here in memory, and she has no friends either. But notice each one has a reference count of 1, and 1 because p1 points at Michael, so that's 1, p2 points at Sarah, so she gets 1. However, Michael and Sara are friends, so we're gonna go over to Michael, p1, and say "add to the friends, or appended to the friends list, p2, that's Sarah". So that means that Sarah is one of Michael's friends. Sarah, being a lovely person, wants to reciprocate that and says, "hey, I'm also a friend. Michael is my friend because I'm his friend", right? So in the same way, we're gonna put Michael into Sarah's list of friends. And now look, each of them have 2 reference counts. And now we decide "hey, we're done with p1", and that's going to take away this link, and the reference count for Michael is 1. The 1 comes from Sarah and her friends list, pointing back, and guess what, we're done with Sarah as well, so we're gonna take away that link, and her reference count goes down to 1 as well because Michael is one of her friends, and she's in his friend list. So look at this. We're in this situation where there's no more variables pointing at either Michael or Sarah, and yet their reference count is 1. What action could possibly happen in this program that will make that go to zero, either for Michael, so that he'll get cleaned up, which will take away the reference count of Sarah and get her to clean up or in reverse? Well, there's nothing left pointed at them. No one can manipulate the friends list because no one even knows about these variables anymore. This is just a fundamental flaw in reference counting garbage collection. If you end up in a situation like this, you're done. Those things will never, ever be cleaned up. I guess you could like C++, remember to always break the cycles. But that's not gonna work, right? It might not be this simple. There could be, Michael has some other friend who has some other object which holds on to other people who then hold on to Sarah, who happens to be one of Michael's friends or, you know, something like some big, long, complicated chain. It's not a 2 person linked cycle. It could have many, many links in that cycle, and it could be really hard to understand. So these cycles, that's what fundamentally breaks reference counting. You can see, we've even set both the variables to none, and there's no mechanism for cleaning up Michael or Sarah because they're in this, like, locked bit where you've gotta wait for one to go away to get to the other, but that's never gonna happen.

Let's look at this cycle thing in the context of our little reference counting app that we've done, and I'm just gonna print out of the top like this running reference counting demo, cause we're gonna have another kind and these are gonna be super similar so let's go over here and just make a copy, say, this is going to be "gc", and this is gonna be the "gc demo", and this is going to explicitly enable it. We don't have to do that, but I explicitly disabled before so let's make sure of whatever else happened, It's in there. So we're gonna do kind of what I talked about in the previous videos, we're gonna have 2 things, person 1, person 2, and We're gonna say "v1.friends.append(v2)", and vice versa, right? These are 2 people who are friends. Let's go and tell it that this is alright. We're going to tell it "this is a list of friends". There we go. Make sure it's a list. Now, we want to have the id. So let's have this "id1 and id2, v1 and v2" just for the same reason we needed to keep track of those. So, id1, now let's change this to "counts are", and id2. We don't need our step 2, well, we're gonna change what step 2 means. Let's go and actually set "v1 = none", and "v2 = none", and then we'll put our reference count out again, like that, setting it to none twice is not gonna do anything. Let's set it to be the end of the method and see when things get cleaned up. Maybe one of these will go away. Maybe both. We don't know. Let's find out. So we're gonna run this one. Well, look at that. So we've created a new object, doomed1 and doomed2, those are Michael and Sara, for example, and we had up here, I'll make this more legible for you, after this, we had one pointer to each one reference count, now they're friends, so they pointed at each other, that incremented it again, and you can see now the reference counts are 2 and 2, and we threw away the original variables and went back to 1 and 1 but they're never going to get better than that because of the cycle. So we went all the way to the end of the method and they never got cleaned up. What happened? Well, "gc" is non-deterministic. It's based on how the program behaved. We're gonna talk about that in a minute. What are the gates that it has to go through and where the rules it uses to decide when to run, what objects to look at and so on? But the short version is just cause these are free, the garbage collector is not constantly running around. It can only run so often, otherwise things would be super inefficient. So let's go and make some stuff happen. Both to step 3. Down here I'm gonna just, kind of like we had before, just make a whole bunch of allocations happen, and the fact that these are lists and other things that can contain data is actually important. OK, like this isn't enough. You need stuff like this. Anyway, what we're going to do is we're going to allocate a bunch of things, and if there's enough allocations, potentially, it could trigger the garbage collector to run. So we may see cleanup before 3, or maybe not. Let's find out. Look at that. So over here we created the objects. We had originally 2 variables and then the links to each other, so that was a total of 4 (2 and 2), and took away the variables down to 1 and 1, and then eventually after we did a whole ton of allocation, thousands of lists and then multiplying that and right tons of stuff happening here, enough allocation and memory pressure was put on the system that it said "OK, OK, OK, we need to slow up for a minute and run the garbage collector, see if there's any garbage". It found that cycle that we were talking about, that we created there to doomed objects and said "you know what? This is a cycle. It's out", threw it away before this line happened. So it was the behavior of all of this allocation and tell you what line or whatever. But somewhere in this looping around, making a bunch of stuff happen triggered the GC to run, we'll talk about what the scenarios are for that, of course, And then it did a collection, it found the cycle, it deleted them both and then we carried on right? So if I, again, if I comment this out and I run it, you'll see the doomed deleted at the end because it was at the end of the program. But if I do a bunch of stuff, it gets deleted along the way because eventually there's enough memory pressure to trigger the GC to go look around. So, unlike reference counting, GC is not deterministic. But it's not as important because it only applies to objects that are put into a cycle. And in fact, the only place objects that can contain other objects, right, like a list and contain other things or a class can contain other things. But a number, a string, all those things they are always, always reference counted. It's just the container objects, if you will. We're going to talk a lot about the algorithm, but this is the basic way in which it works.

Let's look at our GC example just one more time. There's one thing that I think is worth exploring on its own. you'll recall, what we did is we had these two objects here, we create these two doomed and we said they represent people and people have friends. So what we're gonna do is we're gonna say person1 is a friend of person2, and person2 has person1 as a friend. And then we're gonna blank them out, and because these references point to each other, we saw that they're not going to be deleted or cleaned up by reference counting, they have to use garbage collection. So over here, we had put this back the way it was. Here we go. We had this and we did a bunch of allocations here, created a list, and by doing all this allocation, we created enough objects that the garbage collector had to run, so between step 2 and 3, you can see these doomed objects were collected, the cycle was detected, they were GC'd, everything was good. That seems straightforward, right? Let's make a very small difference here. Let's suppose instead of creating things that can contain other things, like lists and so on, well, we're gonna do is we're gonna go down here, and actually that may make a copy so you have both in the source code to play with. We're gonna go down here, and we're just going to allocate some variables. We're gonna allocate about the same amount of stuff here. We're gonna create X, which is whatever the range is squared, and then we're gonna create a string of it. And then we're gonna create, replicate that 100 times within that string and store that in Y. Now, never mind that it says it's not used in PyCharm. It's still used as far as the garbage collector is done until we get to the end of this function, okay? so, let's run it. Wait a minute. What just happened? Look where the stuff got cleaned up. For some reason, even though we're doing basically the same stuff, I mean really, really similar as before, we're not getting a garbage collection triggered. Why is that? Well, there's something super subtle going on here. Over in this version, we're creating container objects. Lists. We created a class that would work. If we created say, a tuple, it could contain other things. What we're doing here is we're creating objects, here a list, which has a bunch of strings and so on, and then we're putting it in the list that technically doesn't really matter. But we're creating objects that could potentially have cycles. Like a list containing an item which could then point back to the other list and things like that, right? These container objects: dictionaries, sets, tuples, classes and so on, they potentially can contain other objects so they potentially could contain a cycle. But down here in this one instead, we have numbers, we have an "i" which is a number between 1 and 1000. We have "X", which is the square of that number. We have a string created from X, a temporary one, and then it's created, replicated 100 times, and we store that in "y". Can any of those hold other objects? Create a cycle to them? No. They're all immutable, actually. Right? So the things we're creating are not of interest to the garbage collector. So when we talk about allocation and the garbage collector even paying attention to it, what you're going to see is that the garbage collector doesn't track everything that Python works with or does. The garbage collector pays attention to container objects: dictionaries, lists, classes and so on, Okay? So the things that are even eligible for garbage collection are a massive subset of what you generally work with in Python, and it's super important to realize what triggers and interacts with the garbage collector and what doesn't.

We've seen this mysterious GC, this mysterious garbage collector in action, but we don't really know what triggers it. We saw that it somehow has to do with container objects versus non container objects like lists and dictionaries rather than strings and numbers. So what we're going to do in the next couple of little sections here is talk about how this works. So Python has what's called a "Generational Garbage Collector", and this is a really important optimization that most modern garbage collectors have. And the way it works is all the objects start out in was called "Generation Zero". It talks about how many times have they been collected, in this case never, and then once an object has been inspected but not deleted, it was inspected, and it decided that it was still alive for whatever reason, it's promoted into Generation one, and then sometimes we'll see about 1/10 of the time, the stuff in generation zero and generation one will be compared. Most the time we'll just focus on Generation zero, and that's because the overwhelming pattern is that new, temporary little things are created for a moment and then thrown away. So objects will live very briefly. So typically it's enough to just look in generation zero, but we'll see the mechanism by which Python determines it needs to look broader. So it'll look at maybe generation one and zero items if it needs to look a little broader and if it finds something in generation one that is still valid, but it has been inspected twice, It's now sent to two, and same thing applies. Sometimes Gen 1 and 0 will be inspected, but rarely will Gen 2. So these things are even older and even more infrequently inspected, and we're gonna talk about the ratios and how you can see and configure that and all in just a minute. But by default, things start out in generation zero. If they happen to survive a garbage collector run, an inspection, then they get promoted and they get inspected less often. If they survive a second or further time, they get promoted to generation two, and over in generation two, they don't get inspected nearly as often. Remember, reference cycles only occur in container objects: lists, dictionaries, classes and tuples, and so on. For that reason, only those objects are subject to Python's garbage collector, right? We saw that we worked with strings and numbers and so on, No problem. They did not interact or trigger the garbage collector. When you think about all the stuff and all these algorithms, it's easy to think "Well, every little thing I do in Python has this applied to it on top of reference counting". No, just container objects. So that's worth keeping in mind, too.

Now we've seen what this garbage collector is and how it restricts what it pays attention to, both the types of objects that it pays attention to and the frequency which it checks different ages or generations of objects. But when exactly does it run? Is it some odd heuristic that we don't really know about? Or is there something more concrete? Well, Python, unlike a lot of different systems, is pretty straight forward in what it does. So we can actually call this function "gc.get_threshold()". You'll get a tuple of numbers back. You're gonna get 700, 10, and 10 in the current system. Now what's confusing about this, what's unclear about this, unless you go look at the documentation, this is not how often Generation zero runs versus generation one versus generation two, the units on these things are different, okay? So the first one is different than the other two. The first number, the 700, This is a generation zero collection, so one of the cheaper, easier collections. This is triggered when the number of allocations surviving reference counting minus the ones that have been cleaned up exceeds 700. That's what that 700 on the left means, it means we've allocated 700 more things that have lived even when you take into account the ones that have been deleted. Now Remember, this is not all allocations. If that were the case, this wouold run like crazy all the time. This is going to run only when you've got container objects: classes or dictionaries and so on. If you create, let's say, 800 classes and only 50 of them get cleaned up well, that could trigger a garbage collection. Okay, so that's this first number. That's the 700. The second number says we're going to trigger a Generation one collection This is a little bit broader search of the memory space and can be more expensive because we're looking at more objects potentially. Now, this number 10 here means we're gonna do a Generation one collection for every 10 generation zeros. To write the generation zeros on, we do that every 700 extra containers, and then one in 10 of those we're gonna actually look a little broader, and we're gonna get a generation one. So that's what this 10 means. From generation zero to generation one, the ratio of those is 10 to 1. If we look at the final 10, this is when a generation two collection is triggered, and that's when the number of Generation one collections is greater than this number. So for every 10 generation one collections, there's a gen two collection. So that might sound a little bit complicated, but let's break it down. For every time that we have this exceeding of some fixed number of allocations of container objects, 700, that's going to generate a Gen 01, and then for every 10 generation zeros Let's say, we're have 1 generation one collection. And then every 10 generation one collections we're gonna have a generation two. So it's a 1 to 10 to 100 from Gen zero to gen one to gen two number of collections, and it just happens to be the thing that starts it all off is the number of allocations of surviving container objects like classes and dictionaries and so on. All right, Hopefully, even though that's a little bit of a lot to keep in your mind it gives you a sense of what's going on here. Like, when is the this whole process going to get started? It's pretty easy to see we've got 100 to a 10 to a 1 ratio for all of these in the generations for the GC. It's the allocation thing that kicks off the base of that whole process. That's a little bit unclear. It's also worth pointing out that there's a "gc.set_threshold()". So if you want to change these, you can. Personally, I haven't really tried that, but it seems like there might be some pretty interesting performance benefits you could get. You know, I'm thinking of things like I go to a SQLalchemy model, and I do a query against the database and that is going to return 1000 records, that's gonna trigger a garbage collection. But, you know, it's very unlikely those things are gonna have a cycle. They were just created, right? So, you could do things to say, maybe be in those situations where cycles are very rare, you could kick up that base number. It seems like something you might be able to play with. We're gonna look at some examples of people doing way more insane stuff than that, but, it seems like playing with that base number, you could probably get some pretty interesting performance benefits or maybe drawbacks. It depends, but you could definitely make a big impact by changing that first number, that 700, to something bigger or smaller.

If we go over to the CPython documentation for almost the latest version, we're doing 3.8.5 but here's 3.8.4 and you look up right at the top of the garbage collector module, you're going to see some very interesting stuff that I've highlighted here. It starts out telling you about it. It provides an interface to the optional garbage collector, and it provides the ability to disable it, to tune the frequency, which is what we just spoke about, add debugging options and so on. It also lets you ask which things are unreachable but cannot be freed for various reasons and so on. But check out this underlying thing. You can disable the garbage collector if you're sure your program does not create reference cycles. Automatic collection can be disabled simply by calling "gc.disable()". How crazy is that? So if you have a program that you're sure doesn't create cycles or honestly, if it doesn't create too many cycles, right? If this is like a command line script and it runs for two seconds, it doesn't create a lot of cycles even if it leaks a little memory, who cares? if it's a Web server that runs for ever, maybe that's a problem. Maybe not. We will see. But to me, it is super interesting that this garbage collector is considered optional and that right at the top of the documentation, it's like "you know what? If you're feeling confident, turn it off, you might not need it". So that's pretty cool. It also says here "to debug a leaky program call "gc.set_debug(gc.DEBUG_LEAK)", and this includes DEBUG_SAVEALL causing garbage collected objects to be saved in some place in memory rather than been cleaned up for inspection". So that's also interesting, but this ability to disable it, this is intriguing. I wonder if you could do it. If anybody would, What would the outcome be? Well, you might have heard of this place called "Instagram". I think they do something with photos. They actually do an insane amount of work with Python. All of Instagram runs on Django, at least their back end API's and their website and so on. And I think they're one of the largest deployments of Django in the world. They've got a massive set of servers and so on. They wrote this article over on their engineering blog, which they have a lot of cool Python stuff They talk about, called "Dismissing Python Garbage Collection at Instagram". And it's pretty intriguing, it says, "by dismissing the Python garbage collection mechanism, which reclaims memory by collecting and freeing unused data, Instagram can run 10% more efficiently". Yes, you heard it, by disabling GC entirely, we can reduce the memory footprint, not increase it, reduce it by, I think they said, maybe 25% or something like that. Quite a bit, and improve the runtime performance by improving the CPU LLC cache hit ratio, and you want to know more why you can check out this article here at the bottom. It's probably better just Google it. It's one of these yucky medium URL's, but nonetheless, quite, quite interesting. So here's the TLDR version. So they were able to determine that the way web servers work is they'll create not just one version of the server for running your Python code but they'll make many of them. So for example, at Talk Python, we use uWSGI, and when we run it for the training site, we actually have eight copies of that process running. Eight independent, separate copies of the Python web app that you very likely are using in some form or another right now. There's a lot of memory that's shared between those things and the operating system is pretty good at saying "we're only going to consume more memory for all these different processes if they're going to start changing it, but if it's actually just the same, let's just point them all at the same bit of memory" okay? When that's the case, you actually get a lower memory usage because even though we have eight processes, instead of having eight times the memory, we might have, you know, 10% - 20% extra memory that has to be created, and 80% could be shared. I don't know if that's the actual ratio, but you know, that's the general idea that a lot of the core startup runtime bits are all the same, and then there's what that process is done this particular time since it started. Okay, so what they found by disabling the GC, it was actually mucking with the memory in a way that would actually not allow that memory to be shared by the operating system. So even though they may have had a few cycles that created, you know, some issues for them, what they found was they got, I think, they said 25% reduction in memory usage. So they saved 8GB per server by turning off the GC. And also because the memory is more similar across these different processes, it's more likely that as different processes process requests, that data is going to be in the cache, the CPU cache, and CPU cache access is much faster. So a rule of thumb might be "if I'm going to read from disk versus something from memory, could 200- 400 times faster to read it from RAM than it is from disk". So obviously you think something in RAM is blazing and something on disk, even a fast disk, is relatively slow. Same for the cache, though. That cache is like 400 times faster than RAM. So if you can get more of these cache hits, you can get your code to run much faster. So they came up with this 10% number, sort of like a CPU performance boost plus memory reduction so we can run more things on the same server and so on. The number is not really important, the general idea is, and the fact that they were able to apply this is pretty interesting. But if you want to do this, you should read the article because it's not straight forward how they did it or whether that worked well for them. So, depending on what you're trying to do, it might be as simple as calling "gc.disable()", but read the article and you'll see there's actually more to what they had to do in their fairly complicated set up. All that said, I'd probably leave the garbage collector on, maybe turned that first number, that 700, up much higher. But I know that it's kind of my first impression as I'm thinking through these problems, But, you know, this is one of the types of things that maybe let it just work the way it is. But if you feel like you could give this boost, these are some of the knobs and ideas that you can play with to improve it pretty easily around garbage collection and memory management.

If you thought about multi-threaded programming or parallel or asynchronous programming in Python, you've surely heard about this thing, the "GIL", or the "Global Interpreter Lock", and you often hear this presented as a negative. What this thing does is it Only allows a single Python instruction to be executed at any given moment, no matter how many threads you have or how many cores you have, none of that, Python instruction one at a time. So that could be very limiting for how parallelism works in Python. Now there's a lot of situations where it works awesome, actually I go through it in my asynchronous programming course. But things like "I'm waiting on a network connection from a response from a Web service or some kind of API I'm calling", that doesn't count is executing a Python statement, that's down somewhere in the OS. So if you're waiting on other things, actually the parallelism is great. But for computational stuff where the computational bits are in Python, this GIL is a big problem. Actually, Eric Snow and some of the other core developers are working on PEP 554 which will create what are called sub interpreters or multiple interpreters that take sort of a copy of everything we've talked about and replicate separate, isolated versions and each version can run on a thread and it'll potentially solve or alleviate some of these problems by not sharing objects. That's in the future. I have a lot of hope for it. It sounds really cool, but I want to just bring up this GIL cause it's actually super interesting around what we've just talked about about, not the garbage collection, more the reference counting side of things. So let's think about a world without the GIL. We have stuff running. We've got these different references to the PyObjects, different pointers that are all pointing back from potentially different threads, and they're coming and going, as they do, multi-threaded, right? In parallel. So in that case, you're gonna have to do some kind of thread lock, some kind of critical section or muText or something around that piece of data that is the reference count on the PyObject, every single one, even things like numbers and strings. You can imagine that's gonna drag down and be super, super, slow right? That's a lot of overhead for every single normal operation. So the tradeoff was made that said, Well, "let's only let one bit of Python run at a time", in that case, there's no possibility of a race condition around the reference count, so we don't have to have thread-locking on it and it'll be much faster. So that's really what the GIL is all about. People think of it as like a threading protection thing and it kind of, sort of is. But really, what it is, is a reference counting protection mechanism that allows Pythons reference counting to be done without thought or care of parallelism or thread safety or all those things that are somewhat hard but certainly expensive relative to not doing them, and they can just freely work with these objects and because of the way it runs, you're never gonna get a race condition on that pointer. That's a global interpreter lock. When I always thought about it, when I first heard about, was like "this is a threading thing", and technically yes, but really the most relevant part of it is this has to do with reference counting. So it's a Python memory management feature, if you will. Actually, you can read more about it over at Real Python Check out the article. They always have a bunch of good articles on this type of stuff, so they've done a really good one exploring the Python GIL. You can see the URL at the bottom. If you want to learn even more about it, go check it out there, but keep in mind when you hear about the GIL, it's actually there to make reference counting much faster and easier. Yes, it's a trade off, but it's an interesting one to consider, and a lot of times, if you're not doing parallelism, you're better off because of it.

So what we've done so far is we've tried to understand Python memory management techniques and how it works. We've talked about allocation, we've talked about cleanup with reference counting and how they're problems with reference counting. So there's also this garbage collection thing and so on. All of that was kind of FYI, so you'd have a better understanding, right? Well, what we want to accomplish is to come out of this course not just with a better understanding, but with the ability to actually make decisions that can dramatically improve the amount of memory we use, by making it less, or speed up things, or ideally, both. So we're going to start down that path in this chapter and carry on in the chapters that follow. So let's start by talking about the goals of this chapter. To some degree, you're going to see that what we can do will actually make things better. But also, sometimes it's just easier to go with the grain and understanding how the garbage collector and reference counting and allocation works, you can either go with the grain or you can go against it. Obviously, working with the way that Python is already gonna work is good. It's better. So I want you to keep this in mind as we were talking about all of these things and especially looking back at what we've done, it's important to just know what the direction of the grain is, what the way the system wants to work so that you can not fight it and work along with it. But again, in this chapter, but especially the next, we're gonna be looking at some actual techniques that will let you change the way things work or maybe do this to an extreme where we actually change the performance in really important ways.

There are gonna be a couple of glaring omissions of what we're covering during this chapter. We're not going to talk about functions and we're not gonna talk about classes. Not really. Obviously we're gonna use some functions and we'll probably have some classes. In fact, I'm certain that we will. But we're not going to focus on some of the techniques that we can apply directly to them to get huge, huge gains. In fact, it's because they're so important that we're going to have a dedicated chapter on optimizing functions and a dedicated chapter on optimizing classes. So I want to just set the expectation that we're not gonna deal with things that maybe address how functions use memory or how classes can be restructured to use memory better in the Python language features. We will talk about using classes, but not the language features to make it better yet because those were really important and they're coming up soon, but we have some foundational stuff to cover first.

I hope this article from Instagram about how they disabled Pythons garbage collection and used less memory and not more made an impression on you. It sure did on me when I first heard about it. It honestly kind of blew my mind. What was it that made this possible? Why could they do this? Well, let's go back to the Python documentation. Right here in the center line bit, once again it says "you can disable the collector if you are a sure your program does not create reference cycles", and I would change that a little bit like "it doesn't create too many reference cycles" because, you know, if it's short lived and it creates a couple, who cares, right? If it leaks a little bit of memory. But if, like the core thing you're doing, the core data you're working with require cycles, well, I guess you can't do it. So what we're going to do in this next section is we're gonna look at a situation that has reference cycles and say "is there some way to restructure the algorithm so we no longer have these cycles in our data?" And then it would be possible to turn off the garbage collector. Not necessarily saying we have to do that, but we're going to talk about these cycles and basically avoiding creating them, using different algorithms and slightly different data structures.

Alright, let's write some code and create some cycles. Now we sort of saw that in the GC section before, but I want to start from scratch so we have some nice, clean examples to work with, and I'm gonna create another person class here that's similar, but not the same as what we had before. I'm just gonna paste this because we kind of did do this before. Over here, we've got our person class. It's created, it has a name. It also auto generates its id, and it has a list of friends. It has this class method that will just auto-increment the id, like 1, 2, 3 and so on, and then finally, this class will tell you if it was cleaned up. This will tell you that, yes, the cycle has been broken and the thing got cleaned up, or, you know what? It didn't. So, this is off to a good start. The other thing that I'm gonna create is something to play with it. I'll just call it "app_cycles" following my pattern that we're going to run things that are named "app" and just trigger so I can hit the hockey to make it run and do our fmain live template and boom, we've got something to start from. So what I wanna do is creates two people, have a person, and the name will be Michael and a second person whose name is Sarah. Okay, we got are two people here. Now I want to let you play with it in different ways. Sometimes they'll have cycles, sometimes they won't. So I'm gonna ask a question. I'll say if we'll ask the user "do you want cycles?" We'll do like a "[y/n]?" We'll say "If that is equal to yes, then we're going to create some friends" You know, we'll do that friend thing. So p1, that's Michael, append p2, and then we'll do the reverse like so. So if they say yes, that's going to create a cycle. And then we're going to zero them out here, like that and we'll print out "program closing, kthxbye". PyCharm thinks it's misspelled, but no, no, K, thanks, bye. That's a good thing. And let's just flush it so we can see stuff right away. Well, let's go ahead and run this and we'll see if we have the cycle, this we've kind of already looked at, but we're about to do something new here. So we're going to see that if we do create the cycle, this will not clean it up. So the goodbye people, they're deleted will happen after this. But if we say no, then they'll be cleaned up right here, Yeah? let's do that. Did we will create a cycle? Let's say no, and person was cleaned up. Michael and Sarah. K, thanks, bye. But this time, if we say yes, the garbage collector does not have enough container object allocations. Remember it needs 700 and we've done 2. So that's not enough, so it's not going to trigger any sort of GC. So the program exits and then basically as it goes out the door, it does its final cleanup here. Okay, so this is pretty interesting, but I'd like the program itself to know if the cycle is detected. Now, previously, we use that memutil thing that could tell us how many references were pointing at a thing. But I want to introduce you to another idea that we can play with. We could use the other one, but I think we'll learn something here as well. So I'm gonna create this thing called a "weak reference" and we can go and just say "we're gonna create a weakref" by importing the library and we'll say "refer to p1". What's gonna happen here is this is gonna retain something that maybe can point to the object. But it can only point the object if it hasn't been cleaned up. So previously we could say, Well, "there was this thing out in memory and it created a cycle and it didn't get cleaned up, but we only know that it used to be there". You can't say "let me access it as if it were still alive" or revive it In a sense, this weak reference will let us do that. So we'll do the same here, and down at the bottom, we'll do, I guess we'll say the things already closing or whatever, but then we'll say "if we want to check is this thing still around?" The way you do that with a weak reference is You invoke it and it'll return p1 if it can, if it's still around, otherwise it will return none. we'll say "if either these come back with an object, we'll print..." This, we'll say, if either of them are alive, we're gonna say "that's a cycle because nothing else points at it", and we'll let people know, otherwise, "no cycles found". So let's run it just one more time with our cool weak reference. So, no. So they were cleaned up when we set them to none. K, thanks, bye. No cycles found. Let's run this again. Yes. Create the cycle. Ah, cycles found, cycles found. And then those things got cleaned up. Here we are in this situation where we have these person classes, they have friends. Most importantly, we have these two things and they need to know about each other, right? Michael needs to know about Sarah. Sarah needs to know about Michael. You might think, Well, that's just the way it is. Is there any other possible data structure or mechanism we could use that's not going to create a cycle? Remember, these things are going to survive a while. If there's going many of them, they're going to get promoted into Gen 1 and then to Gen 2, and when those things get inspected, it's going to be expensive. They're gonna hang on around in memory longer than they otherwise should. remember, Gen 1 is 10 times less likely and Gen 2 is 100 times less likely to be even inspected. So avoiding creating the cycle in the first place may be a really good idea. Alright, So how do we do that? How do we create this? Well, the stage is set. In the next demo we're going to go and use a slightly different data structure that will have the same accomplishment. Like you could say, a friend of Michael is Sarah, and vice versa without creating cycles.

Alright, let's go and change this program just a little bit to work with our version that doesn't have cycles. I'm gonna have this "app_no_cycles", and let's even print it out. So we can see here, print "version with no cycles" this is the one that has data structures that use cycles. Okay, so what we're gonna do is we're gonna make a tiny difference here. Let's add one more person, a person 3, Zoe, and it'll still ask "do you want to have these people like this?" And we can go, we're gonna change this right here, "TODO: improve this with new data structures". Let's go ahead and just ask whether people are friends. And then down here, We'll just say "yes" like this, "Yes if p1 in p2.friends" for the moment. Again, we're going to be changing this. "else, no". like this. Alright, other than that, that's the same. And we can also ask this for "is p1 a friend of p3" just so that we have, you know, sort of both cases covered. Alright, let's run this thing and see how it's doing. Alright. Create cycles? Yes. Is Michael a friend of Sarah? Yes. Is Michael Friend Zoe? No. Program closing. Ah, cycles found. Okay, that's a problem. Let's just run it one more time with no cycles. No. No friends. We didn't create the friend relationships and no cycles and so on. We're also not zeroing out Zoe, so I guess we could go ahead and do that here as well, that way you'll see them all go together. Okay, Super. Now we've got the stage set. How are we gonna change this? How are we going to store them so they don't create these references back to each other and create these cycles and all the problems that can come from there? Well, we're gonna use a different data structure rather than storing them on the classes themselves, like we had indicated right there.

Alright, we're gonna bring some cool stuff together, and we're going to solve this problem by using a new data structure for our friends. Alright, so what I want to do is I want to create a data structure whose job is to manage the relationships. So that thing will exist and it'll hold on to say, p1 and p2, and it will know that we should point from p1 to 2 or 2 to 3 or 3 to 1, or whatever the relationship is. And then when we're done using them, we can say "we're done with this person", and it can drop all of those references. And actually, you'll see, It won't even really need to do that to work correctly. So I'm gonna call this data structure a "friend map". Now, at the heart of this is we're gonna have a thing I'm gonna call a "map", but notice that's a function. So we can call it "mp" or whatever, but I also want it to not actually be directly manipulatable from the outside. And so we can do that in Python by putting a double underscore and then it'll effectively be hidden. So this, I'm gonna have some help here, so this is gonna be a dictionary of, given an integer, this is the user id, it's gonna return a list, which is one of these, and same for dictionary. So given a person id, like 1, 7 or 500 or whatever, return me a list of their friends. And a dictionary would be fine, but I would like to also have the case when you ask for somebody who we've not entered or dealt with yet, to just say "no, no friends for that person". So we'll just say "defaultdict" rather than a regular dictionary. And it's going to be calling the list function when it finds a missing item. So, for example, nothing's added yet, right? If I asked for a map of 5, 6 or whatever it is, that's going to return an empty list and I could even append to that without crashing, without a key error or any of that kind of stuff. So default dicts are cool. This is hidden, so we're gonna write a few functions to deal with this. First of all, let's add a friend relationship here, so I'll say "def add_friend, and we'll have a person, a person like this. We'll go ahead and import them over there. We'll have a friend, which is also a person". Super. So we're gonna say "here's the person, here's their friend, set up their relationship so, you know, whoever that is, is the friend of this person". Pretty straightforward. But we want to do a little error checking, so if "not person or not friend" so you can't be a friend of none. You also, unless your little bit funky, you know, or maybe you always should be. Whatever, I don't know if you should be a friend of yourself. Right, you certainly don't need this thing to know whether you should be a friend of yourself. And then finally, we're gonna have a function down here "def is_friend", the exact same thing here that returns a bool. Alright, now return False. You've already set this relationship up, we don't need to do it as well. So "if is_friend (person, friend)" return. We don't need to enter them into this data structure twice. Alright, Now, here's where it gets interesting. So what we're gonna do is we're gonna say your current friends, the friends of person, not this one, but let's call it existing friends, existing friends or current friends, current. Current friends. So this relationship is stored over in the map, we'll go to the map and we'll get the friend id, so a person.id and remember if they've never set this up before, this is going to create an empty list for person.id and then we're gonna add them, or if it's already there, it's just going to return another list. Now, here's where it gets interesting. Do we just do this "append friend"? Well, if we do, we're kind of back in the same situation. Now, this thing, right, if we somehow forget to clean up the memory here, well, this thing is gonna hold our reference to friend, and it will never get cleaned up. That would be a bummer, right? So we're gonna do something a little bit different. We're going to go and add a weak reference. Remember, we just spoke about that, but we can use that here like this so we'll have the ability to get all the friends back for the friends who have not been cleaned up or deleted or vanished and the ones who have will be able to test "Oh, that person's already gone. They don't exist in the system anymore, so they're not really friends" alright? So what we can do is we can actually store this weak reference. But if you ask for the friends of somebody, yeah, you'll be able to just give it back. How cool is that? Okay, so this is going to let us add a friend and let's go and start by Fixing this here. We'll say "friend_map" and import that at the top. And we'll say, "add_friend", so p1 and p2 two and vice versa, like that. Now, down here, we have to change this. Instead of the p1 that's in here, we'll say "friend_map.is_friend" we're referencing p2 and the friend would be p1. So just like that. Now remember this one always returns false. We haven't finished this one. Yeah, p3. It goes like that. So, let's go ahead and run it and see what happens. Yes. Create some cycles. Are they friends? No. Because we said, just No. But notice there's no cycles found, okay? Well, that doesn't really prove anything yet. We haven't been able to accomplish the same relationships we wanted before, right? So, we're part way there. We're able to add these friends theoretically, and we haven't been able to test them, but we're getting there. So we're gonna have to write "is_friend", and it turns out that this function is not too bad.

Alright, well, let's actually implement this "is_friend" thing. So we actually want exactly the same tests here. So, if there's no person or no friend, there can't be a relationship between them, so we'll say "return False". If it's the same person, it's you, Let's say True, False, I don't know. Are you a friend of yourself or not? You can decide what the right metaphysical or philosophical answer is right there. But what we want to do is we actually want to say, "give us the friends of this person" and this is actually going to be a list. It's gonna come out like so, but not any old list. In fact, this is going to be a list of "weakref". So that's what we're storing. That's what we put in over here. So when we get it back, it's a list of weak references. These will not actually keep those friends alive, but we will be able to get them back if we need it. So we'll say something like this "for f in friends", like so, if I say f or ref, this is gonna be, we'll call it a reference, I don't know. A little bit complicated. This is a weak reference to the friend, not the friend themselves. If we can get a hold of them, we'll do it like this, we'll say, F is this And we'll say, "if there's a reference, gotta check that first, and that the id.." say this is a person, so we get auto complete here. So "this is an id which is equal to the friend.id". So the friend we're asking about is in the list of friends because it was added here. It's still around, right here. It's not cleaned up, so it still exists in the system, and it's actually the same person, right? So we're basically going through all the people, return True, return False. All right, there we go. Let's see if I got this right. First we want to create cycles. Yes. Is Michael a friend of Sarah? Oh, yes. Is Michael a friend of Zoe? No, they're not, they're not together. How cool is that? Pretty interesting. What we'll see is that these are not actually keeping those objects around, or this actually is not keeping around over there if we clean them up. Okay, so this is interesting. you might think. Well, "okay, Michael, this is way overkill with this weak reference thing because we're just saying True or False, all you have to store is the id of the friend and the id of the person and you're good". Yes, but in the previous example, I could actually get the friends. I could say "here is my list of friends for this person" check this out. So we can still do that without keeping these references, or these cycles, around. So we can say "get friends of person" and it's going to return a concrete list of person. So, these are gonna be real people, not weak reference type things. We'll say "friends is a list of weakref, just like before, equals