So, it’s really been a long time since I last wrote something. A lot has happened in the intervening time and I currently don’t have the luxury of time to dwell on my thoughts much. So, I will keep it brief and defer a lengthy post, sort of a catharsis, for a later time. So first things first. I have been learning Python and I am loving it. I rue why I didn’t encounter it earlier. Anyway since I am learning a great many concepts each day, I thought I better make little notes of the things I learn and save the links to the resources for reference in the future. So here are some tidbits I have been learning over the past few days:
1) GIL – Global Interpreter Lock. It’s a feature of the CPython implementation of Python – not a part of the core language itself. Other implementations like Jython, IronPython don’t have GIL. The CPython interpreter is not fully thread safe and therefore it employs a mechanism so that only one thread at a time can have access to the Python interpreter in order to support multi-threaded Python programs. GIL is a big mutex-type global lock (to protect reference counters) that must be held by the current thread before it can safely access Python objects. This ensures that only one thread executes Python bytecode at a time.
However, it is a coarse-grained locking at the interpreter level (as opposed to a fine-grained locking where every separate structure has its own lock at the application level) sufficient only to keep Python’s own structures (the CPython interpreter) in a consistent state. In short, the GIL only just protects the interpreter internally.
In order to support multi-threaded Python programs, the interpreter regularly releases and reacquires the lock – by default, every 10 bytecode instructions (this can be changed with sys.setcheckinterval()). The lock is also released and reacquired around potentially blocking I/O operations like reading or writing a file, so that other threads can run while the thread that requests the I/O is waiting for the I/O operation to complete.
Since only one Python thread can execute at a time, the multithreading is due to time-division multiplexing and in mutli-core systems, only one core will be executing instructions while others will be idle. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.
Due to tradeoffs between fine-grained and coarse-grained locking, it is difficult to remove GIL without affecting the performance of the interpreter for single-threaded Python programs. It’s because there’s more book-keeping to do when locking is done at a finer level and as a result single-threaded programs run slower. However, fine-grained locking allows greater parallelism – two threads can execute in parallel when they don’t share any resources. However there is a much larger administrative overhead. For every line of code, you may need to acquire and release several locks.
Some links for further details: