The line number calculation in libpython.py did not properly handle
negative (signed) line table deltas.

I think there is a race condition in runtest_mp where if a worker
has already pushed its final output to the queue, but is still
"alive", then the main thread waits forever on the the queue.
This might fix that issue, but there's still a delay of up
to ~30 secs.

See https://bugs.python.org/issue46205

This adds _PY_LIKELY, _PY_UNLIKELY, _Py_ALWAYS_INLINE, and macros to
test for SSE2 and NEON support.

Some of the subinterpreter tests run code using a PyThreadState from the
"wrong" native thread. This is going to be difficult to support and we
may want to instead create a "correct" PyThreadState for the active
native thread. For now, just disable the tests.

This adds a "status" field to each PyThreadState. The GC status will be
useful for implementing stop-the-world garbage collection.

This adds a recursive lock that will be used for locking I/O streams.
The lock performs a direct handoff if the waiting thread has been paused
for at least 1 ms. Otherwise, the lock supports barging.

The design is inspired by WTF locks (WebKit) and locking in Go.
See https://webkit.org/blog/6161/locking-in-webkit/

The "parking lot" implementation will likely need to be improved before
release. Currently, it uses a fixed-size hashtable (251 taken from Go)
and the same linked-list for waiters and collisions. Note that Go uses
a treap (random binary serach tree) for collisions, while WebKit resizes
the hashtable to ensure 3x as many buckets as threads.

Critical sections are helpers to replace the global interpreter lock
with finer grained locking. They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves. Critical
sections are implicitly ended whenever the GIL would be released. They
are resumed when the GIL would be acquired. Nested critical sections
behave as-if they're interleaved.

This changes SOABI and sys.implementation.name to "nogil" to avoid
installing incompatible binary wheels. It also defines the macros
Py_NOGIL and PY_NOGIL in patchlevel.h to identify nogil builds.

The GIL is controlled by the environment variable PYTHONGIL or the flag
"-X nogil". The GIL is enabled by default. The interpreter will
currently crash when running multi-threaded programs without the GIL.

The eval_breaker variable is used as a signal to break out of the
interpreter loop to handle signals, asynchronous exceptions, stop for
GC, or release the GIL to let another thread run.

We will be able to have multiple active threads running the interpreter
loop so it's useful to move eval_breaker to per-thread state so that
notifications can target a specific thread.

The specific signals are combined as bits in eval_breaker to simplify
atomic updates.

There is a deadlock hazard when a thread acquires a lock in a finalizer
that may also be held during a stop-the-world GC. Typical GC
implementations avoid this by running finalizers in separate thread
while other threads are resumed. This is awkward in Python because
some code and tests expect finalizers to have finished by the time
gc.collect() finishes.

In CPython with the GIL the issue can be mitigated with a RLock because
the GC doesn't actually stop other threads -- other threads can acquire
the GIL and run during many parts of a GC cycle.

For ease of implementation, we add a private API _thread.CriticalLock
which disables the current thread from parking while the lock is held.
This prevents deadlock by ensuring that the lock is released before
the GC is run.

This uses _thread.CriticalLock to prevent garbage collection while
modifying the finalizer registry and in the resource tracker.

There is a potential deadlock when checking if the resource tracker is
running. This exists in upstream CPython, but occurs much more frequently
with biased reference counting. The call to _check_alive can trigger a
garbage collection that collects a multiprocessing lock. The finalizer
for that lock calls back into resource tracker leading to a deadlock.

This is rare in upstream CPython because most function calls do not
trigger a garbage collection.

When Py_REF_DEBUG is defined, Python aggregates the number of
refcounting changes. This adds a per-thread counter and only aggregates
across threads when _Py_GetRefTotal() is called. Threads also add their
local counters to the global counter when they exit.
counter on exit.

pyperformance: 9.96% regression

About 3% points of the slow-down appears to be in the
_Py_MergeZeroRefcount function. If this is rewritten to always avoid
atomic operations than the regression is only 6.7%.

We should be able to use the fast-path merge & dealloc for more types
once we start using mimalloc and remove object caches.

The scheme will be used to allow safe reads from dicts and lists that
don't acquire the collection's lock.

pyperformance: 13.9% regression

WrapperTest.test_create_at_shutdown_with_encoding fails with something
like:

```
Exception ignored in: <function C.__del__ at 0x101895ae0>
Traceback (most recent call last):
  File "x.py", line 10, in __del__
  File "/Users/sgross/Projects/nogil/Lib/_pyio.py", line 2030, in __init__
  File "/Users/sgross/Projects/nogil/Lib/_pyio.py", line 1021, in seekable
ValueError: I/O operation on closed file.
```

The problem is that the GC calls the finalizer on `self.buf` before the
`__del__` on `class C`. I don't think Python guarantees any ordering on
destructors.

Modified src/alloc.c to remove include of alloc-override.c
Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c

…ypes

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

These are changes to support separate heaps for Python objects,
Python objects with GC header, and non Python objects.

Python threads have complicated lifecycles. After fork(), only one
thread remains alive. In that case, we want to abandon the heaps of the
threads that didn't survive fork() so that the we can still find all GC
objects after we delete those dead PyThreadState.

There's a similar issue with daemon threads. Python deletes the
PyThreadState for daemon threads potentially before (or concurrently)
with thread exit.

The optimistic lock-free reads will require preserving some data across
malloc/free calls. For refcounted Python objects, the refcount field
must remain valid. For dict keys the entire object must remain valid,
other than the first word (dk_usable).

This adds a field debug_offset that is the offset from the start of
the block where the allocator is allowed to fill dead objects with
0xDD or 0xD0. The value -1 signifies that the entire block must remain
valid, other than the first word, which is used to store the free list.

This changes mimalloc to tag pages for PyObjects with the current qsbr
"goal". These pages can't be freed or used in a different heap until
after the qsbr shared read counter reaches the goal.

This is going to be important once the GIL is removed for a few reasons:

 - We won't be maintaining the GC linked list. Scanning the heap is
   O(N), best to do it in proprotion to the number of live objects.

 - Frequent stop-the-world pauses will become a bottleneck in
   multi-threaded programs

 - Python programs don't realy adhere to the generational hypothesis,
   so there isn't much benefit to a generational collector. Most objects
   seen by the GC *don't* die young! (Most survive the GC cycles.)

The GC now uses separate mimalloc heaps for GC and non-GC objects and
only maintains gc lists during garbage collection.

 - PyGC_Head._gc_prev is now the first word in the object. This ensures
   that a deallocated memory block does not look like a tracked object.
   The first word is used for _gc_prev when allocated and for the
   free-list when not allocated. The free-list never has the
   least-significant bit (_PyGC_PREV_MASK_TRACKED) set because
   objects are naturally aligned.

Also reduce GC frequency in "collect_in_thread" to improve test_weakref
speed when running without the GIL.

This adds a mutex around pow5mult to protect the global "p5s" cache of
powers of 5. This also removes the non-thread-safe memory pools.

 - _PyUnicode_FromId is now thread-safe
 - static strings are initialized at Python start-up and are immortal
 - adds a mutex to protect accesses to the "interned" dict

Previously, _thread.lock would modify a shared boolean "locked" after
the underlying lock was released. This swaps the order of those
statements.

The static cache isn't thread-safe.

The cache doesn't even seem to be used frequently (if ever). Typically,
resolve_slotdups is called with different names so the cache seems to
be usually invalid.

This replaces the _tstate_lock used for Thread.join() with an event
object implemented in C. This avoids calls into the Python API
after the thread state is deleted, which had caused reference counting
race conditions.

This uses _PyMutex and the PyParkingLot functions to implement a
thread-safe unbouneded MPMC queue.

This is the first step in removing the C implementation of OrderedDict.

Rely on the ordering of the dict superclass. This fixes some of the
direct uses of dict methods on OrderedDict. Previously, these would
raise KeyError later on. Now they work.

There are still a few missing features:

 - No support for the more memory efficient split tables

 - Iterators hold onto dicts until the iterator is freed (instead of
   until the iterator is exhausted)

 - TODO: make the thread-safety scheme more similar to dict. Move the
   allocated field to the backing array and add a version counter.

 - TODO: use correct memory ordering for list assignment

The "accumulator" data structure is used for JSON encoding and maintains
an internal list. We don't need to use a lock for modifications to this
list because it's only accessed by the thread that created it.

 - Use atomics to increment max_module_number
 - TODO: (Fix data race in imports)
 - ???

 - Make rlock thread-safe
 - Use atomics to modify interp->num_threads

The fix relies on the internal locking of PyDictObject. It
would be nice, in the future, to use an external lock or
move the store task to the loop object.

 - Use PyDict_GetItemWithError2 in _PyObject_GenericGetAttrWithDict to
   get the attribute as a new reference.
 - Use PyDict_GetItemWithError2 in _PyObject_GetMethod

This adds a recursive mutex to each LRU cache. The mutex is held
while the cache is updated, but released when calling the annotated
function. Note that this still potentially can lead to deadlocks that
did not exist with the GIL, but I think only in rather esoteric cases.

This adds support for protecting functions wrapped by argument clinic
with recursive mutexes.

The co_zombieframe is expected to be NULL when running without the GIL.

The test_threads function checks that at least one of the running
threads is blocked on the GIL and that the behavior is displayed in the
GDB backtrace. Of course, this is no longer the case when running
without the GIL.

This adds a mutex to every deque object. It's acquired around
most operations. Some remaining functions, which are not yet
thread-safe are:

 - deque.copy
 - deque.__init__

PyImport_ImportModuleLevelObject previously used the variable
module.__spec__._initializing to check if a module is currently being
initialized. This access could race with modifications to the module's
dict.

Instead, use the new md_initialized field on PyModule to check if the
module is already initialized. This can mean that a duck-typed module
doesn't get the benefit of the fast-path, but I think other than that
the change should still preserve the important behavior.

This fixes a hang in test_multiprocessing when running without the GIL
due to lost modifications to count in the seamphore.

The module name is copied to "__module__" attribute of functions,
including closures. Make the name interned (and immortalized) to avoid
bottlenecks on reference counting it.

The tracing references feature is not thread-safe without the GIL. I
think it's possible to get the same functionality in a thread-safe way
by traversing the mimalloc heaps (like we do in the garbage collector),
but that's not currently implemented.

For now the "--with-trace-refs" argument is ignored by configure with a
warning about unrecognized options.

Fixes colesbury#3

Heap type objects are a common source of reference count contention
because every created instance increments the reference count of its
type. The existing deferred reference count scheme isn't sufficient
because instances are not necessarily visible to the garbage collector.

This stores most reference count modifications in a per-thread array
with an entry for each heap type object. Each heap type object has a
new field 'tp_typeid' which is used to index the thread-local array.
Thread-local reference counts are merged into the type's own reference
count field during garbage collection and before the thread exits.

Changes "test_cancel_futures_wait_false" to pause a tiny bit before
calling "shutdown" with cancel_futures=True. Otherwise, the work item
may be cancelled before it is started. In practice, this only happens
when the GIL is disabled. With the GIL on, the thread grabs the work
item before t.shutdown() is called due to the 5 ms GIL switch interval.

The importlib frozen module is regenerated because of the switch of some
strings to interned.

The modified pip package looks in a custom package index URL for
compatible nogil Python packages.

See https://github.com/colesbury/pip/tree/21.3.1-nogil for modifications