Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop Python 2.7 support #2480

Closed
giampaolo opened this issue Dec 19, 2024 · 1 comment · Fixed by #2481
Closed

Drop Python 2.7 support #2480

giampaolo opened this issue Dec 19, 2024 · 1 comment · Fixed by #2481

Comments

@giampaolo
Copy link
Owner

giampaolo commented Dec 19, 2024

About dropping Python 2.7 support, 3 years ago I stated:

[...] not a chance, for many years to come. [Python 2.7] currently represents 7-10% of total downloads, aka around 70k /100k downloads per day

3 years later (and to my surprise) downloads for Python 2.7 dropped to 0.36%. These are downloads per month:

$ pypinfo --percent psutil pyversion
Served from cache: False
Data processed: 4.65 GiB
Data billed: 4.65 GiB
Estimated cost: $0.03

| python_version | percent | download_count |
| -------------- | ------- | -------------- |
| 3.10           |  23.84% |     26,354,506 |
| 3.8            |  18.87% |     20,862,015 |
| 3.7            |  17.38% |     19,217,960 |
| 3.9            |  17.00% |     18,798,843 |
| 3.11           |  13.63% |     15,066,706 |
| 3.12           |   7.01% |      7,754,751 |
| 3.13           |   1.15% |      1,267,008 |
| 3.6            |   0.73% |        803,189 |
| 2.7            |   0.36% |        402,111 |
| 3.5            |   0.03% |         28,656 |
| Total          |         |    110,555,745 |

According to pypistats.org it's 0.28% of the total, and around 15.000 downloads per day.

Maintaining 2.7 support has become increasingly difficult, but still possible. E.g. we can still run tests by using old PYPI backports. GitHub Actions can still be tweaked to run tests and produce wheels on Linux and macOS. Not Windows though, for which we have to use a separate service (Appveyor).

Still, the amount of hacks in psutil source code necessary to support Python 2.7 piled up over the years, and became quite big. Some disadvantages that come to mind:

  • (high) having to maintain various python compatibility layers (e.g. psutil/_compat.py) + all the compromises that come with it (extra imports, extra code, str vs. unicode differences, etc.)
  • (medium) having to maintain a C compatibility layer (#if PY_MAJOR_VERSION <= 3, etc.)
  • (medium) inability to use modern language features, especially f-strings
  • (low) inability to freely use enums, which creates a difference on how CONSTANTS are exposed in terms of API
  • (medium) having to install a specific version of pip and other (outdated) deps
  • (high) relying on third-party Appveyor CI service, just to run tests on python 2.7 and produce wheels, when we could rely on a single CI service instead (GitHub)
    • (high) soon I want to distribute wheels via GitHub instead of manually via twine, so that'll be a problem (CC @potiuk)
  • (high) gradual lack of support from third-party libraries and services
  • (medium) 4 extra CI jobs which are run on every commit (Linux, macOS, Windows 32-bit, Windows 64-bit)
  • (medium) the distribution of 7 wheels specific for Python 2.7. From last release:
    • psutil-6.1.1-cp27-cp27m-macosx_10_9_x86_64.whl
    • psutil-6.1.1-cp27-none-win32.whl
    • psutil-6.1.1-cp27-none-win_amd64.whl
    • psutil-6.1.1-cp27-cp27m-manylinux2010_i686.whl
    • psutil-6.1.1-cp27-cp27m-manylinux2010_x86_64.whl
    • psutil-6.1.1-cp27-cp27mu-manylinux2010_i686.whl
    • psutil-6.1.1-cp27-cp27mu-manylinux2010_x86_64.whl
  • etc.

As such I decided to finally drop support for Python 2.7. Current psutil 6.1.1 release will still support Python 2.7, but next 7.0.0 will not.

We can still make a promise that the 6.1.* line (EDIT: see python2 branch) will keep supporting Python 2.7 and will receive critical bug-fixes only (no new features).

In 7.0.0 we can keep the setup.py script compatible with Python 2.7 in terms of syntax, so that it can emit an informative error message on pip install. E.g. the user will see something like this:

$ pip2 install psutil
As of version 7.0.0 psutil no longer supports Python 2.7. 
Latest version supporting Python 2.7 is psutil 6.1.X.  
Install it with: "pip2 install psutil==6.1.*".

Related tickets:

@potiuk
Copy link

potiuk commented Dec 20, 2024

Good call @giampaolo. And really cool to see the stats of 2.7 dropping to 0.36% :).

I think pretty much all libraries (even those that for good reasons kept Python 2.7 compatibility) is one of the last things to finally announce Python 2.7 a "dead snake". There ae still some industries that have some 2.7 codebase (anecdotally gaming industry is apparently one of them to my surprise), but it only makes sense to shift the burden of maintaining the code to them rather than having maintainers of the open-source projects to do it. If anyone wants still to use Python 2.7 and use psutil - even if they will need new features, it should be on them to backport anything they need - not you.

This was referenced Dec 20, 2024
giampaolo added a commit that referenced this issue Dec 21, 2024
As part of the dropping of Python 2.7 support (#2480), we can now take advantage of chained exceptions machinery (`raise x from y` and `raise x from None`). In practical terms, this is what changes:

# Shorter tracebacks

When adding the full traceback info adds no value, we now shorten tracebacks if `raise X from None`.
A similar (hackish) attempt was made in 633d8019, when we were still stuck with Python 2. The notable example is passing a PID that does not exist to the `Process` class:

```python
psutil.Process(333)
```

Before we got:

```
Traceback (most recent call last):
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1647, in wrapper
    return fun(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 464, in wrapper
    raise err from None
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 462, in wrapper
    return fun(self)
           ^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1713, in _parse_stat_file
    data = bcat(f"{self._procfs_path}/{self.pid}/stat")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 794, in bcat
    return cat(fname, fallback=fallback, _open=open_binary)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 782, in cat
    with _open(fname) as f:
         ^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 746, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/341244/stat'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 350, in _init
    self._ident = self._get_ident()
                  ^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 391, in _get_ident
    return (self.pid, self.create_time())
                      ^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 773, in create_time
    self._create_time = self._proc.create_time()
                        ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1647, in wrapper
    return fun(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1885, in create_time
    ctime = float(self._parse_stat_file()['create_time'])
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1659, in wrapper
    raise NoSuchProcess(pid, name) from err
psutil.NoSuchProcess: process no longer exists (pid=341244)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/giampaolo/svn/psutil/foo.py", line 5, in <module>
    psutil.Process(341244)
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 317, in __init__
    self._init(pid)
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 363, in _init
    raise NoSuchProcess(pid, msg=msg)
psutil.NoSuchProcess: process PID not found (pid=341244)
```

Now we get:

```
Traceback (most recent call last):
  File "/home/giampaolo/svn/psutil/foo.py", line 5, in <module>
    psutil.Process(341244)
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 317, in __init__
    self._init(pid)
  File "/home/giampaolo/svn/psutil/psutil/__init__.py", line 363, in _init
    raise NoSuchProcess(pid, msg=msg) from None
psutil.NoSuchProcess: process PID not found (pid=341244)
```

# Different wording for "translated" exceptions

By "translated" I mean psutil's `NoSuchProcess`, `ZombieProcess` and `AccessDenied`.
Given the following code:

```python
import psutil
from psutil.tests import spawn_testproc

sproc = spawn_testproc()
p = psutil.Process(sproc.pid)
p.terminate()
p.wait()
p.name()
```

Before we got:

```
Traceback (most recent call last):
  [...]
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 746, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/105496/stat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  [...]
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1659, in wrapper
    raise NoSuchProcess(pid, name)
psutil.NoSuchProcess: process no longer exists (pid=105496)
```

Now we get:

```
Traceback (most recent call last):
  [...]
  File "/home/giampaolo/svn/psutil/psutil/_common.py", line 746, in open_binary
    return open(fname, "rb", buffering=FILE_READ_BUFFER_SIZE)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/proc/105496/stat'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...]
  File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1659, in wrapper
    raise NoSuchProcess(pid, name) from None
psutil.NoSuchProcess: process no longer exists (pid=105496)
```

Diff:

```diff
     FileNotFoundError: [Errno 2] No such file or directory: '/proc/105496/stat'
 
-    During handling of the above exception, another exception occurred:
+    The above exception was the direct cause of the following exception:
 
     Traceback (most recent call last):
       [...]
       File "/home/giampaolo/svn/psutil/psutil/_pslinux.py", line 1659, in wrapper
-        raise NoSuchProcess(pid, name)
+        raise NoSuchProcess(pid, name) from None
     psutil.NoSuchProcess: process no longer exists (pid=105496)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants