Better exclude_address function in ipaddress.py

Open
#97,610 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
networking

Research direction

Start in ipaddress.py at address_exclude and read the existing subnets() and summarize_address_range() paths. Compare the proposed integer-range approach with the current behavior, preserving the demonstrated network results while addressing the reported performance problem.

Written by the indexing model from the issue text.

Description

stdlib type-feature

Feature or enhancement

Increase the speed of exclude_address by treating IPs as integer ranges making exclude_address O(1).

Pitch

Currently exclude_address will take a subnet and split it by calling subnets()

import ipaddress
>>> ip = ipaddress.IPv4Network('10.0.0.0/8')
>>> [i for i in ip.subnets()]
[IPv4Network('10.0.0.0/9'), IPv4Network('10.128.0.0/9')]

each time it is split, it checks which one contains the subnet being excluded and splits again.
This ends up being a lot of work.

Instead I propose we can use the fact that IPs are represented as integers behind the scene

For a given subnet, it can be thought of as an integer range between its network and broadcast address

>>> ip.network_address._ip
167772160
>>> ip.broadcast_address._ip
184549375

Given these can be treated as an integer range, excluding is just removing the range integers you wish to exclude

>>> [i for i in ipaddress.summarize_address_range(
    ipaddress.IPv4Address(
        ip.network_address._ip),
        ipaddress.IPv4Address(exclude_ip.network_address._ip-1
))]
[IPv4Network('10.0.0.0/22')]

>>> [i for i in ipaddress.summarize_address_range(
    ipaddress.IPv4Address(
        exclude_ip.broadcast_address._ip)+1,
        ipaddress.IPv4Address(ip.broadcast_address._ip
))]
[IPv4Network('10.0.5.0/24'), IPv4Network('10.0.6.0/23'), IPv4Network('10.0.8.0/21'), IPv4Network('10.0.16.0/20'), IPv4Network('10.0.32.0/19'), IPv4Network('10.0.64.0/18'), IPv4Network('10.0.128.0/17'), IPv4Network('10.1.0.0/16'), IPv4Network('10.2.0.0/15'), IPv4Network('10.4.0.0/14'), IPv4Network('10.8.0.0/13'), IPv4Network('10.16.0.0/12'), IPv4Network('10.32.0.0/11'), IPv4Network('10.64.0.0/10'), IPv4Network('10.128.0.0/9')]

>>> sorted([i for i in ip.address_exclude(
    ipaddress.IPv4Network('10.0.4.0/24'
))])
[IPv4Network('10.0.0.0/22'), IPv4Network('10.0.5.0/24'), IPv4Network('10.0.6.0/23'), IPv4Network('10.0.8.0/21'), IPv4Network('10.0.16.0/20'), IPv4Network('10.0.32.0/19'), IPv4Network('10.0.64.0/18'), IPv4Network('10.0.128.0/17'), IPv4Network('10.1.0.0/16'), IPv4Network('10.2.0.0/15'), IPv4Network('10.4.0.0/14'), IPv4Network('10.8.0.0/13'), IPv4Network('10.16.0.0/12'), IPv4Network('10.32.0.0/11'), IPv4Network('10.64.0.0/10'), IPv4Network('10.128.0.0/9')]

Previous discussion

https://discuss.python.org/t/ipaddress-py-exclude-address-speed-up/19445

Dominant language
Python
Stars
77.2k
Forks
36k
Avg merge
1d 9h
Merged PRs (30d)
558

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python/cpython

All issues in python/cpython

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.