JuliaCollections / JuliaCollections/DataStructures.jl
Priority Queue implemented as Bucket Queue for Integer weights (4x speed)
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 745
- Forks
- 261
- PR merge metrics
- No merged PRs in 30d
Description
I suggest adding the bucket queue algorithm, which is very fast for small integer weights.
A lot of problems don't require a perfect 'float' priority queue but can use a rounded integer version that is multiple times faster by using a fixed range of possible integer weights.
For my application its runtime is 4s compared to 28s of the standard PriorityQueue{Int, Int}.
I would create a pull request, but I'm not sure how to best integrate it in the existing PriorityQueue. The underlying data structure and algorithm is different, so the whole struct would need a second version, I guess.
My code for the bucket queue is very short:
mutable struct PQueue{T}
min::Int
nbins::Int
content::Vector{Vector{T}}
end
# initialize the buckets
PQueue{T}(nbins) where T = PQueue(nbins + 1, nbins, [Vector{T}() for _ in 1:nbins])
Base.isempty(q::PQueue) = q.min > q.nbins
function Base.push!(q::PQueue, item, weight)
push!(q.content[weight], item)
q.min = min(q.min, weight)
return q
end
function Base.pop!(q::PQueue)
elem = pop!(q.content[q.min])
# increase smallestbin, if elem was last in bin
while q.min ≤ q.nbins && isempty(q.content[q.min])
q.min += 1
end
return elem
end
And a script showing a 4x higher speed in a use case, where the queue is growing:
using DataStructures
include("priorityqueue.jl")
nbins = 255 # small number
ndata = 10000000
randdata = rand(1:nbins, ndata)
function pq_datastructures()
pq = PriorityQueue{Int, Int}()
for i in 1:ndata
enqueue!(pq, i, randdata[i])
if iseven(i)
dequeue!(pq)
end
end
end
function pq_bucketqueue()
pq = PQueue{Int}(nbins)
for i in 1:ndata
push!(pq, i, randdata[i])
if iseven(i)
pop!(pq)
end
end
end
pq_datastructures()
pq_bucketqueue()
@time pq_datastructures()
@time pq_datastructures()
@time pq_bucketqueue()
@time pq_bucketqueue()
# output
6.768914 seconds (30.00 M allocations: 920.507 MiB, 4.74% gc time)
6.738860 seconds (30.00 M allocations: 920.507 MiB, 5.00% gc time)
1.551067 seconds (35.00 M allocations: 814.256 MiB, 13.11% gc time)
1.496581 seconds (35.00 M allocations: 814.256 MiB, 13.03% gc time)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the proposed PQueue implementation with the existing PriorityQueue API and the included benchmark script. Determine whether the bucket queue should be a separate type or integrated as a second implementation, then benchmark the chosen design against PriorityQueue for bounded integer weights. Done means the queue supports the intended operations and demonstrates the reported performance improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100