dmlc / dmlc/MXNet.jl

BoundsError if the batch size is smaller than the dimension of the output

Open
#264 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
No language data
Stars
369
Forks
63
PR merge metrics
No merged PRs in 30d

Description

Originally filed here: https://stackoverflow.com/questions/45406537/boundserror-in-julia-mxnet-when-using-small-batch-size (reported by: Robert Crandall)

quote:
'm trying to reproduce some Python MXNet code in Julia 0.6.0, and I'm getting a BoundsError if I try to use a batch size that is smaller than the dimension of the output. If I use a larger batch size in a toy example, things work properly and the network converges to the correct solution, but in my application the output dimension is large so this isn't practical.
Here's a linear regression example that gives this error:
```julia
using MXNet
net = mx.Variable(:data)
net = mx.FullyConnected(net, name=:fc0, num_hidden=5)
net = mx.LinearRegressionOutput(net, name=:output)
mod = mx.FeedForward(net, context=mx.cpu(0))

batch_size = 4 # works for batch_size > 4
A = randn(5,100)
train_in = randn(100,1000)
train_out = A*train_in + .1*randn(5,1000)

train_provider = mx.ArrayDataProvider(:data=>train_in,
:output_label=>train_out,
shuffle=true,
batch_size=batch_size)

optimizer = mx.SGD(lr=0.001, momentum=0.9, weight_decay=0.00001)
mx.fit(mod, optimizer, train_provider)
```

This produces

> INFO: Start training on MXNet.mx.Context[CPU0]
> INFO: Initializing parameters...
> INFO: Creating KVStore...
> INFO: TempSpace: Total 0 MB allocated on CPU0
> INFO: Start training...
> ERROR: LoadError: BoundsError: attempt to access 5×4 Array{Float32,2} at index [Base.Slice(Base.OneTo(5)), 5]

Update from Viacheslav Kovalevskyi:
When I try to reproduce the bug on the master I got:

> julia> mx.fit(mod, optimizer, train_provider)
> INFO: Start training on MXNet.mx.Context[CPU0]
> INFO: Initializing parameters...
> INFO: Creating KVStore...
> INFO: TempSpace: Total 0 MB allocated on CPU0
> INFO: Start training...
> ERROR: BoundsError: attempt to access 5_4 Array{Float32,2} at index [Base.Slice(Base.OneTo(5)), 5]
> Stacktrace:
> [1] throw_boundserror(::Array{Float32,2}, ::Tuple{Base.Slice{Base.OneTo{Int64}},Int64}) at ./abstractarray.jl:433
> [2] checkbounds at ./abstractarray.jl:362 [inlined]
> [3] view at ./subarray.jl:113 [inlined]
> [4] _update_single_output(::MXNet.mx.Accuracy, ::Array{Float32,2}, ::Array{Float32,2}) at /home/ubuntu/.julia/v0.6/MXNet/src/metric.jl:211
> [5] macro expansion at /home/ubuntu/.julia/v0.6/MXNet/src/metric.jl:58 [inlined]
> [6] macro expansion at /home/ubuntu/.julia/v0.6/MXNet/src/ndarray.jl:783 [inlined]
> [7] _update!(::MXNet.mx.Accuracy, ::Array{MXNet.mx.NDArray,1}, ::Array{MXNet.mx.NDArray,1}, ::Val{false}) at /home/ubuntu/.julia/v0.6/MXNet/src/metric.jl:55
> [8] update!(::MXNet.mx.Accuracy, ::Array{MXNet.mx.NDArray,1}, ::Array{MXNet.mx.NDArray,1}) at /home/ubuntu/.julia/v0.6/MXNet/src/metric.jl:34
> [9] #fit#8303(::Array{Any,1}, ::Function, ::MXNet.mx.FeedForward, ::MXNet.mx.SGD, ::MXNet.mx.ArrayDataProvider) at /home/ubuntu/.julia/v0.6/MXNet/src/model.jl:524
> [10] fit(::MXNet.mx.FeedForward, ::MXNet.mx.SGD, ::MXNet.mx.ArrayDataProvider) at /home/ubuntu/.julia/v0.6/MXNet/src/model.jl:351

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at src/metric.jl, especially _update_single_output, which the stack trace identifies as the failing entry point during mx.fit. Reproduce the example with batch_size=4 and verify that training completes without a BoundsError when the output dimension exceeds the batch size.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.