how to handle large context by Machine Comprehension(bi-att-flow) &Resource Exhausted error
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 667
- PR merge metrics
- No merged PRs in 30d
Description
I have a test file, whose name is mytest1.json. The context is a large text, which has so many words in it.
when i run the folowing:
basic/run_single.sh $HOME/data/squad/mytest1.json single.json
some errors happen, do you guys have an idea about how to solve this... Thanks so much.
`File "/home/weijiang/bi-att-flow/inference/main.py", line 29, in main
eval_data = _forward(config, data, shared)
File "/home/weijiang/bi-att-flow/inference/main.py", line 88, in _forward
models = get_multi_gpu_models(config)
File "/home/weijiang/bi-att-flow/inference/model.py", line 19, in get_multi_gpu_models
model = Model(config, scope, rep=gpu_idx == 0)
File "/home/weijiang/bi-att-flow/inference/model.py", line 58, in __init__
self._build_forward()
File "/home/weijiang/bi-att-flow/inference/model.py", line 164, in _build_forward
p0 = attention_layer(config, self.is_train, h, u, h_mask=self.x_mask, u_mask=self.q_mask, scope="p0", tensor_dict=self.tensor_dict)
File "/home/weijiang/bi-att-flow/inference/model.py", line 421, in attention_layer
u_a, h_a = bi_attention(config, is_train, h, u, h_mask=h_mask, u_mask=u_mask, tensor_dict=tensor_dict)
File "/home/weijiang/bi-att-flow/inference/model.py", line 398, in bi_attention
is_train=is_train, func=config.logit_func, scope='u_logits') # [N, M, JX, JQ]
File "/home/weijiang/bi-att-flow/my/tensorflow/nn.py", line 127, in get_logits
new_arg = args[0] * args[1]
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 751, in binary_op_wrapper
return func(x, y, name=name)
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 910, in _mul_dispatch
return gen_math_ops.mul(x, y, name=name)
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1519, in mul
result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
op_def=op_def)
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2380, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/weijiang/anaconda2/envs/tensorflow-0.11-py3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1298, in __init__
self._traceback = _extract_stack()
ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1,1,34680,7,200]
[[Node: model_0/main/p0/bi_attention/mul = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"](model_0/main/p0/bi_attention/Tile, model_0/main/p0/bi_attention/Tile_1)]]
[[Node: model_0/main/g2/BW/BW/Assert/AssertGuard/Assert/Switch/_333 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_4307_model_0/main/g2/BW/BW/Assert/AssertGuard/Assert/Switch", tensor_type=DT_BOOL, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.