DeepLab with TensorFlow Mobile or TensorFlow Lite

Hello @aquariusjay,

We just want to run this modal on Android. We have tried two approach TensorFlow Mobile and TensorFlow Lite.

With TensorFlow Mobile, we download the pre-trained modals with MobileNetV2:
mobilenetv2_coco_voc_trainaug
mobilenetv2_coco_voc_trainval
mobilenetv2_coco_cityscapes_trainfine

We can successfully load the modal, but when run the inference, we get the following error:

05-16 16:06:34.122 611-635/? E/AndroidRuntime: FATAL EXCEPTION: AsyncTask #1
    Process: com.orange.labs.colorme.dev, PID: 611
    java.lang.RuntimeException: An error occurred while executing doInBackground()
        at android.os.AsyncTask$3.done(AsyncTask.java:325)
        at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:354)
        at java.util.concurrent.FutureTask.setException(FutureTask.java:223)
        at java.util.concurrent.FutureTask.run(FutureTask.java:242)
        at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
        at java.lang.Thread.run(Thread.java:761)
     Caused by: java.lang.IllegalArgumentException: No OpKernel was registered to support Op 'Slice' with these attrs.  Registered devices: [CPU], Registered kernels:
      device='CPU'; T in [DT_BOOL]
      device='CPU'; T in [DT_FLOAT]
      device='CPU'; T in [DT_INT32]
    
    	 [[Node: SemanticPredictions = Slice[Index=DT_INT32, T=DT_INT64](ArgMax, SemanticPredictions/begin, SemanticPredictions/size)]]
        at org.tensorflow.Session.run(Native Method)
        at org.tensorflow.Session.access$100(Session.java:48)
        at org.tensorflow.Session$Runner.runHelper(Session.java:298)
        at org.tensorflow.Session$Runner.runAndFetchMetadata(Session.java:260)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.run(TensorFlowInferenceInterface.java:220)
        at org.tensorflow.contrib.android.TensorFlowInferenceInterface.run(TensorFlowInferenceInterface.java:197)
        at com.dailystudio.deeplab.DeeplabV3.segment(DeeplabV3.java:104)
        at com.dailystudio.deeplab.DeeplabApplication$1.doInBackground(DeeplabApplication.java:46)
        at com.dailystudio.deeplab.DeeplabApplication$1.doInBackground(DeeplabApplication.java:22)
        at android.os.AsyncTask$2.call(AsyncTask.java:305)
        at java.util.concurrent.FutureTask.run(FutureTask.java:237)

I think this is caused by the output node "SemanticPredictions" call the operation Slice with INT64 data. This is not supported by TensorFlow Mobile yet.

With TensorFlow Lite, we use the following command to convert is to tflite format:

toco \
	--input_file=$(pwd)/model/frozen_inference_graph.pb \
	--input_format=TENSORFLOW_GRAPHDEF \
	--output_format=TFLITE \
	--output_file=$(pwd)/model/deeplabv3_mnv2_pascal_trainval.tflite \
	--inference_type=FLOAT \
	--input_type=QUANTIZED_UINT8 \
	--input_arrays=ImageTensor \
	--output_arrays=SemanticPredictions \
	--input_shapes=1,513,513,3 \
	--default_ranges_min=0 --default_ranges_max=255

We get the following warnings:

2018-05-16 16:19:29.429205: W tensorflow/contrib/lite/toco/toco_cmdline_flags.cc:245] --input_type is deprecated. It was an ambiguous flag that set both --input_data_types and --inference_input_type. If you are trying to complement the input file with information about the type of input arrays, use --input_data_type. If you are trying to control the quantization/dequantization of real-numbers input arrays in the output file, use --inference_input_type.
2018-05-16 16:19:29.492371: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1270] Converting unsupported operation: Equal
2018-05-16 16:19:29.492813: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1270] Converting unsupported operation: LogicalAnd
2018-05-16 16:19:29.521856: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 812 operators, 1241 arrays (0 quantized)
2018-05-16 16:19:29.550535: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 802 operators, 1222 arrays (0 quantized)
2018-05-16 16:19:29.582482: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 802 operators, 1222 arrays (0 quantized)
2018-05-16 16:19:29.610492: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 148 operators, 358 arrays (0 quantized)
2018-05-16 16:19:29.613697: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 2: 148 operators, 358 arrays (0 quantized)
2018-05-16 16:19:29.616876: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 3: 143 operators, 348 arrays (0 quantized)
2018-05-16 16:19:29.619981: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 4: 142 operators, 346 arrays (0 quantized)
2018-05-16 16:19:29.622935: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 5: 141 operators, 344 arrays (0 quantized)
2018-05-16 16:19:29.626348: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before dequantization graph transformations: 141 operators, 344 arrays (0 quantized)
2018-05-16 16:19:29.629622: I tensorflow/contrib/lite/toco/allocate_transient_arrays.cc:329] Total transient array allocated size: 3158144 bytes, theoretical optimal value: 3158144 bytes.
2018-05-16 16:19:29.631849: F tensorflow/contrib/lite/toco/tflite/export.cc:315] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops. Here is a list of operators for which you will need custom implementations: ExpandDims, Slice, Stack, TensorFlowShape.
./convert-lite.sh: line 11: 70835 Abort trap: 6           /Volumes/Workspace/tensorflow/workspace/tensorflow/bazel-bin/tensorflow/contrib/lite/toco/toco --input_file=$(pwd)/model/frozen_inference_graph.pb --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --output_file=$(pwd)/model/deeplabv3_mnv2_pascal_trainval.tflite --inference_type=FLOAT --input_type=QUANTIZED_UINT8 --input_arrays=ImageTensor --output_arrays=SemanticPredictions --input_shapes=1,513,513,3 --default_ranges_min=0 --default_ranges_max=255

The model could not be loaded successfully. I think it is caused the warning:
Here is a list of operators for which you will need custom implementations: ExpandDims, Slice, Stack, TensorFlowShape.

Is it possible to update node SemanticPredictions to use INT32 data type on Slice operation? Or do you have any suggestion on how to run it with TensorFlow lite?