BMNet Compiler

Introduction

The BMNET library is designed to convert the neural networks defined by CAFFE to target instructions. It seems like a compiler which translates high-level language into machine instruc- tions. It also contains three phase which are the front end, the optimizer and the back end. The front end parses source code, extracts network prototxt and weights. The optimizer is responsible for doing a broad variety of transformations to try to improve the code’s running time. The back end (also known as the code generator) then maps the code onto the target instruction set. In the BM1880 platform,we add a new feature called INT8 computation,it can provide better performance such as inference speedup. INT8 computation need an calibration table to modify network parameter,you can refer section 2 for how to generate a network’s calibration table. Refer this document,you can convert a network from FP32 to INT8 without significant accuracy loss.

General Description

We provide multiple utility tools to convert CAFFE models into machine instructions. These instructions, as well as model’s weights, would be packed into a file named bmodel (model file for BITMAIN targets), which can be executed in BITMAIN board directly. BMNet has implemented many common layers, the full list of build-in layers is in below table, and many more layers are in developing:
Activation
BatchNorm
Concat

Convolution
Eltwise
Flatten
InnerProduct
Join
LRN
Normalize
Permute
Pooling
PReLU
PriorBox
Reorg
Reshape
Scale
Split
Upsample
If layers of your network model are all supported in BMNet, it is very convenient to use command line to compile the network, otherwise you can refer to Chapter 3 to add customized layers.

Programing model

The BMNET library offers a set of API and tool to convert caffemodel into machine instructions, which is saved in bmodel file. The bmodel file also keeps more information of network model, such as network name, target name, shape, weight, etc.

Calibration Caffe Tool Guide

Introduction

● The tool would use caffe as inference framework and collect required statistics.
● Given a caffe prototxt and fp32 caffemodel ​can automatically generate both​ calibration table which contain calibration info and int8 caffemodel.

Support network list

resnet18|resnet50|alexnet_winograd|unet|custom_model|alexnet|yolov3|yolov2|mobilenet|vgg16|googlenet|resnet50_winograd|lenet|det1|densenet|det2|resnet101|SSD_300x300|det3|googlenet_v3_winograd|googlenet_v3|resnet152|mobilenet_v2

Get calibration caffe toolkit

You can download the toolkit from:
1
wget https://sophon-file.bitmain.com.cn/sophon-prod/drive/19/02/09/16/BM1880-Calibration-Tools.zip
Copied!
and the files included in this toolkit as below, the CustomModel and the Resnet50 offered just for examples.
1
calibration-caffe-tools
2
├── BMNet_1880_api.pdf
3
├── build
4
│   └── calibration_math.so
5
├── calibration_caffe.bin
6
├── Calibration Tool Guide.pdf
7
├── CustomModel
8
│   ├── custom.caffemodel
9
│   ├── custom.prototxt
10
│   ├── input.txt
11
│   ├── test2.jpg
12
│   └── test.jpg
13
├── custom_model.json
14
└── Resnet50
15
├── deploy.prototxt
16
├── ILSVRC2012_val
17
│   └── input.txt
18
├── resnet50_input_1_3_224_224.bin
19
└── ResNet-50-model.caffemodel
20
Copied!

Usage

    Put the prototxt and fp32 caffemodel in the directory of output_path.
    The calibration table and int8 caffemodel would generate in the
    ouput_path, the script as following:
1
./calibration_caffe.bin [​net_name​] [​output_path​] --iteration [iteration batch size]
Copied!
    The relative parameter is as following:
    ○ [net_name]: net name ○ [output_path]: output path of prototxt and int8 caffemodel. ○ [iteration batch size]: calibration batch size.

Input data layer

    You can define the calibration data in the caffe prototxt. You can see
    example in the deploy.prototxt.
    The relative parameter is as following:
      [data_list]: The file describes where to find the images. Each line is the path to image.
      [h]: input data height
      [w]: input data width
      [color_format]: Specify which color format you want to use. Only
      support RGB/BGR.
      [r/g/b_mean]: RGB mean value. If > 0, all image will subtract the
      mean value.
      [scale]: The scale value of the data. It will multiply to the data after minus the r/g/b mean value. The default value is 1
      [mirror]: Specify the data needs to mirror or not. The default value is 0.
        0: no need to mirror
        1: vertical and horizontal
        2: vertical
        3: horizontal
      [transpose]: Specify the data transpose axises. The default is [2,0,1] (equal to [c,h,w] order).
      [padding]: If true, padding is added when resize to the target size to keep the original aspect ratio.The default is false.
      [debug]: Save each data as specific format in “debug” folder in the working directory. Note that it is saved before doing transpose. The default is None.
        npy: Save as numpy array
        image: Save as image

Custom model support

If you want to calibrate your own model, you can define ‘custom_model.json’. The relative parameter is as following:
    [name]: net name
    [in_prototxt]: input caffe prototxt file
    [in_caffemodel]: input caffe model
The out file will be ‘bmnet_​your_net_name​_calibration_table.pb2’ and ‘bmnet_​your_net_name_​int8.caffemodel’

Example

Enter calibration_tool folder while you get the toolkit from
1
wget https://sophon-file.bitmain.com.cn/sophon-prod/drive/19/02/09/16/BM1880-Calibration-Tools.zip
Copied!
1
Resnet50/
2
├── deploy.prototxt
3
├── ILSVRC2012_val //ImageNet dataset,you can download from www.image-net-org
4
└── ResNet-50-model.caffemodel
Copied!
You can download the dataset ILSVRC2012_val here: http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_img_val.tar
Describe your jpeg dataset path in deploy.prototxt
1
name: "ResNet-50"
2
input_dim: 1
3
input_dim: 3
4
input_dim: 224
5
input_dim: 224
6
7
layer {
8
name: 'input'
9
type: 'Python'
10
top: 'data'
11
python_param {
12
module: 'custom_data_layer.general_data_layer'
13
layer: 'DataLayer'
14
param_str: "{'data_list': ./Resnet50/ILSVRC2012_val/input.txt, 'color_format': rgb, 'h': 224, 'w': 224, 'r_mean': 123.68, 'g_mean': 116.779, 'b_mean': 103.939}"
15
}
16
}
17
...
18
...
Copied!
Calibration toolkit also support lmdb dataset, describe your lmdb-dataset path in deploy.prototxt
1
layer {
2
name: "data"
3
type: "Data"
4
top: "data"
5
top: "label"
6
include {
7
phase: TEST
8
}
9
transform_param {
10
mirror: false
11
crop_size: 224
12
}
13
data_param {
14
source: "/path/to/imagenet/ilsvrc12_val_lmdb"
15
batch_size: 1
16
backend: LMDB
17
}
18
}
Copied!
and the input.txt describe the path of jpeg files, content as below:
1
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000001.JPEG
2
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000002.JPEG
3
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000003.JPEG
4
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000004.JPEG
5
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000005.JPEG
6
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000006.JPEG
7
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000007.JPEG
8
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000008.JPEG
9
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000009.JPEG
10
./Resnet50/ILSVRC2012_val/ILSVRC2012_val_00000010.JPEG
11
...
Copied!
Now, you can start the Resnet50 caffemodel calibration
1
./calibration_caffe.bin resnet50 ./resnet50 --iteration 100
2
WARNING: Logging before InitGoogleLogging() is written to STDERR
3
W1229 09:45:44.144830 12707 _caffe.cpp:139] DEPRECATION WARNING - deprecated use of Python interface
4
W1229 09:45:44.144860 12707 _caffe.cpp:140] Use this instead (with the named "weights" parameter):
5
W1229 09:45:44.144862 12707 _caffe.cpp:142] Net('Resnet50/deploy.prototxt', 1, weights='Resnet50/ResNet-50-model.caffemodel')
6
I1229 09:45:44.147876 12707 layer_factory.hpp:77] Creating layer input
7
I1229 09:45:44.153451 12707 net.cpp:220] Creating Layer input
8
I1229 09:45:44.153475 12707 net.cpp:516] input -> data
9
I1229 09:45:44.154451 12707 net.cpp:258] Setting up input
10
I1229 09:45:44.154466 12707 net.cpp:265] Top shape: 1 3 224 224 (150528)
11
I1229 09:45:44.154469 12707 net.cpp:273] Memory required for data: 602112
12
I1229 09:45:44.154474 12707 layer_factory.hpp:77] Creating layer conv1
13
I1229 09:45:44.154484 12707 net.cpp:220] Creating Layer conv1
14
I1229 09:45:44.154486 12707 net.cpp:542] conv1 <- data
15
I1229 09:45:44.154491 12707 net.cpp:516] conv1 -> conv1
16
I1229 09:45:44.154544 12707 net.cpp:258] Setting up conv1
17
I1229 09:45:44.154551 12707 net.cpp:265] Top shape: 1 64 112 112 (802816)
18
I1229 09:45:44.154556 12707 net.cpp:273] Memory required for data: 3813376
19
I1229 09:45:44.154564 12707 layer_factory.hpp:77] Creating layer bn_conv1
20
I1229 09:45:44.154572 12707 net.cpp:220] Creating Layer bn_conv1
21
I1229 09:45:44.154575 12707 net.cpp:542] bn_conv1 <- conv1
22
I1229 09:45:44.154580 12707 net.cpp:503] bn_conv1 -> conv1 (in-place)
23
I1229 09:45:44.154624 12707 net.cpp:258] Setting up bn_conv1
24
I1229 09:45:44.154630 12707 net.cpp:265] Top shape: 1 64 112 112 (802816)
25
I1229 09:45:44.154633 12707 net.cpp:273] Memory required for data: 7024640
26
I1229 09:45:44.154642 12707 layer_factory.hpp:77] Creating layer scale_conv1
27
I1229 09:45:44.154649 12707 net.cpp:220] Creating Layer scale_conv1
28
I1229 09:45:44.154652 12707 net.cpp:542] scale_conv1 <- conv1
29
I1229 09:45:44.154656 12707 net.cpp:503] scale_conv1 -> conv1 (in-place)
30
I1229 09:45:44.154667 12707 scale_layer.cpp:52] num_axes=1
31
I1229 09:45:44.154672 12707 layer_factory.hpp:77] Creating layer scale_conv1
32
...
33
...
34
...
35
res5b 23.8178100586 0
36
res5c 32.5854530334 0
37
bn2a_branch1 9.26634883881 4
38
res2c_relu 10.6714229584 0
39
res4e 15.3665313721 0
40
res5a_relu 27.6742744446 0
41
bn5b_branch2c 8.03337287903 6
42
res4d_branch2a 7.00451898575 8
43
bn4f_branch2b 10.0731954575 5
44
bn2b_branch2a 7.84093761444 6
45
bn2b_branch2b 7.20697212219 6
46
bn2b_branch2c 10.2082700729 6
47
res5a_branch2c 1.2891266346 5
48
res5a_branch1 5.15547370911 6
49
res5a_branch2a 7.32843542099 7
50
res3d_branch2b 6.73829030991 7
51
res3d_branch2c 2.87677979469 6
52
res3d_branch2a 9.06925201416 7
53
res3d_branch2b_relu 10.6139621735 0
54
res2a_relu 15.9442462921 0
55
bn4e_branch2b 6.69408226013 6
56
scale5a_branch2a 9.30217647552 6
57
scale5a_branch2c 12.7850370407 6
58
scale5a_branch2b 9.97138404846 7
59
Time: 146.367562s
60
Copied!
    In the path of ​Resnet50/, you can see bmnet_​resnet50_​ calibration_table.1x10.pb2 and bmnet_​resnet50_​int8.1x10.caffemodel.
1
Resnet50/
2
├── bmnet_resnet50_calibration_table.1x10
3
├── bmnet_resnet50_calibration_table.1x10.pb2
4
├── bmnet_resnet50_calibration_table.1x10.prototxt
5
├── bmnet_resnet50_int8.1x10.caffemodel
6
├── deploy.prototxt
7
├── ILSVRC2012_val
8
└── ResNet-50-model.caffemodel
Copied!

bm_builder.bin

Description

The bm_builder.bin combines frontend, optimizer and backend modules into one executable binary, and links to libbmnet.so. It takes network’s caffemodel and deploy.prototxt as inputs, and finally generates bmodel after compiled.
Get the bm_builder toolkit:
you need install the BMNNSDK usb mode on your ubuntu x86-64 host,how to install BMNSDK usb mode.
bm_builder.bin in path:
1
/opt/bmtap2/bm1880-usb_1.0.3.1/bin$ ll
2
-rwxr-xr-x 1 root root 60472 1226 11:14 bm_builder.bin*
Copied!

End-user Options

1
bm_builder.bin
2
-t or --target
3
Specify the name of BITMAIN target board (bm1880).
4
-n or --name
5
Specify the name of deep learning network.
6
-s or --shape
7
Specify the input shape of network. Dims should be separated by commas and no backspace is
8
allowed.
9
-u or --plugin
10
Specify the directories of cpu op plugins.
11
-c or --caffemodel
12
Specify the caffemodel generated by calibration_caffe.bin.
13
-m or --modified_proto
14
If you want to modify the prototxt, specify the modified deploy.prototxt of network.
15
-o or --out_model
16
Specify the output bmodel file.
17
-d or --in_ctable=input_ctable_file
18
Specify the calibration table generated by calibration_caffe.bin.
19
-e or --out_ctable=output_ctable_file
20
Specify the output of optimizer calibration table.
21
-p or --out_proto=output_prototxt_file
22
Specify the optimizer prototxt file of network.
23
--enable -weight -optimize=yes|no [no]
24
Specify the option(yes or no) to enable or disable optimization .
25
Copied!

Example

1
cd calibation_tool/Resnet50
2
/opt/bmtap2/bm1880-usb_1.0.3.1/bin/bm_builder.bin
3
-t bm1880
4
-n resnet50
5
-c bmnet_resnet50_int8.1x10.caffemodel
6
--in_ctable=bmnet_resnet50_calibration_table.1x10.pb2
7
--out_ctable=bmnet_resnet50_calibration_opt_table.1x10.pb2
8
--enable-weight-optimize=yes
9
--enable-layer-group=yes
10
--fc-left-shift=6
11
-s 1,3,224,224
12
-p resnet50_frontend_opt.proto
13
-o resnet50.bmodel
Copied!
please pay attention that parameter bm_builder.bin used above should be one line
resnet50.bmodel can be generated.
1
Resnet50/
2
├── bmnet_resnet50_calibration_opt_table.1x10.pb2
3
├── bmnet_resnet50_calibration_table.1x10
4
├── bmnet_resnet50_calibration_table.1x10.pb2
5
├── bmnet_resnet50_calibration_table.1x10.prototxt
6
├── bmnet_resnet50_int8.1x10.caffemodel
7
├── bmnet.s
8
├── deploy.prototxt
9
├── ILSVRC2012_val
10
├── resnet50.bmodel
11
├── resnet50_frontend_opt.proto
12
├── resnet50_input_1_3_224_224.bin
13
└── ResNet-50-model.caffemodel
Copied!

bmodel testing

How to test the bmodel you can follow the instructions below If you use the Edge Development Board SoC mode,
1
on ubuntu x86-64 host pc:
2
git clone https://github.com/BM1880-BIRD/bm1880-ai-demo-program.git
3
//copy the bmtap2-bm1880-soc_basic_test folder to rootfs of BM1880 EDB,such as /system/
4
//copy resnet50.bmodel,resnet50_input_16_3_224_224.bin to rootfs of BM1880 EDB,such as /system/
5
6
on BM1880 shell:
7
cd /system/data
8
./load_driver.sh
9
ldconfig
10
cd /system/bmtap2-bm1880-soc_basic_test/bin/
11
./test_bmnet_bmodel resnet50_input_1_
12
3_224_224.bin resnet50.bmodel ouput.bin 1 3 224 224
13
<CMD> ./test_bmnet_bmodel resnet50_input_1_3_224_224.bin resnet50.bmodel ouput.bin 1 3 224 224
14
Runtime Version : [1.0.3]
15
input size: 150528
16
output size:1000
17
outputs[0]: [1,1,1,1000], "fc1000"
18
test_bmnet_bmodel: load 112 us, run 15216 us, read 11 us
19
WARNING: Logging before InitGoogleLogging() is written to STDERR
20
I0101 08:02:39.781909 238 bm_device_soc.cpp:216] device[0] closed
Copied!
For NNS or usb mode of Edge Development Board, you can follow the instruction below:
1
/opt/bmtap2/bm1880-usb_1.0.3.1/bin/test_bmnet_bmodel resnet50_input_1_3_224_224.bin resnet50.bmodel ouput.bin 1 3 224 224
2
<CMD> /opt/bmtap2/bm1880-usb_1.0.3.1/bin/test_bmnet_bmodel res/resnet50_input_1_3_224_224.bin r50_batch_1.bmodel output.bin 1 3 224 224
3
Runtime Version : [1.0.3]
4
input size: 150528
5
WARNING: Logging before InitGoogleLogging() is written to STDERR
6
I0114 21:07:43.254546 15189 bm_firmware.cpp:76] check firmware status ...
7
I0114 21:07:43.255086 15189 bm_firmware.cpp:82] firmware loading ...
8
I0114 21:07:43.361970 15189 bm_firmware.cpp:47] firmware load success
9
I0114 21:07:49.373096 15189 bm_firmware.cpp:98] firmware is running
10
I0114 21:07:49.373186 15189 bm_device.cpp:104] device[0] opened,gmem_size : 0x40000000
11
output size:4000
12
outputs[0]: [1,1,1,1000], "prob_1"
13
[Softmax run]: count: 1000 axis: 1 channels: 1000 outer_num: 1 inner_num: 1 threshold_x: 28.1843 threshold_y: 128
14
cpu_op: softmax_op done
15
test_bmnet_bmodel: load 116 us, run 15200 us, read 13 us
16
Copied!

Calibration ONNX Tool Guide

Description

Calibration tool for onnx model.

Get calibration ONNX tool

You can download the toolkit from:
and the files included in this toolkit as below, the CustomModel and the Resnet50 offered just for examples.
1
calibration_onnx_tool/
2
├── bin
3
│   ├── bm_builder_onnx.bin
4
│   ├── onnx_calibration.bin
5
│   ├── run_resnet50_bmodel_armv8
6
│   └── run_resnet50_bmodel_x86
7
├── lib
8
│   ├── armv8
9
│   ├── libbmkernel.so
10
│   ├── libbmnet_caffe_pb.so
11
│   ├── libbmnet.so
12
│   ├── libbmodel.so
13
│   ├── libbmruntime.so
14
│   ├── libcaffe2.so
15
│   ├── libonnx_proto.so
16
│   ├── libonnx.so
17
│   └── x86_64
18
├── Readme.txt
19
├── res
20
│   ├── imagenet_classes.txt
21
│   ├── imagenet_partial
22
│   └── resnet50_input_1_3_224_224.bin
23
└── Resnet50
24
├── resnet50.onnx
25
└── run_resnet50_bmodel.cpp
26
Copied!

Usage

1
./bin/onnx_calibration.bin
2
parse arg ret: 0
3
Invalide arguments
4
onnx_calibration
5
-d dataset path
6
-m onnxmodel file
7
-o output model file
8
-i iteration number
9
-b batch size
Copied!

Example

1
$ cd calibration_onnx_tool
2
$ export LD_LIBRARY_PATH=./lib/
3
$ bin/onnx_calibration.bin
4
-m ./Resnet50/resnet50.onnx
5
-d ./res/imagenet_partial/LSVRC-2012/
6
-o r50.cal.onnx
7
-i 10
8
-b 1
9
10
onnxmodel ./Resnet50/resnet50.onnx
11
dataset ./res/imagenet_partial/LSVRC-2012/
12
output model r50.cal.onnx
13
iteration 10
14
input batch size 1
15
parse arg ret: 0
16
<CMD> bin/onnx_calibration.bin -m ./Resnet50/resnet50.onnx -d ./res/imagenet_partial/LSVRC-2012/ -o r50.cal.onnx -i 10 -b 1
17
This version of onnx-caffe2 targets ONNX operator set version 7, but the model we are trying to import uses version 8. We will try to import it anyway, but if the model uses operators which had BC-breaking changes in the intervening versions, import will fail.
18
This version of onnx-caffe2 targets ONNX operator set version 7, but the model we are trying to import uses version 8. We will try to import it anyway, but if the model uses operators which had BC-breaking changes in the intervening versions, import will fail.
19
This version of onnx-caffe2 targets ONNX operator set version 7, but the model we are trying to import uses version 8. We will try to import it anyway, but if the model uses operators which had BC-breaking changes in the intervening versions, import will fail.
20
This version of onnx-caffe2 targets ONNX operator set version 7, but the model we are trying to import uses version 8. We will try to import it anyway, but if the model uses operators which had BC-breaking changes in the intervening versions, import will fail.
21
This version of onnx-caffe2 targets ONNX operator set version 7, but the model we are trying to import uses version 8. We will try to import it anyway, but if the model uses operators which had BC-breaking changes in the intervening versions, import will fail.
22
WARNING: Logging before InitGoogleLogging() is written to STDERR
23
W0115 11:41:41.498390 21843 init.h:99] Caffe2 GlobalInit should be run before any other API calls.
24
W0115 11:41:41.500347 21843 init.h:99] Caffe2 GlobalInit should be run before any other API calls.
25
I0115 11:41:41.803261 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
26
I0115 11:41:41.803356 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
27
I0115 11:41:41.804275 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
28
I0115 11:41:41.804330 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
29
I0115 11:41:41.805279 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
30
I0115 11:41:41.805331 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
31
I0115 11:41:41.806298 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
32
I0115 11:41:41.806352 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
33
I0115 11:41:41.807260 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
34
I0115 11:41:41.807310 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
35
I0115 11:41:41.808189 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
36
I0115 11:41:41.808254 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
37
I0115 11:41:41.809103 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
38
I0115 11:41:41.809151 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
39
I0115 11:41:41.809998 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
40
I0115 11:41:41.810055 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
41
I0115 11:41:41.810925 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
42
I0115 11:41:41.810973 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
43
I0115 11:41:41.811820 21843 Calibration_legacy.cpp:111] pInputDims NCHW: 1 3 224 224
44
I0115 11:41:41.811872 21843 Calibration_legacy.cpp:119] datum CHW: 3 256 256
45
46
...
47
48
layer {
49
name: "res5c_1"
50
blob_param {
51
name: "res5c_1"
52
threshold_y: 33.7025
53
}
54
}
55
layer {
56
name: "res5c_relu_1"
57
blob_param {
58
name: "res5c_relu_1"
59
threshold_y: 33.7025
60
}
61
}
62
layer {
63
name: "pool5_1"
64
blob_param {
65
name: "pool5_1"
66
threshold_y: 14.885545
67
}
68
}
69
layer {
70
name: "OC2_DUMMY_106"
71
blob_param {
72
name: "OC2_DUMMY_106"
73
threshold_y: 14.885545
74
}
75
blob_param {
76
name: "OC2_DUMMY_1"
77
threshold_y: 0
78
}
79
}
80
layer {
81
name: "fc1000_1"
82
blob_param {
83
name: "fc1000_1"
84
threshold_y: 28.184326
85
}
86
}
87
layer {
88
name: "prob_1"
89
blob_param {
90
name: "prob_1"
91
threshold_y: 0.062530234
92
}
Copied!
You can see the the file r50.cal.onnx be generated.
1
./
2
├── bin
3
│   ├── bm_builder_onnx.bin
4
│   ├── onnx_calibration.bin
5
│   ├── run_resnet50_bmodel_armv8
6
│   └── run_resnet50_bmodel_x86
7
├── lib
8
│   ├── armv8
9
│   ├── libbmkernel.so
10
│   ├── libbmnet_caffe_pb.so
11
│   ├── libbmnet.so
12
│   ├── libbmodel.so
13
│   ├── libbmruntime.so
14
│   ├── libcaffe2.so
15
│   ├── libonnx_proto.so
16
│   ├── libonnx.so
17
│   └── x86_64
18
├── r50.cal.onnx
19
├── Readme.txt
20
├── res
21
│   ├── imagenet_classes.txt
22
│   ├── imagenet_partial
23
│   └── resnet50_input_1_3_224_224.bin
24
└── Resnet50
25
├── resnet50.onnx
26
└── run_resnet50_bmodel.cpp
27
Copied!

bm_build_onnx.bin

Description

The bm_builder_onnx.bin combines frontend, optimizer and backend modulesinto one executable binary, and links to libbmnet.so. It takes network’scalibrated .onnx model as inputs, and finally generates bmodel after compiled.

End-User Options

1
bm_builder_onnx.bin [options]
2
3
-enable-less-loss-fc -enable less-loss FC
4
-fc-left-shift=<int> - FC left-shift for partial sum
5
-force-less-loss-fc - force less-loss FC
6
7
-dump-all-neuron -
8
-dump-layer-group-info=<file> -Specify log location
9
-enable-layer-group -
10
=yes -enable
11
=no -disable
12
-enable-softmax
13
=yes -enable
14
=no -disable
15
-ignore-bank-conflict -
16
-layer-group-fix - fix layer group for more pass
17
-layer-group-sese - use Single-Entry-Single-Exit tiling mode
18
-name=<string> -name of network
19
-onnx=<file> -input onnx model
20
-out-cmdbuf=<file> -out cmdbuf file
21
-out-model=<file> -output bmodel file
22
-out-proto=<file> -output prototxt file
23
-plugin=<string> -path of cpu op plugin
24
-quantized-onnx-model=<string> -the path for the quantized-onnx-model
25
-shape=<string> -n,c,h,w, give 0 mean using shape info in model,
26
example : -s 2, 0, 0, 0
27
-target - Specify your target
28
=bm1880 - BM1880
29
30
-help - Display available options (-help-hidden for more)
31
-help-list - Display list of available options (-help-list-hidden for more)
32
-version - Display the version of this program
33
34
Copied!

Example

1
./bin/bm_builder_onnx.bin -t bm1880
2
-n r50
3
-c r50.cal.onnx
4
-s 1,0,0,0
5
--enable-layer-group=yes
6
-p r50_bmnet_opt_from_onnx.proto
7
-o r50_batch_1.bmodel
8
-u ./lib/
9
10
<CMD> ./bin/bm_builder_onnx.bin -t bm1880 -n r50 -c r50.cal.onnx -s 1,0,0,0 --enable-layer-group=yes -p r50_bmnet_opt_from_onnx.proto -o r50_batch_1.bmodel -u ./lib/
11
target bm1880
12
W0115 13:52:28.163966 28228 OnnxFrontendContext.cpp:112]
13
W0115 13:52:28.164011 28228 OnnxFrontendContext.cpp:113] OnnxFrontendContext: converting to bmnet model
14
W0115 13:52:30.412812 28228 BM188xBackendContext.cpp:114]
15
W0115 13:52:30.412829 28228 BM188xBackendContext.cpp:115] BM188xBackendContext: building cmdbuf
16
W0115 13:52:30.420598 28228 conv_parallel_bmkernel.cpp:164] [ConvFixedParallel::split], coeff too large: 65536, not supported
17
W0115 13:52:30.420828 28228 conv_parallel_bmkernel.cpp:164] [ConvFixedParallel::split], coeff too large: 73728, not supported
18
W0115 13:52:30.421167 28228 conv_parallel_bmkernel.cpp:164] [ConvFixedParallel::split], coeff too large: 73728, not supported
19
W0115 13:52:30.421496 28228 conv_parallel_bmkernel.cpp:164] [ConvFixedParallel::split], coeff too large: 73728, not supported
20
frontend_out_ctable_file.pb2 fc1000_1
21
frontend_out_ctable_file.pb2 data_0
22
Copied!
You can see the r50_batch_1.bmodel generated:
1
./
2
├── bin
3
│   ├── bm_builder_onnx.bin
4
│   ├── onnx_calibration.bin
5
│   ├── run_resnet50_bmodel_armv8
6
│   └── run_resnet50_bmodel_x86
7
├── bmnet.s
8
├── frontend_out_ctable_file.pb2
9
├── frontend_out_ctable_file.prototxt
10
├── lib
11
│   ├── armv8
12
│   ├── libbmkernel.so
13
│   ├── libbmnet_caffe_pb.so
14
│   ├── libbmnet.so
15
│   ├── libbmodel.so
16
│   ├── libbmruntime.so
17
│   ├── libcaffe2.so
18
│   ├── libonnx_proto.so
19
│   ├── libonnx.so
20
│   └── x86_64
21
├── optimizer_out_ctable_file.pb2
22
├── output.bin
23
├── r50_batch_1.bmodel
24
├── r50_bmnet_opt_from_onnx.proto
25
├── r50.cal.onnx
26
├── Readme.txt
27
├── res
28
│   ├── imagenet_classes.txt
29
│   ├── imagenet_partial
30
│   └── resnet50_input_1_3_224_224.bin
31
└── Resnet50
32
├── resnet50.onnx
33
└── run_resnet50_bmodel.cpp
34
35
7 directories, 25 files
36
Copied!

bmodel testing

The testing method is same as the method that caffe bmodel used, you can visit the section above.
Last modified 2yr ago