Caffe

Network Configuration

TensorOp

TensorOp represents a BMNET IR, which is a bridge between front end and back end. it provides lots of member method to set information to or get from it. Below is the prototype:
1
namespace bmnet {
2
class TensorOp {
3
public:
4
int input_shape_size();
5
int output_shape_size();
6
const TensorShape& input_shape(int index);
7
const TensorShape& output_shape(int index);
8
TensorShape* add_output_shape();
9
u64 global_input(int index);
10
u64 global_output(int index);
11
TGCustomizedParameter* mutable_tg_customized_param();
12
const TGCustomizedParameter& tg_customized_param();
13
};
14
}
Copied!

TensorOp::input_shape_size

1
void TensorOp::input_shape_size()
Copied!
Return the number of inputs.

TensorOp::output_shape_size

1
void TensorOp::output_shape_size()
Copied!
Return the number of outputs.

TensorOp::input_shape

1
const TensorShape& TensorOp::input_shape( int index)
Copied!
Return shape of input by index.
Parameter
Type
Description
index
int
[Required] index of input that to be returned.

TensorOp::output_shape

1
const TensorShape& TensorOp::output_shape(int index)
Copied!
Return shape of output by index.

Parameter
Type
Description
index
int
[Required] index of output that to be returned.

TensorOp::add_output_shape

1
TensorShape* TensorOp::add_output_shape()
Copied!
Return a mutable pointer to a new added TensorShape of outputs. The returned TensorShape could be modified latter.

TensorOp::global_input

1
u64 TensorOp::global_input(int index)
Copied!
Return offset of input tensor by index, while it was stored in device memory.
Parameter
Type
Description
index
int
[Required] index of input that to be returned.

TensorOp::global_output

1
u64 TensorOp::global_output(int index)
Copied!
Return offset of output tensor by index, while it was stored in device memory.
Parameter
Type
Description
index
int
[Required] index of output that to be returned.

TensorOp::mutable_tg_customized_param

1
TGCustomizedParameter* TensorOp::mutable_tg_customized_param()
Copied!
Return a mutable pointer to parameters of customized BMNET IR.

TensorOp::tg_customized_param

1
const TGCustomizedParameter& TensorOp::tg_customized_param()
Copied!
Return reference of customized BMNET IR’s paramters.

CustomizedCaffeLayer

CustomizedCaffeLayer is abstract class, which is used to implement a Layer to convert CAFFE Layer into BMNet IR(please refer to Chapter 5 for details about BMNet IR). If you want to introduce a customized CAFFE layer into BMNet, please inherit this class and implement all pure virtual functions of it. The CustomizedCaffeLayer inherits from CaffeLayer/Layer class. Below are the prototypes of them:
1
namespace bmnet {
2
class Layer {
3
public:
4
Layer();
5
virtual ~Layer(void);
6
virtual std::string layer_name() = 0;
7
virtual void dump () = 0;
8
virtual void codegen(TensorOp *op) = 0;
9
10
protected:
11
void add_output_offset(int offset);
12
};
13
}
14
15
namespace bmnet {
16
17
class CaffeLayer : public Layer {
18
public:
19
CaffeLayer(){}
20
virtual ~CaffeLayer(void);
21
protected:
22
caffe::LayerParameter &layer_;
23
};
24
}
25
26
namespace bmnet {
27
28
class CustomizedCaffeLayer : public CaffeLayer {
29
public:
30
CustomizedCaffeLayer();
31
~CustomizedCaffeLayer();
32
void setup(TensorOp* op) override {
33
...
34
...
35
TGCustomizedParameter* param = op->mutable_tg_customized_param ();
36
param->set_sub_type(layer_name());
37
}
38
};
39
}
Copied!

CustomizedCaffeLayer::layer_name

1
std::string CustomizedCaffelayer::layer_name()
Copied!
Pure virtual function, return type of new added CAFFE layer.

CustomizedCaffeLayer::dump

1
Pure virtual function, is used to print information of CAFFE Layer.
Copied!

CustomizedCaffeLayer:: setup

1
void CustomizedCaffelayer::setup()
Copied!
Option. It is used to set sub type of Customized Layer only. Implement by default. If child class will override it, this parent class setup function must be call first.

CustomizedCaffeLayer::codegen

Pure virtual function, is used to setup BMNET IR according to LayerParameter of CAFFE Layer. In this function, you should setup output shape and fill parameters to TensorOp.
1
TBD
Copied!
Parameter
Type
Description
op
TensorOp*
[Required] pointer to a instance of BMNET IR

CustomizedCaffeLayer::add_output_offset

1
void CustomizedCaffelayer::add_output_offset (int offset)
Copied!
Protected member method, should be called when setup output offset of Layer’s top.
Parameter
Type
Description
offset
int
[Required] offset of output, should be 0.

CustomizedCaffeLayer::layer_

1
caffe::LayerParameter CustomizedCaffelayer::&layer_
Copied!
Protected member variable, which is reference of customized CAFFE layer’s LayerParameter.

CustomizedTensorFixedInst

CustomizedTensorFixedInst is abstract class, which is used to implement a Layer to convert BMNET IR into instructions by BMKernel APIs. Please inherit this class and implement all pure virtual functions of it. The CustomizedTensorFixedInst inherits from TensorFixedInst/ TensorInst class. Below are the prototypes of them:
1
namespace bmnet {
2
class TensorFixedInst: public TensorInst {
3
public:
4
TensorFixedInst() : TensorInst() {}
5
TensorFixedInst(TensorOp &op) : TensorInst(op) {}
6
virtual ~ TensorFixedInst (void);
7
void SetCalibrationParameter(
8
const LayerCalibrationParameter &calibration_parameter) {
9
m_calibrationParameter = calibration_parameter;
10
}
11
void AddInputCalibrationParameter(
12
const LayerCalibrationParameter &calibration_parameter){
13
m_inputCalibrationParameter.push_back(calibration_parameter);
14
}
15
protected:
16
LayerCalibrationParameter m_calibrationParameter;
17
std::vector <LayerCalibrationParameter >
18
m_inputCalibrationParameter;
19
};
20
}
21
22
namespace bmnet {
23
class TensorInst {
24
public:
25
TensorInst();
26
virtual ~TensorInst(void);
27
virtual std::string inst_name() = 0;
28
virtual void dump () = 0;
29
virtual void encode () = 0;
30
31
protected:
32
TensorOp &op_;
33
};
34
}
35
36
namespace bmnet {
37
38
class CustomizedTensorFixedInst : public TensorFixedInst {
39
public:
40
CustomizedTensorFixedInst ();
41
~CustomizedTensorFixedInst ();
42
protected:
43
u64 get_global_neuron_base();
44
u64 get_global_weight_base();
45
};
46
}
47
Copied!

CustomizedTensorFixedInst::inst_name

1
std::string CustomizedTensorFixedInst::inst_name()
Copied!
Pure virtual function, return type of customized BMNET IR.

CustomizedTensorFixedInst::dump

1
void CustomizedTensorFixedInst::dump()
Copied!
Pure virtual function, is used to print information of BMNET IR.

CustomizedTensorFixedInst::encode

1
void CustomizedTensorFixedInst::encode()
Copied!
Pure virtual function, is used to convert BMNET IR into instructions using BMKernel APIs.

CustomizedTensorFixedInst::get_global_neuron_base

1
u64 CustomizedTensorFixedInst::get_global_neuron_base()
Copied!
Protected member method, return the base address, where the neurons are stored in device memory.

CustomizedTensorFixedInst::get_global_weight_base

1
u64 CustomizedTensorFixedInst::get_global_weight_base()
Copied!
Protected member method, return the base address, where weight is stored in device memory.

CustomizedTensorFixedInst::op_

1
TensorOp CustomizedTensorFixedInst::&op_
Copied!
Protected member variable, which is reference of BMNET IR.

TGCustomizedParamter

TGCustomizedParamter represents a customized BMNET IR’s parameters. It provides member methods to set parameters to or get from it. Below is the prototype:
1
namespace bmnet {
2
3
class TGCustomizedParameter {
4
public:
5
int i32_param_size();
6
int f32_param_size();
7
int i32_param(int index);
8
float f32_param(int index);
9
void add_i32_param(int value);
10
void add_f32_param(float value);
11
};
12
}
Copied!

TGCustomizedParamter::i32_param_size

1
void TGCustomizedParamter::i32_param_size()
Copied!
Return the number of int parameters, which stored in TGCustomizedParamter.

TGCustomizedParamter::f32_param_size

1
void TGCustomizedParamter::f32_param_size()
Copied!
Return the number of float parameters, which stored in TGCustomizedParamter.

TGCustomizeParamter::i32_param

1
int TGCustomizedParamter::i32_param(int index)
Copied!
Return int parameter by index.
Parameter
Type
Description
index
index
[Required] index of int parameter that to be returned.

TGCustomizeParamter::f32_param

1
float TGCustomizedParamter::f32_param( int index)
Copied!
Return int parameter by index.
Parameter
Type
Description
index
index
[Required] index of float parameter that to be returned.

TGCustomizeParamter::add_i32_param

1
void TGCustomizedParamter::add_i32_param(int value)
Copied!
Append a new int parameter to TGCustomizedParamter.
Parameter
Type
Description
value
int
[Required] int parameter.

TGCustomizeParamter::add_f32_param

1
void TGCustomizedParamter::add_f32_param(int value)
Copied!
Append a new int parameter to TGCustomizedParamter.
Parameter
Type
Description
value
float
[Required] float parameter.

TensorShape

TensorShape represents a shape of tensor. Below is the prototype:
1
namespace bmnet {
2
3
class TensorShape {
4
public:
5
void CopyFrom(const TensorShape& from);
6
int dim_size() const;
7
int dim(int index);
8
void add_dim(int value);
9
};
10
}
Copied!

TensorShape::dim_size

1
int TensorShape::dim_size()
Copied!
Return the number of dims.

TensorShape::dim

1
int TensorShape::dim(int index)
Copied!
Return one dim by index.
Parameter
Type
Description
Index
int
[Required] index of dim that to be returned.

TensorShape::add_dim

1
void TensorShape::add_dim(int value)
Copied!
Append a dim to TensorShape.
Parameter
Type
Description
value
int
[Required] new dim to be appended.

TensorShape::CopyFrom

1
void TensorShape::CopyFrom(const TensorShape& from)
Copied!
Copy from another TensorShape instance.
Parameter
Type
Description
value
const TensorShape&
[Required] source TensorShape instance.

CaffeBuilder

CaffeBuilder is a class, which provides a uniform interface to combine front end/optimizer/back end core code into one, to compile CAFFE neuron network graph into bmodel file. The CaffeBuilder inherits from Builder class, which is a base compiler class. Below are the prototypes of them:
1
namespace bmnet {
2
3
class Builder {
4
public:
5
Builder(CHIP_VER ver);
6
virtual ~Builder();
7
void addCustomizedTensorInst(TensorInst *inst);
8
void build(int n, int c, int h, int w, int opt);
9
void store_prototxt(const char *dst);
10
void store_model(const char *net_name, const char *dst);
11
};
12
}
Copied!
1
namespace bmnet {
2
3
class CaffeBuilder : public Builder {
4
public:
5
CaffeBuilder(CHIP_VER ver, const char *modified_proto , const char *caffemodel , const char *weight_bin , const char *in_ctable , const char *out_ctable);
6
~CaffeBuilder();
7
void addCustomizedLayer(Layer *layer);
8
};
Copied!

CaffeBuilder::CaffeBuilder

1
CaffeBuilder::CaffeBuilder(
2
CHIP_VER ver ,
3
const char *modified_proto ,
4
const char *caffemodel ,
5
const char *weight_bin ,
6
const char *in_ctable ,
7
const char *out_ctable)
Copied!
Constructor function of CaffeBuilder class.
Parameter
Type
Description
ver
CHIP_VER
[Required] The target chip version. Currently only BM_CHIP_BM1880 is
available.
modified_proto
const char*
[Optional] The modified prototxt file, please refer Chapter 4 to get more detail.
caffemodel
const char*
[Required] The specified caffemode file of network
weight_bin
const char*
[Optional] The specified weight file of network
in_ctable
const char*
[Required] The specified calibration table file of network
out_ctable
const char*
[Required] The specified weight file of network
modified_proto are optional parameters, that means you no need to fill all of this parameters. Below combination are valid: 1) caffemodel only; 2) caffemodel, as well as modified_protos

CaffeBuilder::Builder

Core member function of CaffeBuilder class, used to compile the network by specifying input shape and optimization level.
Parameter
Type
Description
n,c,h,w
int
[Required] The input shape
opt
int
[Optional] The input
optimization options. The
default value is BM_OPT_LAYER_GROUP_WITH_WEIG
Below are the values for opt.
value
Description
OPT_NONE
No optimization
BM_OPT_LAYER_GROUP
Divides layers into clusters to optimize the bandwidth overhead.
BM_OPT_LAYER_GROUP_WITH_WEIG
Add additional optimization to reduce the device memory footprint and reshape weight.

CaffeBuilder::store_prototxt

store the optimized network graph as a file.
Parameter
Type
Description
dst
const char*
[Required] File to be stored

CaffeBuilder::store_model

1
void CaffeBuilder::store_model(
2
const char* net_name ,
3
const char* dst,
4
onst char* plugin_path=nullptr)
Copied!
Store compiled instructions, weight and other information of the network as a bmodel file.
Parameter
Type
Description
net_name
const char*
[Required] the network name.
dst
const char*
[Required] File to store bmodel.
Plugin_path
const char*
[Required] cpu op plugins.

CaffeBuilder::addCustomizedLayer

1
void CaffeBuilder::addCustomizedLayer( Layer* layer)
Copied!
Register a new added customized layer, which used to convert CAFFE layer into BMNet IR (Intermediate representation).
Parameter
Type
Description
Layer
Layer*
[Required] pointer to instance of Class Layer

CaffeBuilder::addCustomizedTensorInst

1
void CaffeBuilder::addCustomizedTensorInst(TensorInst* inst)
Copied!
Register a new added customized TensorInst (Tensor Instruction), which used to convert BMNet IR into instructions.
Parameter
Type
Description
inst
TensorInst*
[Required] pointer to instance of Class Layer
Last modified 2yr ago