Caffe

Network Configuration
TensorOp
TensorOp represents a BMNET IR, which is a bridge between front end and back end. it provides lots of member method to set information to or get from it. Below is the prototype:
1
namespace bmnet { 
2
class TensorOp {
3
public:
4
  int input_shape_size();
5
  int output_shape_size();
6
  const TensorShape& input_shape(int index);
7
  const TensorShape& output_shape(int index);
8
  TensorShape* add_output_shape();
9
  u64 global_input(int index);
10
  u64 global_output(int index);
11
  TGCustomizedParameter* mutable_tg_customized_param(); 
12
  const TGCustomizedParameter& tg_customized_param();
13
};
14
}
Copied!
TensorOp::input_shape_size
1
void TensorOp::input_shape_size()
Copied!
Return the number of inputs.
TensorOp::output_shape_size
1
void TensorOp::output_shape_size()
Copied!
Return the number of outputs.
TensorOp::input_shape
1
const TensorShape& TensorOp::input_shape( int index)
Copied!
Return shape of input by index.
Parameter
Type
Description
index
int
[Required] index of input that to be returned.
TensorOp::output_shape
1
const TensorShape& TensorOp::output_shape(int index)
Copied!
Return shape of output by index. 
Parameter
Type
Description
index
int
[Required] index of output that to be returned.
TensorOp::add_output_shape
1
TensorShape* TensorOp::add_output_shape()
Copied!
Return a mutable pointer to a new added TensorShape of outputs. The returned TensorShape could be modified latter.
TensorOp::global_input
1
u64 TensorOp::global_input(int index) 
Copied!
Return offset of input tensor by index, while it was stored in device memory.
Parameter
Type
Description
index
int
[Required] index of input that to be returned.
TensorOp::global_output
1
u64 TensorOp::global_output(int index) 
Copied!
Return offset of output tensor by index, while it was stored in device memory.
Parameter
Type
Description
index
int
[Required] index of output that to be returned.
TensorOp::mutable_tg_customized_param
1
TGCustomizedParameter* TensorOp::mutable_tg_customized_param() 
Copied!
Return a mutable pointer to parameters of customized BMNET IR.
TensorOp::tg_customized_param
1
const TGCustomizedParameter& TensorOp::tg_customized_param() 
Copied!
Return reference of customized BMNET IR’s paramters.
CustomizedCaffeLayer
CustomizedCaffeLayer is abstract class, which is used to implement a Layer to convert CAFFE Layer into BMNet IR(please refer to Chapter 5 for details about BMNet IR). If you want to introduce a customized CAFFE layer into BMNet, please inherit this class and implement all pure virtual functions of it. The CustomizedCaffeLayer inherits from CaffeLayer/Layer class. Below are the prototypes of them:
1
namespace bmnet {
2
class Layer {
3
 public:
4
   Layer();
5
   virtual  ~Layer(void);
6
   virtual  std::string layer_name() = 0;
7
   virtual  void dump () = 0;
8
   virtual  void codegen(TensorOp *op) = 0;
9
​
10
protected:
11
   void add_output_offset(int offset);
12
};
13
} 
14
​
15
namespace bmnet {
16
​
17
class CaffeLayer : public Layer { 
18
public:
19
   CaffeLayer(){}
20
   virtual ~CaffeLayer(void); 
21
protected:
22
   caffe::LayerParameter &layer_; 
23
};
24
} 
25
​
26
namespace bmnet {
27
​
28
class CustomizedCaffeLayer : public CaffeLayer { 
29
public:
30
   CustomizedCaffeLayer(); 
31
   ~CustomizedCaffeLayer();
32
   void setup(TensorOp* op) override {
33
 ... 
34
 ...
35
     TGCustomizedParameter* param = op->mutable_tg_customized_param ();
36
     param->set_sub_type(layer_name()); 
37
    }
38
};
39
}
Copied!
CustomizedCaffeLayer::layer_name
1
std::string CustomizedCaffelayer::layer_name() 
Copied!
Pure virtual function, return type of new added CAFFE layer.
CustomizedCaffeLayer::dump
1
Pure virtual function, is used to print information of CAFFE Layer.
Copied!
CustomizedCaffeLayer:: setup
1
void CustomizedCaffelayer::setup() 
Copied!
Option. It is used to set sub type of Customized Layer only. Implement by default. If child class will override it, this parent class setup function must be call first.
CustomizedCaffeLayer::codegen
Pure virtual function, is used to setup BMNET IR according to LayerParameter of CAFFE Layer. In this function, you should setup output shape and fill parameters to TensorOp.
1
TBD
Copied!
Parameter
Type
Description
op
TensorOp*
[Required] pointer to a instance of BMNET IR
CustomizedCaffeLayer::add_output_offset
1
void CustomizedCaffelayer::add_output_offset (int offset)
Copied!
Protected member method, should be called when setup output offset of Layer’s top.
Parameter
Type
Description
offset
int
[Required] offset of output, should be 0.
CustomizedCaffeLayer::layer_
1
caffe::LayerParameter CustomizedCaffelayer::&layer_ 
Copied!
Protected member variable, which is reference of customized CAFFE layer’s LayerParameter.
CustomizedTensorFixedInst
CustomizedTensorFixedInst is abstract class, which is used to implement a Layer to convert BMNET IR into instructions by BMKernel APIs. Please inherit this class and implement all pure virtual functions of it. The CustomizedTensorFixedInst inherits from TensorFixedInst/ TensorInst class. Below are the prototypes of them:
1
namespace bmnet {
2
class TensorFixedInst: public TensorInst { 
3
public:
4
  TensorFixedInst() : TensorInst() {} 
5
  TensorFixedInst(TensorOp &op) : TensorInst(op) {} 
6
  virtual ~ TensorFixedInst (void);
7
  void SetCalibrationParameter(
8
     const LayerCalibrationParameter &calibration_parameter) { 
9
	m_calibrationParameter = calibration_parameter;
10
}
11
  void AddInputCalibrationParameter(
12
     const LayerCalibrationParameter &calibration_parameter){  
13
	m_inputCalibrationParameter.push_back(calibration_parameter);
14
} 
15
protected:
16
  LayerCalibrationParameter m_calibrationParameter; 
17
  std::vector <LayerCalibrationParameter >
18
     m_inputCalibrationParameter;
19
};
20
}
21
​
22
namespace bmnet {
23
class TensorInst {
24
public:
25
  TensorInst();
26
  virtual ~TensorInst(void);
27
  virtual std::string inst_name() = 0;
28
  virtual void dump () = 0;
29
  virtual void encode () = 0;
30
​
31
protected:
32
  TensorOp &op_; 
33
};
34
}
35
​
36
namespace bmnet {
37
​
38
class CustomizedTensorFixedInst : public TensorFixedInst { 
39
public:
40
  CustomizedTensorFixedInst ();
41
  ~CustomizedTensorFixedInst (); 
42
protected:
43
  u64 get_global_neuron_base();
44
  u64 get_global_weight_base();
45
}; 
46
}
47
​
Copied!
CustomizedTensorFixedInst::inst_name
1
std::string CustomizedTensorFixedInst::inst_name() 
Copied!
Pure virtual function, return type of customized BMNET IR.
CustomizedTensorFixedInst::dump
1
void CustomizedTensorFixedInst::dump() 
Copied!
Pure virtual function, is used to print information of BMNET IR.
CustomizedTensorFixedInst::encode
1
void CustomizedTensorFixedInst::encode()
Copied!
Pure virtual function, is used to convert BMNET IR into instructions using BMKernel APIs.
CustomizedTensorFixedInst::get_global_neuron_base
1
u64 CustomizedTensorFixedInst::get_global_neuron_base() 
Copied!
Protected member method, return the base address, where the neurons are stored in device memory.
CustomizedTensorFixedInst::get_global_weight_base
1
u64 CustomizedTensorFixedInst::get_global_weight_base() 
Copied!
Protected member method, return the base address, where weight is stored in device memory.
CustomizedTensorFixedInst::op_
1
TensorOp CustomizedTensorFixedInst::&op_ 
Copied!
Protected member variable, which is reference of BMNET IR.
TGCustomizedParamter
TGCustomizedParamter represents a customized BMNET IR’s parameters. It provides member methods to set parameters to or get from it. Below is the prototype:
1
namespace bmnet {
2
​
3
class TGCustomizedParameter {
4
public:
5
  int i32_param_size();
6
  int f32_param_size();
7
  int i32_param(int index);
8
  float f32_param(int index);
9
  void add_i32_param(int value);
10
  void add_f32_param(float value);
11
};
12
}
Copied!
TGCustomizedParamter::i32_param_size
1
void TGCustomizedParamter::i32_param_size() 
Copied!
Return the number of int parameters, which stored in TGCustomizedParamter.
TGCustomizedParamter::f32_param_size
1
void TGCustomizedParamter::f32_param_size() 
Copied!
Return the number of float parameters, which stored in TGCustomizedParamter.
TGCustomizeParamter::i32_param
1
int TGCustomizedParamter::i32_param(int index) 
Copied!
Return int parameter by index.
Parameter
Type
Description
index
index
[Required] index of int parameter that to be returned.
TGCustomizeParamter::f32_param
1
float TGCustomizedParamter::f32_param( int index) 	
Copied!
Return int parameter by index.
Parameter
Type
Description
index
index
[Required] index of float parameter that to be returned.
TGCustomizeParamter::add_i32_param
1
void TGCustomizedParamter::add_i32_param(int value)
Copied!
Append a new int parameter to TGCustomizedParamter.
Parameter
Type
Description
value
int
[Required]  int parameter.
TGCustomizeParamter::add_f32_param
1
void TGCustomizedParamter::add_f32_param(int value)
Copied!
Append a new int parameter to TGCustomizedParamter.
Parameter
Type
Description
value
float
[Required]  float parameter.
TensorShape
TensorShape represents a shape of tensor. Below is the prototype:
1
namespace bmnet {
2
​
3
class TensorShape { 
4
public:
5
  void CopyFrom(const TensorShape& from); 
6
  int dim_size() const;
7
  int dim(int index);
8
  void add_dim(int value);
9
};
10
}
Copied!
TensorShape::dim_size
1
int TensorShape::dim_size()
Copied!
Return the number of dims.
TensorShape::dim
1
int TensorShape::dim(int index) 
Copied!
Return one dim by index.
Parameter
Type
Description
Index
int
[Required]  index of dim that to be returned.
TensorShape::add_dim
1
void TensorShape::add_dim(int value)
Copied!
Append a dim to TensorShape.
Parameter
Type
Description
value
int
[Required]  new dim to be appended.
TensorShape::CopyFrom
1
void TensorShape::CopyFrom(const TensorShape& from)
Copied!
Copy from another TensorShape instance.
Parameter
Type
Description
value
const TensorShape&
[Required]  source TensorShape instance.
CaffeBuilder
CaffeBuilder is a class, which provides a uniform interface to combine front end/optimizer/back end core code into one, to compile CAFFE neuron network graph into bmodel file. The CaffeBuilder inherits from Builder class, which is a base compiler class. Below are the prototypes of them:
1
namespace bmnet { 
2
​
3
class Builder {
4
public:
5
  Builder(CHIP_VER ver);
6
  virtual ~Builder();
7
  void addCustomizedTensorInst(TensorInst *inst);
8
  void build(int n, int c, int h, int w, int opt);
9
  void store_prototxt(const char *dst);
10
  void store_model(const char *net_name, const char *dst);
11
};
12
}
Copied!
1
namespace bmnet { 
2
​
3
class CaffeBuilder : public Builder { 
4
public:
5
  CaffeBuilder(CHIP_VER ver, const char *modified_proto , const char *caffemodel , const char *weight_bin , const char *in_ctable , const char *out_ctable);
6
  ~CaffeBuilder();
7
  void addCustomizedLayer(Layer *layer);
8
};
Copied!
CaffeBuilder::CaffeBuilder
1
CaffeBuilder::CaffeBuilder(
2
CHIP_VER ver ,
3
const char *modified_proto ,
4
const char *caffemodel ,
5
const char *weight_bin ,
6
const char *in_ctable ,
7
const char *out_ctable)
Copied!
Constructor function of CaffeBuilder class.
Parameter
Type
Description
ver
CHIP_VER
[Required]  The target chip version. Currently only BM_CHIP_BM1880 is
available.
modified_proto
const char*
[Optional] The modified prototxt file, please refer Chapter 4 to get more detail.
caffemodel
const char*
[Required] The specified caffemode file of network
weight_bin
const char*
[Optional] The specified weight file of network
in_ctable
const char*
[Required] The specified calibration table file of network
out_ctable
const char*
[Required] The specified weight file of network
 modified_proto are optional parameters, that means you no need to fill all of this parameters. Below combination are valid: 1) caffemodel only; 2) caffemodel, as well as modified_protos
CaffeBuilder::Builder
Core member function of CaffeBuilder class, used to compile the network by specifying input shape and optimization level.
Parameter
Type
Description
n,c,h,w
int
[Required]  The input shape
opt
int
[Optional] The input optimization options. The default value is BM_OPT_LAYER_GROUP_WITH_WEIG
Below are the values for opt.
value
Description
OPT_NONE
No optimization 
BM_OPT_LAYER_GROUP
Divides layers into clusters to optimize the bandwidth overhead.
BM_OPT_LAYER_GROUP_WITH_WEIG
Add additional optimization to reduce the device memory footprint and reshape weight.
CaffeBuilder::store_prototxt
store the optimized network graph as a file.
Parameter
Type
Description
dst
const char*
[Required]  File to be stored
CaffeBuilder::store_model
1
void CaffeBuilder::store_model(
2
const char* net_name ,
3
const char* dst,
4
onst char* plugin_path=nullptr)
Copied!
Store compiled instructions, weight and other information of the network as a bmodel file.
Parameter
Type
Description
net_name
const char*
[Required]  the network name.
dst
const char*
[Required]  File to store bmodel.
Plugin_path
const char*
[Required]  cpu op plugins.
CaffeBuilder::addCustomizedLayer
1
void CaffeBuilder::addCustomizedLayer( Layer* layer)
Copied!
Register a new added customized layer, which used to convert CAFFE layer into BMNet IR (Intermediate representation).
Parameter
Type
Description
Layer
Layer*
[Required]  pointer to instance of Class Layer
CaffeBuilder::addCustomizedTensorInst
1
void CaffeBuilder::addCustomizedTensorInst(TensorInst* inst)
Copied!
Register a new added customized TensorInst (Tensor Instruction), which used to convert BMNet IR into instructions.
Parameter
Type
Description
inst
TensorInst*
[Required]  pointer to instance of Class Layer