# Neural Network Stick ## Introduction ### Product Overview The Bitmain Sophon Neural Network Stick (NNS) a fan less USB stick that designed for Deep Learning inference on various edge application. NNS is powered by high performance, low power Sophon BM1880 chip. BM1880 chip supports DNN/CNN/RNN/LSTM models or uniquely trained networks, and can perform facial detection, recognition, facial expression analysis, object detection, recognition, vehicle license plate recognition, voiceprint recognition, etc. The NNS enables traditional product with AI functions, and can be used in smart IPC, Robots, industrial PC, etc. ![Neural Network Stick (NNS)](https://2115705518-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LOME7sTpZZ0QmlqOlnw%2F-LPo5H6y1LGeJqyE4Sq6%2F-LPo5LAGddH2YdeJufPr%2Fqq.jpg?alt=media\&token=af39e5ac-806a-4e04-b6d0-1fbc32874182) ### Product Features * Power and all data provided over a single USB type A port * Supports DNN/CNN/RNN/LSTM models profiling, compiling and tuning * Real-time inference in edge device * Quickly deploy existing DNN/CNN/RNN/LSTM models or uniquely trained networks * Features Bitmain SophonTM BM1880 with energy efficient DNN/CNN/RNN/LSTM processing ### Target AI functions * Facial detection (Frame by Frame), recognition and expression analysis, such as Age, Gender, etc. * Human attributes and pose analysis * Object detection and recognition * Vehicle license plate recognition * Voice print recognition ### Product Technical Specification | Processor | Sophon BM1880 | | ------------------------------------ | ------------------------------------------------------------------------- | | Supported Framework | Caffe, ONNX, Tensorflow, Pytorch | | Supported AI Models | ResNet50, Yolo V2, GoogleNet V1, MobileNet v1\&v2, SSD300, AlexNet, VGG16 | | H.264 decoder, MJPEG encoder/decoder | 1x 1080p @60fps or 2x 1080p @30fps H.264 decoder, 75fps for FHD images | | Connectivity | USB3.0/USB2.0 slave mode, Type A | | Dimensions | 93.5*27*15mm | | Operating environmental temperature | 0 – 40C (commercial level) | | Hot plugin/plugoff | Yes | | Certification | EMC/FCC/3C/CE | | Minimum system requirement | X86\_64 computer running Ubuntu 16.04 USB3.0 or USB2.0 port | ### Toolkit * BMNet: Bitmain Compiler which can convert supported AI models to internal format accelerated by Sophon TPU. * ONNX: Compiler which can convert ONNX format to internal format accelerated by Sophon TPU. * Quantization Tool: Convert FP32 to INT8 and support calibration function. ## Get Started Setup the Neural Network Stick (NNS) by following the steps provided below. ### Step 1: Prepare Your Equipment In USB mode, BMNNSDK is installed on the X86 host, and calling NNS’s computing for deep learning, the environment requirement are as follows. ![](https://2115705518-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LOME7sTpZZ0QmlqOlnw%2F-LPjXIA_q2Gy4i_FLYlP%2F-LPjXciMLJ4Sw7rDyJ_k%2F1122.jpg?alt=media\&token=5173e130-34f6-4e51-92b9-0d93df9ac4ad) * Hardware environment: X86 host with usb type-A * Bitmain Sophon Neural Network Stick (NNS) * Operating system: Ubuntu\* 16.04 operating system, or Ubuntu VirtualBox instance * Package: libgoogle-glog-dev、libboost-all-dev、libprotobuf-dev, libusb-1.0-0-dev * Bitmain Sophon Neural Network Software Development Kit (SDK), such as **bmtap2-bm1880-usb-x.y.z.tar.gz** you can download the compression package from ![](https://2115705518-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LOME7sTpZZ0QmlqOlnw%2F-LU4fwjIKZZlQCLt4Kqr%2F-LU4gR1X4OTsCKTUkrhs%2F%E5%B1%8F%E5%B9%95%E5%BF%AB%E7%85%A7%202018-12-19%20%E4%B8%8B%E5%8D%884.59.02.png?alt=media\&token=9aff53d5-4e48-4a7b-b092-c992102610cb) ### Step 2:Install the BMNNSDK You can execute below commands to install package ```bash $ sudo apt-get install libgoogle-glog-dev libboost-all-dev libprotobuf-dev libusb-1.0-0-dev ``` There are two ways to install BMNSDK, you can have a choice #### Compression package installation Compression package include installation script, please copy the compression package to x86, unzip and execute the script install.sh. ```bash $ tar zxvf bmtap2-bm1880-usb-x.y.z.tar.gz $ cd bmtap2-bm1880-usb-x.y.z $ sudo ./install.sh ``` The script will install the BMNNSDK to default directory: /opt/bmtap2, and Symbolic Link will be created in the system content: ```bash // you will see the installation folder in the/opt/bmtap2 after successful installation $ ls /opt/bmtap2 // symbolic link will be found in /usr/bin、/usr/lib、/usr/include $ ls -alh /usr/bin/bm* $ ls –alh /usr/lib/bm* $ ls –alh /usr/include/bmtap2 // if the host has BM1880 NNS，you will find device fold in the directory/dev $ ls /dev/ttyACM* /dev/ttyACM0 ``` To uninstall the BMNNSDK ,simply execute the uninstall script in the installation directory. ```bash $ sudo /opt/bmtap2/bm1880-usb-x.y.z/uninstall.sh ``` #### Step 3: Basic function testing After installation the SDK, you can estimate whether the software and hardware environment is normal by running the test program in the installation directory(the file system needs to support bash for script execution correctly). The Basic function testing use the network model and data include in the installation package for Neural Network inference, the test results will be output to the out subdirectory of the current directory, please make sure that the user has the write rights to the current directory. ```bash $ source /opt/bmtap2/bm1880--x.y.z /test/test_bmnet.sh $ mkdir out $ run_test_bmnet_bm1880 ``` The test results are as follow: ``` # run_test_bmnet_bm1880 begin to test net: mtcnn det1 <1,3,12,12> test_bmnet_bmodel /opt/bmtap2/bm1880-usb_1.0.1/test/../models/mtcnn/det1_input_1_3_12_12.bin /opt/bmtap2/bm1880-usb_1.0.1/test/../models/mtcnn/det1_1_3_12_12.bmodel out//det1_output.bin 1 3 12 12 input size: 432 WARNING: Logging before InitGoogleLogging() is written to STDERR I1026 19:41:03.865362 2635 bm_firmware.cpp:76] check firmware status ... I1026 19:41:03.865628 2635 bm_firmware.cpp:82] firmware loading ... I1026 19:41:03.929674 2635 bm_firmware.cpp:47] firmware load success I1026 19:41:08.937716 2635 bm_firmware.cpp:98] firmware is running I1026 19:41:08.937757 2635 bm_device.cpp:104] device[0] opened,gmem_size : 0x40000000 output size:6 outputs[0]: [1,4,1,1], "conv4-2" outputs[1]: [1,2,1,1], "conv4-1" test_bmnet_bmodel: load 107 us, run 106 us, read 76 us ('/opt/bmtap2/bm1880-usb_1.0.1/test/../models/mtcnn/det1_output_1_3_12_12_ref.bin', ' vs. ', 'out//det1_output.bin') Result matched @ epsilon = 0.000000 det1<1,3,12,12> passed begin to test net: mobilenet mobilenet <1,3,224,224> test_bmnet_bmodel /opt/bmtap2/bm1880-usb_1.0.1/test/../models/mobilenet/mobilenet_input_1_3_224_224.bin /opt/bmtap2/bm1880-usb_1.0.1/test/../models/mobilenet/mobilenet_1_3_224_224.bmodel out//mobilenet_output.bin 1 3 224 224 input size: 150528 WARNING: Logging before InitGoogleLogging() is written to STDERR I1026 19:41:09.092887 2662 bm_firmware.cpp:76] check firmware status ... I1026 19:41:09.093133 2662 bm_firmware.cpp:85] firmware is running I1026 19:41:09.093152 2662 bm_device.cpp:104] device[0] opened,gmem_size : 0x40000000 output size:1000 outputs[0]: [1,1000,1,1], "fc7" test_bmnet_bmodel: load 3773 us, run 2820 us, read 222 us ('/opt/bmtap2/bm1880-usb_1.0.1/test/../models/mobilenet/mobilenet_output_1_3_224_224_ref.bin', ' vs. ', 'out//mobilenet_output.bin') Result matched @ epsilon = 0.000000 mobilenet<1,3,224,224> passed test_bmnet_bm1880 success ``` ##