Neural Network Module

Introduction

‌Product Overview

‌The Bitmain Sophon Neural Network Module (NNM) is a USB module that designed for Deep Learning inference on various edge application. NNM is powered by high performance, low power Sophon BM1880 chip. BM1880 chip supports DNN/CNN/RNN/LSTM models or uniquely trained networks, and can perform facial detection, recognition, facial expression analysis, object detection, recognition, vehicle license plate recognition, voiceprint recognition, etc. The NNM enables traditional product with AI functions, and can be used in smart IPC, Robots, industrial PC, etc.

Product Features

• Power and all data provided over USB 9pin (pin width: 1.25mm) interface

• Supports DNN/CNN/RNN/LSTM models profiling, compiling and tuning

• Real-time inference in edge device Quickly deploy existing

• DNN/CNN/RNN/LSTM models or uniquely trained networks Features

• Sophon BM1880 with energy efficient DNN/CNN/RNN/LSTM processing

Target AI Functions

• Facial detection (Frame by Frame), recognition and expression analysis, such as Age, Gender, etc.

• Human attributes and pose analysis

• Object detection and recognition

• Vehicle license plate recognition

• Voice print recognition

Product Technical Specification

Processor	Sophon BM1880
Supported Framework	Caffe, ONNX, Tensorflow, Pytorch
Supported AI Models	ResNet50, Yolo V2, GoogleNet V1, MobileNet v1&v2, SSD300, AlexNet, VGG16
H.264 decoder, MJPEG encoder/decoder	1x 1080p @60fps or 2x 1080p @30fps H.264 decoder, 75fps for FHD images
Connectivity	USB 9pin (pin width: 1.25mm) interface
Dimensions	38*38 mm
Operating environmental temperature	0 – 40C (commercial level)
Hot plugin/plugoff	Yes
Certification	EMC/FCC/3C/CE
Minimum system requirement	X86_64 computer running Windows or Ubuntu USB3.0 or USB2.0 port

Toolkit

• BMNet: Bitmain Compiler which can convert supported AI models to internal format accelerated by Sophon TPU.

• ONNX: Compiler which can convert ONNX format to internal format accelerated by Sophon TPU.

• Quantization Tool: Convert FP32 to INT8 and support calibration function.

Get Started

You can freely download BM1880 BSP/SDK package and demo programs at Sophon BM1880 GitHub https://github.com/BM1880-BIRD/.‌

In following sections, we will guide you how setup hardware and software environment, and how to build and run demo programs. Please check Sophon BM1880 GitHub for more information.‌

Setup Neural Network Module

Setup Neural Network Module by the steps provided below.‌

Step 1: Preparation

Before you start, make sure you have the following:‌

• Host PC with Ubuntu 18.04 operation system & OpenCV 3.4.0 & Python 2.7 & Python 3 & libusb & libgoogle-glog

• BM1880 SDK from https://github.com/BM1880-BIRD/bm1880-system-sdk​

• Demo programs from https://github.com/BM1880-BIRD/bm1880-ai-demo-program​

• Neural Network Module with USB cable

• 3 pins USB-to-UART cable

• Serial communication program (minicom) at host PC

• UVC Camera on host PC

Step 2: Hardware Installation

Step 2.1: Connect NNM and host PC with USB cable

Connect NNM to host PC via USB type A interface, host PC will identify NNM automatically, it will generate one USB device node named /dev/ttyACM0 if connection succeed. You can check it by below command:

$ ls -al /dev/ttyACM0

crw-rw---- 1 root dialout 166, 0 十二 12 18:03 /dev/ttyACM0

Install pyusb on host PC. It will download image to NNM by python script later.

$ sudo apt-get install python-pip
$ pip install pyusb

Note (Ubuntu18.04 ONLY, JUST DO IT ONCE):

Please

1. Add Bitmain USB PID/VID to filter list of modem manager using the following command:

sudo vim /lib/udev/rules.d/78-mm-whitelist-internal-modem.rules

ACTION!="add|change|move", GOTO="mm_whitelist_internal_modem_end"
ATTRS{idVendor}=="30b1", ATTRS{idProduct}=="1000", ENV{ID_MM_DEVICE_IGNORE}="1"
LABEL="mm_whitelist_internal_modem_end"
ACTION!="add|change|move", GOTO="mm_whitelist_internal_modem_end"
ATTRS{idVendor}=="30b1", ATTRS{idProduct}=="1001", ENV{ID_MM_DEVICE_IGNORE}="1"
LABEL="mm_whitelist_internal_modem_end"
ACTION!="add|change|move", GOTO="mm_whitelist_internal_modem_end"
ATTRS{idVendor}=="30b1", ATTRS{idProduct}=="1003", ENV{ID_MM_DEVICE_IGNORE}="1"
LABEL="mm_whitelist_internal_modem_end"
ACTION!="add|change|move", GOTO="mm_whitelist_internal_modem_end"
ATTRS{idVendor}=="0559", ATTRS{idProduct}=="1000", ENV{ID_MM_DEVICE_IGNORE}="1"
LABEL="mm_whitelist_internal_modem_end"

sudo udevadm control --reload

sudo udevadm trigger

Or

2. Stop modem manager

sudo systemctl stop ModemManager.service

Or

3. Disable modem manager (permanently) to avoid it restarting after a reboot

sudo systemctl disable ModemManager.service

Step 2.2: Plug UART cable to NNM

To show messages from NNM, plug UART cable to NNM as below. And then connect USB-to-UART cable to host PC.

‌

Setup Com port with below configurations:

115200-8-N-1 no hardware flow control

Step 2.3: Plug USB cable to NNM

Step 3‌: Build image for NNM

Go to https://github.com/BM1880-BIRD/bm1880-system-sdk and follow the README to build image on host PC for NNM.

Step 4: Download image to NNM

Now we can start to download image to BM1880 USB module.

‌$ cd bm1880/bm1880-system-sdk/install/soc_bm1880_asic_nnm/bm1880_emmc_dl_v1p1
$ sudo python bm1880_emmc_download.py

It will show:

bm1880_emmc_download v1.1
Connecting to ROM code...
Waiting for BM1880 USB port:  |

‌

Re-plug NNM into host PC, and it will start downloading.

bm1880_emmc_download v1.1
Connecting to ROM code...
/dev/ttyACM0BM1880 USB port:  \
USB VID:PID=30B1:1000 SER=123456789ABC LOCATION=1-2:1.0

bm_dl_magic.bin is 128 bytes
Send to address 0x4003000
--- 0.01 Seconds ---
Done
Sending prg.bin...
prg.bin is 79728 bytes
Send to address 0x4003000
--- 3.17 Seconds ---
Done
Jumping to prg.bin...
/dev/ttyACM0BM1880 USB port:  \
USB VID:PID=30B1:1000 SER=123456789ABC LOCATION=2-3:1.0

Done
Sending FIP.bin...
fip.bin is 462543 bytes
Send to address 0x8003000
--- 3.42 Seconds ---
Done
Downloading FIP.bin to eMMC then reboot to uboot...
/dev/ttyACM0BM1880 USB port:  /
USB VID:PID=30B1:1001 SER=123456789ABC LOCATION=2-3:1.0

Done
Sending ramboot_mini.itb...
ramboot_mini.itb is 7210918 bytes
Send to address 0x10f100000
--- 0.14 Seconds ---
Done
Booting linux in ramboot_mini.itb...
Doneing for BM1880 USB port:  \
Sending emmc.tar.gz...
emmc.tar.gz is 377074923 bytes
Send to address 0x4003000
--- 14.58 Seconds ---
Done
Processing emmc.tar.gz and then reboot...
It takes about 150 seconds, please wait...
/dev/ttyACM0BM1880 USB port:  |
USB VID:PID=30B1:1000 SER=123456789ABC LOCATION=1-2:1.0

verify id = ['VID:PID=30B1:1003']
/dev/ttyACM0BM1880 USB port:  |
USB VID:PID=30B1:1003 SER=0123456789 LOCATION=2-3:1.1
Done
Download complete
--- 285.29 Seconds ---

It can check with BM1880 UART output.

Starting kernel ...

[    0.000000] [0] Booting Linux on physical CPU 0x0
[    0.000000] [0] Linux version 4.9.38 (bmuser@ubuntu) (gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05) ) #1 SMP Thu Sep 19 16:13:42 CST 2019
[    0.000000] [0] Boot CPU: AArch64 Processor [410fd034]
[    0.000000] [0] OF: fdt:earlycon: stdout-path serial0 not found
[    0.000000] [0] efi: Getting EFI parameters from FDT:
[    0.000000] [0] efi: UEFI not found.
[    0.000000] [0] OF: reserved mem: allocated memory for 'ion' node: base 0x0000000128000000, size 384 MiB
[    0.000000] [0] Ion: Ion memory setup at 0x0000000128000000 size 384 MiB
[    0.000000] [0] OF: reserved mem: initialized node ion, compatible id ion-region
[    0.000000] [0] OF: reserved mem: allocated memory for 'jpu' node: base 0x0000000127000000, size 16 MiB
[    0.000000] [0] OF: reserved mem: allocated memory for 'vpu' node: base 0x0000000123000000, size 64 MiB
[    0.000000] [0] OF: reserved mem: allocated memory for 'linux,cma' node: base 0x0000000122000000, size 16 MiB
[    0.000000] [0] Reserved memory: created CMA memory pool at 0x0000000122000000, size 16 MiB
[    0.000000] [0] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] [0] crashkernel reserved: 0x000000011d000000 - 0x0000000122000000 (80 MB)
[    0.000000] [0] On node 0 totalpages: 241664
[    0.000000] [0]   DMA zone: 3776 pages used for memmap
[    0.000000] [0]   DMA zone: 0 pages reserved
[    0.000000] [0]   DMA zone: 241664 pages, LIFO batch:31
[    0.000000] [0] psci: probing for conduit method from DT.
[    0.000000] [0] psci: PSCIv1.0 detected in firmware.
[    0.000000] [0] psci: Using standard PSCI v0.2 function IDs
[    0.000000] [0] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] [0] percpu: Embedded 22 pages/cpu @ffffffc01cf28000 s50456 r8192 d31464 u90112
[    0.000000] [0] pcpu-alloc: s50456 r8192 d31464 u90112 alloc=22*4096
[    0.000000] [0] pcpu-alloc: [0] 0 [0] 1
[    0.000000] [0] Detected VIPT I-cache on CPU0
[    0.000000] [0] CPU features: enabling workaround for ARM erratum 845719
[    0.000000] [0] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 237888
[    0.000000] [0] Kernel command line: console=ttyS0,115200 earlycon debug user_debug=31 loglevel=10 no_console_suspend crashkernel=512M-1G:80M,1G-:128M
[    0.000000] [0] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] [0] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.000000] [0] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.000000] [0] Memory: 444384K/966656K available (6332K kernel code, 828K rwdata, 1780K rodata, 704K init, 369K bss, 505888K reserved, 16384K cma-reserved)
[    0.000000] [0] Virtual kernel memory layout:
[    0.000000] [0]     modules : 0xffffff8000000000 - 0xffffff8008000000   (   128 MB)
[    0.000000] [0]     vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000   (   250 GB)
[    0.000000] [0]       .text : 0xffffff8008080000 - 0xffffff80086b0000   (  6336 KB)
[    0.000000] [0]     .rodata : 0xffffff80086b0000 - 0xffffff8008870000   (  1792 KB)
[    0.000000] [0]       .init : 0xffffff8008870000 - 0xffffff8008920000   (   704 KB)
[    0.000000] [0]       .data : 0xffffff8008920000 - 0xffffff80089ef200   (   829 KB)
[    0.000000] [0]        .bss : 0xffffff80089ef200 - 0xffffff8008a4b764   (   370 KB)
[    0.000000] [0]     fixed   : 0xffffffbefe7fd000 - 0xffffffbefec00000   (  4108 KB)
[    0.000000] [0]     PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000   (    16 MB)
[    0.000000] [0]     vmemmap : 0xffffffbf00000000 - 0xffffffc000000000   (     4 GB maximum)
[    0.000000] [0]               0xffffffbf00000000 - 0xffffffbf01000000   (    16 MB actual)
[    0.000000] [0]     memory  : 0xffffffc000000000 - 0xffffffc040000000   (  1024 MB)
[    0.000000] [0] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] [0] Hierarchical RCU implementation.
[    0.000000] [0]      Build-time adjustment of leaf fanout to 64.
[    0.000000] [0]      RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[    0.000000] [0] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=2
[    0.000000] [0] NR_IRQS:64 nr_irqs:64 0
[    0.000000] [0] GIC: Using split EOI/Deactivate mode
[    0.000000] [0] arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (phys).
[    0.000000] [0] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[    0.000003] [0] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[    0.000103] [0] Console: colour dummy device 80x25
[    0.000119] [0] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[    0.000128] [0] pid_max: default: 32768 minimum: 301
[    0.000200] [0] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.000208] [0] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.000611] [0] ftrace: allocating 21676 entries in 85 pages
[    0.028811] [0] ASID allocator initialised with 65536 entries
[    0.030066] [0] EFI services will not be available.
[    0.030370] [1] Detected VIPT I-cache on CPU1
[    0.030408] [1] CPU1: Booted secondary processor [410fd034]
[    0.030459] [0] Brought up 2 CPUs
[    0.030470] [0] SMP: Total of 2 processors activated.
[    0.030478] [0] CPU features: detected feature: 32-bit EL0 Support
[    0.030486] [0] CPU: All CPU(s) started at EL2
[    0.030499] [0] alternatives: patching kernel code
[    0.030874] [0] devtmpfs: initialized
[    0.043560] [0] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.043579] [0] futex hash table entries: 512 (order: 4, 65536 bytes)
[    0.044071] [0] pinctrl core: initialized pinctrl subsystem
[    0.044336] [0] NET: Registered protocol family 16
[    0.045022] [0] vdso: 2 pages (1 code @ ffffff80086b7000, 1 data @ ffffff8008924000)
[    0.045035] [0] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.045539] [0] DMA: preallocated 256 KiB pool for atomic allocations
[    0.046183] [0] clk reset: nr_reset=64 resource_size=8
[    0.049932] [0] bm-pinctrl 50010400.pinctrl: initialized Bitmain pin control driver
[    0.050660] [0] bm-eth-mux eth0-mux: set pin MUX
[    0.050675] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:4 group:4 #group:1 $pin:16
[    0.050683] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x54 = 0x1000100
[    0.050691] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x58 = 0x1000100
[    0.050698] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x58 = 0x1000100
[    0.050705] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x5c = 0x1000100
[    0.050713] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x5c = 0x1000100
[    0.050720] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x60 = 0x1000100
[    0.050727] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x60 = 0x1000100
[    0.050734] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x64 = 0x1000100
[    0.050742] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x64 = 0x1000100
[    0.050749] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x68 = 0x1000100
[    0.050756] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x68 = 0x1000100
[    0.050763] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x6c = 0x1000100
[    0.050770] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x6c = 0x1000100
[    0.050778] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x70 = 0x1000100
[    0.050785] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x70 = 0x1000100
[    0.050792] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x74 = 0x1000100
[    0.051018] [0] bm-eth-mux eth1-mux: set pin MUX
[    0.051030] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:132 group:132 #group:1 $pin:16
[    0.051038] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x74 = 0x1100100
[    0.051046] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x78 = 0x1000110
[    0.051053] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x78 = 0x1100110
[    0.051060] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x7c = 0x1000110
[    0.051067] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x7c = 0x1100110
[    0.051075] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x80 = 0x1000110
[    0.051082] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x80 = 0x1100110
[    0.051089] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x84 = 0x1000110
[    0.051096] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x84 = 0x1100110
[    0.051104] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x88 = 0x1000110
[    0.051111] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x88 = 0x1100110
[    0.051118] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x8c = 0x1000110
[    0.051125] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x8c = 0x1100110
[    0.051132] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x90 = 0x1000110
[    0.051140] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x90 = 0x1100110
[    0.051147] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x94 = 0x110
[    0.051369] [0] bm-i2s-mux i2s0-mux: set pin MUX
[    0.051380] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:133 group:133 #group:1 $pin:5
[    0.051387] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xcc = 0x200000
[    0.051395] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd0 = 0x20
[    0.051402] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd0 = 0x200020
[    0.051409] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd4 = 0x20
[    0.051416] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd4 = 0x200020
[    0.051642] [0] bm-i2s-mux i2s0-mclkin-mux: set pin MUX
[    0.051652] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:134 group:134 #group:1 $pin:1
[    0.051659] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xe0 = 0x100000
[    0.051887] [0] bm-i2s-mux i2s1-mux: set pin MUX
[    0.051897] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:135 group:135 #group:1 $pin:5
[    0.051905] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd8 = 0x20
[    0.051912] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xd8 = 0x200020
[    0.051919] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xdc = 0x20
[    0.051926] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xdc = 0x200020
[    0.051933] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xe0 = 0x100020
[    0.052160] [0] bm-i2s-mux i2s1-mclkin-mux: set pin MUX
[    0.052170] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:136 group:136 #group:1 $pin:1
[    0.052177] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xe4 = 0x10
[    0.062487] [0] bm-i2c-mux i2c0-mux: set pin MUX
[    0.062503] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:43 group:43 #group:1 $pin:2
[    0.062511] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0x9c = 0x100000
[    0.062519] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa0 = 0x10
[    0.062742] [0] bm-i2c-mux i2c1-mux: set pin MUX
[    0.062752] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:44 group:44 #group:1 $pin:2
[    0.062759] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa0 = 0x100010
[    0.062767] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa4 = 0x10
[    0.062986] [0] bm-i2c-mux i2c2-mux: set pin MUX
[    0.062996] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:45 group:45 #group:1 $pin:2
[    0.063004] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa4 = 0x100010
[    0.063011] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa8 = 0x10
[    0.063237] [0] bm-i2c-mux i2c3-mux: set pin MUX
[    0.063246] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:46 group:46 #group:1 $pin:2
[    0.063254] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xa8 = 0x100010
[    0.063262] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xac = 0x10
[    0.063484] [0] bm-i2c-mux i2c4-mux: set pin MUX
[    0.063494] [0] bm-pinctrl 50010400.pinctrl: bm_set_mux seletor:47 group:47 #group:1 $pin:2
[    0.063502] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xac = 0x100010
[    0.063509] [0] bm-pinctrl 50010400.pinctrl: check new reg val on offset 0xb0 = 0x300010
[    0.063744] [0] bm-gpio-mux gpio0-mux: initialized Bitmain gpio MUX driver
[    0.063976] [0] bm-gpio-mux gpio1-mux: initialized Bitmain gpio MUX driver
[    0.064208] [0] bm-gpio-mux gpio2-mux: initialized Bitmain gpio MUX driver
[    0.064444] [0] bm-gpio-mux gpio3-mux: initialized Bitmain gpio MUX driver
[    0.064678] [0] bm-gpio-mux gpio4-mux: initialized Bitmain gpio MUX driver
[    0.064930] [0] bm-gpio-mux gpio5-mux: initialized Bitmain gpio MUX driver
[    0.065166] [0] bm-gpio-mux gpio6-mux: initialized Bitmain gpio MUX driver
[    0.065408] [0] bm-gpio-mux gpio7-mux: initialized Bitmain gpio MUX driver
[    0.065645] [0] bm-gpio-mux gpio8-mux: initialized Bitmain gpio MUX driver
[    0.065884] [0] bm-gpio-mux gpio9-mux: initialized Bitmain gpio MUX driver
[    0.066128] [0] bm-gpio-mux gpio10-mux: initialized Bitmain gpio MUX driver
[    0.066376] [0] bm-gpio-mux gpio11-mux: initialized Bitmain gpio MUX driver
[    0.066618] [0] bm-gpio-mux gpio12-mux: initialized Bitmain gpio MUX driver
[    0.066867] [0] bm-gpio-mux gpio13-mux: initialized Bitmain gpio MUX driver
[    0.067115] [0] bm-gpio-mux gpio14-mux: initialized Bitmain gpio MUX driver
[    0.067368] [0] bm-gpio-mux gpio15-mux: initialized Bitmain gpio MUX driver
[    0.067618] [0] bm-gpio-mux gpio16-mux: initialized Bitmain gpio MUX driver
[    0.067869] [0] bm-gpio-mux gpio17-mux: initialized Bitmain gpio MUX driver
[    0.068121] [0] bm-gpio-mux gpio18-mux: initialized Bitmain gpio MUX driver
[    0.068375] [0] bm-gpio-mux gpio19-mux: initialized Bitmain gpio MUX driver
[    0.068633] [0] bm-gpio-mux gpio20-mux: initialized Bitmain gpio MUX driver
[    0.068902] [0] bm-gpio-mux gpio21-mux: initialized Bitmain gpio MUX driver
[    0.069164] [0] bm-gpio-mux gpio22-mux: initialized Bitmain gpio MUX driver
[    0.069424] [0] bm-gpio-mux gpio23-mux: initialized Bitmain gpio MUX driver
[    0.069685] [0] bm-gpio-mux gpio24-mux: initialized Bitmain gpio MUX driver
[    0.069944] [0] bm-gpio-mux gpio25-mux: initialized Bitmain gpio MUX driver
[    0.070206] [0] bm-gpio-mux gpio26-mux: initialized Bitmain gpio MUX driver
[    0.070468] [0] bm-gpio-mux gpio27-mux: initialized Bitmain gpio MUX driver
[    0.070734] [0] bm-gpio-mux gpio28-mux: initialized Bitmain gpio MUX driver
[    0.070995] [0] bm-gpio-mux gpio29-mux: initialized Bitmain gpio MUX driver
[    0.071266] [0] bm-gpio-mux gpio30-mux: initialized Bitmain gpio MUX driver
[    0.071530] [0] bm-gpio-mux gpio31-mux: initialized Bitmain gpio MUX driver
[    0.071801] [0] bm-gpio-mux gpio32-mux: initialized Bitmain gpio MUX driver
[    0.072069] [0] bm-gpio-mux gpio33-mux: initialized Bitmain gpio MUX driver
[    0.072345] [0] bm-gpio-mux gpio34-mux: initialized Bitmain gpio MUX driver
[    0.072622] [0] bm-gpio-mux gpio35-mux: initialized Bitmain gpio MUX driver
[    0.072909] [0] bm-gpio-mux gpio36-mux: initialized Bitmain gpio MUX driver
[    0.073185] [0] bm-gpio-mux gpio37-mux: initialized Bitmain gpio MUX driver
[    0.073455] [0] bm-gpio-mux gpio38-mux: initialized Bitmain gpio MUX driver
[    0.073733] [0] bm-gpio-mux gpio39-mux: initialized Bitmain gpio MUX driver
[    0.074008] [0] bm-gpio-mux gpio40-mux: initialized Bitmain gpio MUX driver
[    0.074284] [0] bm-gpio-mux gpio41-mux: initialized Bitmain gpio MUX driver
[    0.074561] [0] bm-gpio-mux gpio42-mux: initialized Bitmain gpio MUX driver
[    0.074841] [0] bm-gpio-mux gpio43-mux: initialized Bitmain gpio MUX driver
[    0.075122] [0] bm-gpio-mux gpio44-mux: initialized Bitmain gpio MUX driver
[    0.075406] [0] bm-gpio-mux gpio45-mux: initialized Bitmain gpio MUX driver
[    0.075687] [0] bm-gpio-mux gpio46-mux: initialized Bitmain gpio MUX driver
[    0.075973] [0] bm-gpio-mux gpio47-mux: initialized Bitmain gpio MUX driver
[    0.076256] [0] bm-gpio-mux gpio48-mux: initialized Bitmain gpio MUX driver
[    0.076543] [0] bm-gpio-mux gpio49-mux: initialized Bitmain gpio MUX driver
[    0.076843] [0] bm-gpio-mux gpio50-mux: initialized Bitmain gpio MUX driver
[    0.077133] [0] bm-gpio-mux gpio51-mux: initialized Bitmain gpio MUX driver
[    0.077429] [0] bm-gpio-mux gpio52-mux: initialized Bitmain gpio MUX driver
[    0.077721] [0] bm-gpio-mux gpio53-mux: initialized Bitmain gpio MUX driver
[    0.078020] [0] bm-gpio-mux gpio54-mux: initialized Bitmain gpio MUX driver
[    0.078314] [0] bm-gpio-mux gpio55-mux: initialized Bitmain gpio MUX driver
[    0.078615] [0] bm-gpio-mux gpio56-mux: initialized Bitmain gpio MUX driver
[    0.078914] [0] bm-gpio-mux gpio57-mux: initialized Bitmain gpio MUX driver
[    0.079215] [0] bm-gpio-mux gpio58-mux: initialized Bitmain gpio MUX driver
[    0.079516] [0] bm-gpio-mux gpio59-mux: initialized Bitmain gpio MUX driver
[    0.079825] [0] bm-gpio-mux gpio60-mux: initialized Bitmain gpio MUX driver
[    0.080128] [0] bm-gpio-mux gpio61-mux: initialized Bitmain gpio MUX driver
[    0.080438] [0] bm-gpio-mux gpio62-mux: initialized Bitmain gpio MUX driver
[    0.080739] [0] bm-gpio-mux gpio63-mux: initialized Bitmain gpio MUX driver
[    0.081070] [0] bm-gpio-mux gpio64-mux: initialized Bitmain gpio MUX driver
[    0.081374] [0] bm-gpio-mux gpio65-mux: initialized Bitmain gpio MUX driver
[    0.081685] [0] bm-gpio-mux gpio66-mux: initialized Bitmain gpio MUX driver
[    0.081996] [0] bm-gpio-mux gpio67-mux: initialized Bitmain gpio MUX driver
[    0.097513] [0] SCSI subsystem initialized
[    0.097642] [0] usbcore: registered new interface driver usbfs
[    0.097677] [0] usbcore: registered new interface driver hub
[    0.097705] [0] usbcore: registered new device driver usb
[    0.099219] [0] pps_core: LinuxPPS API ver. 1 registered
[    0.099226] [0] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[e-mail]>
[    0.099239] [0] PTP clock support registered
[    0.099523] [0] Ion: ion_parse_dt_heap_common: id 0 type 2 name carveout align 1000
[    0.099707] [0] Ion: rmem_ion_device_init: heap carveout base 0x0000000128000000 size 0x0000000018000000 dev ffffffc01b08f010
[    0.099716] [0] platform carveout: assigned reserved memory node ion
[    0.168896] [0] platform carveout: [ion] add heap id 0, type 2, base 0x128000000, size 0x18000000
[    0.169285] [0] bm_vpp_probe
[    0.169479] [0] vpp descriptor addr :pdes[0] : 0xffffff8008041000, des_paddr[0] 0x122040000
[    0.169496] [0] vpp descriptor addr :pdes[1] : 0xffffff8008043000, des_paddr[1] 0x122041000
[    0.169512] [0] vpp descriptor addr :pdes[2] : 0xffffff8008045000, des_paddr[2] 0x122042000
[    0.169528] [0] vpp descriptor addr :pdes[3] : 0xffffff8008051000, des_paddr[3] 0x122043000
[    0.169544] [0] vpp descriptor addr :pdes[4] : 0xffffff8008053000, des_paddr[4] 0x122044000
[    0.169560] [0] vpp descriptor addr :pdes[5] : 0xffffff8008055000, des_paddr[5] 0x122045000
[    0.169575] [0] vpp descriptor addr :pdes[6] : 0xffffff8008057000, des_paddr[6] 0x122046000
[    0.169592] [0] vpp descriptor addr :pdes[7] : 0xffffff8008059000, des_paddr[7] 0x122047000
[    0.169617] [0] vpp descriptor addr :pdes[8] : 0xffffff800805b000, des_paddr[8] 0x122048000
[    0.169639] [0] vpp descriptor addr :pdes[9] : 0xffffff800805d000, des_paddr[9] 0x122049000
[    0.169655] [0] vpp descriptor addr :pdes[10] : 0xffffff800805f000, des_paddr[10] 0x12204a000
[    0.169670] [0] vpp descriptor addr :pdes[11] : 0xffffff8008061000, des_paddr[11] 0x12204b000
[    0.169686] [0] vpp descriptor addr :pdes[12] : 0xffffff8008063000, des_paddr[12] 0x12204c000
[    0.169701] [0] vpp descriptor addr :pdes[13] : 0xffffff8008065000, des_paddr[13] 0x12204d000
[    0.169717] [0] vpp descriptor addr :pdes[14] : 0xffffff8008067000, des_paddr[14] 0x12204e000
[    0.169732] [0] vpp descriptor addr :pdes[15] : 0xffffff8008069000, des_paddr[15] 0x12204f000
[    0.169788] [0] get axi2_rst
[    0.169909] [0] ion_region: base 0x0, size 0x0
[    0.169914] [0] jpu_reserved: base 0x0, size 0x0
[    0.169919] [0] vpu_reserved: base 0x0, size 0x0
[    0.170431] [0] Bluetooth: Core ver 2.22
[    0.170454] [0] NET: Registered protocol family 31
[    0.170459] [0] Bluetooth: HCI device and connection manager initialized
[    0.170468] [0] Bluetooth: HCI socket layer initialized
[    0.170476] [0] Bluetooth: L2CAP socket layer initialized
[    0.170493] [0] Bluetooth: SCO socket layer initialized
[    0.170720] [0] clocksource: Switched to clocksource arch_sys_counter
[    0.201788] [0] NET: Registered protocol family 2
[    0.202112] [0] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[    0.202174] [0] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[    0.202289] [0] TCP: Hash tables configured (established 8192 bind 8192)
[    0.202322] [0] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.202347] [0] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.202450] [0] NET: Registered protocol family 1
[    0.202670] [0] RPC: Registered named UNIX socket transport module.
[    0.202676] [0] RPC: Registered udp transport module.
[    0.202681] [0] RPC: Registered tcp transport module.
[    0.202686] [0] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.202954] [0] Unpacking initramfs...
[    0.261306] [0] Freeing initrd memory: 1684K (ffffffc00e135000 - ffffffc00e2da000)
[    0.261568] [0] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
[    0.263161] [0] workingset: timestamp_bits=62 max_order=17 bucket_order=0
[    0.269814] [0] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    0.269825] [0] io scheduler noop registered
[    0.269830] [0] io scheduler deadline registered
[    0.269919] [0] io scheduler cfq registered (default)
[    0.270415] [0] gpio-dwapb 50027000.gpio: cannot get gpio base clk
[    0.271362] [0] gpio-dwapb 50027400.gpio: cannot get gpio base clk
[    0.272227] [0] gpio-dwapb 50027800.gpio: cannot get gpio base clk
[    0.272773] [0] pwm_bm_probe
[    0.272973] [0] pwm_bm_probe
[    0.273089] [0] pwm_bm_probe
[    0.273205] [0] pwm_bm_probe
[    0.306676] [0] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.308127] [0] console [ttyS0] disabled
[    0.308159] [0] 58018000.serial: ttyS0 at MMIO 0x58018000 (irq = 21, base_baud = 31250000) is a 16550A
[    2.345798] [0] console [ttyS0] enabled
[    2.350087] [1] 5801a000.serial: ttyS1 at MMIO 0x5801a000 (irq = 22, base_baud = 31250000) is a 16550A
[    2.359817] [1] 5801c000.serial: ttyS2 at MMIO 0x5801c000 (irq = 23, base_baud = 31250000) is a 16550A
[    2.369538] [1] 5801e000.serial: ttyS3 at MMIO 0x5801e000 (irq = 24, base_baud = 31250000) is a 16550A
[    2.382601] [1] loop: module loaded
[    2.386561] [1] libphy: Fixed MDIO Bus: probed
[    2.392617] [1] usbcore: registered new interface driver uas
[    2.398322] [1] usbcore: registered new interface driver usb-storage
[    2.404699] [1] usbcore: registered new interface driver ums-alauda
[    2.410983] [1] usbcore: registered new interface driver ums-cypress
[    2.417354] [1] usbcore: registered new interface driver ums-datafab
[    2.423726] [1] usbcore: registered new interface driver ums_eneub6250
[    2.430269] [1] usbcore: registered new interface driver ums-freecom
[    2.436639] [1] usbcore: registered new interface driver ums-isd200
[    2.442921] [1] usbcore: registered new interface driver ums-jumpshot
[    2.449381] [1] usbcore: registered new interface driver ums-karma
[    2.455579] [1] usbcore: registered new interface driver ums-onetouch
[    2.462043] [1] usbcore: registered new interface driver ums-realtek
[    2.468413] [1] usbcore: registered new interface driver ums-sddr09
[    2.474698] [1] usbcore: registered new interface driver ums-sddr55
[    2.480985] [1] usbcore: registered new interface driver ums-usbat
[    2.487603] [1] bmusb-otg 50080000.bm-usb-otg: otg: primary host 50090000.bm-usb-host registered
[    2.496400] [1] bmusb-otg 50080000.bm-usb-otg: otg: shared host 50090000.bm-usb-host registered
[    2.505097] [1] bmusb-otg 50080000.bm-usb-otg: otg: can't start till gadget registers
[    2.512926] [1] bmusb-otg 50080000.bm-usb-otg: TODO: install gadget!
[    2.570091] [1] mousedev: PS/2 mouse device common for all mice
[    2.576095] [1] i2c /dev entries driver
[    2.590344] [1] bm1880-cooling bm1880_cooling: Cooling device registered: bm1880_cooling
[    2.599025] [1] Bluetooth: HCI UART driver ver 2.2.1d358e3.20181012-131125
[    2.605905] [1] Bluetooth: HCI H4 protocol initialized
[    2.611045] [1] Bluetooth: HCI Realtek H5 protocol initialized
[    2.616879] [1] rtk_btcoex: rtk_btcoex_init: version: 1.2
[    2.622277] [1] rtk_btcoex: create workqueue
[    2.626546] [1] rtk_btcoex: Coex over UDP
[    2.630673] [1] rtk_btcoex: alloc buffers 1792, 2432 for ev and l2
[    2.637233] [1] sdhci: Secure Digital Host Controller Interface driver
[    2.643763] [1] sdhci: Copyright(c) Pierre Ossman
[    2.648468] [1] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.654853] [1] bm:sdhci_bm_probe
[    2.658239] [1] mmc0: Host Controller version 3
[    2.706733] [1] mmc0: SDHCI controller on 50100000.bm-emmc [50100000.bm-emmc] using ADMA 64-bit
[    2.715465] [1] bm:sdhci_bm_probe
[    2.718845] [1] mmc1: Hardware doesn't specify timeout clock frequency.
[    2.725786] [1] usbcore: registered new interface driver usbhid
[    2.731708] [1] usbhid: USB HID core driver
[    2.736350] [1] NET: Registered protocol family 10
[    2.742597] [1] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    2.749227] [1] NET: Registered protocol family 17
[    2.756082] [1] Bluetooth: RFCOMM socket layer initialized
[    2.761584] [1] Bluetooth: RFCOMM ver 1.11
[    2.766996] [0] registered taskstats version 1
[    2.773538] [0] hctosys: unable to open rtc device (rtc0)
[    2.779182] [0] Bluetooth: Starting self testing
[    2.783805] [0] Bluetooth: Finished self testing
[    2.788954] [0] Freeing unused kernel memory: 704K (ffffffc000870000 - ffffffc000920000)
This script just mounts and boots the rootfs!
[    2.814743] [1]  max_dtr 200000000, host_drv 15, card_drv 31, drv_type 0
[    2.826910] [1] mmc0 : min 0, pass
[    2.830575] [1] mmc0 : max 7, min 0
[    2.834084] [1] mmc0 min:0 max:7
[    2.837729] [1] mmc0: new HS200 MMC card at address 0001
[    2.843531] [1] mmcblk0: mmc0:0001 Q2J55L 7.09 GiB
[    2.848584] [1] mmcblk0boot0: mmc0:0001 Q2J55L partition 1 16.0 MiB
[    2.855021] [1] mmcblk0boot1: mmc0:0001 Q2J55L partition 2 16.0 MiB
[    2.861455] [1] mmcblk0rpmb: mmc0:0001 Q2J55L partition 3 4.00 MiB
[    2.870778] [1]  mmcblk0: p1 p2 p3 p4 p5 p6
/dev/mmcblk0p3 was not cleanly unmounted, check forced.
[    3.064838] [0] random: fast init done
/dev/mmcblk0p3: 7864/131072 files (7.7% non-contiguous), 306239/524288 blocks
/dev/mmcblk0p4: ignoring check interval, broken_system_clock set
/dev/mmcblk0p4: clean, 11/196608 files, 12353/786432 blocks
/dev/mmcblk0p6 contains a file system with errors, check forced.
/dev/mmcblk0p6: Unconnected directory inode 16097 (/???)


/dev/mmcblk0p6: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
        (i.e., without -a or -p options)
e2fsck: No such file or directory while trying to open /dev/mmcblk0p7
Possibly non-existent device?
[    4.926997] [1] EXT4-fs (mmcblk0p3): mounted filesystem without journal. Opts: (null)
init.sh
[    5.228725] [1] EXT4-fs (mmcblk0p3): re-mounted. Opts: block_validity,delalloc,barrier,user_xattr
[    5.290190] [0] bmusb-otg 50080000.bm-usb-otg: otg: gadget gadget registered
[    5.605150] [1] configfs-gadget gadget: super-speed config #1: c
[    5.611744] [1] gs_console_connect: port num [0] is not support console
/ #

Now, image downloaded successfully.

Build and Run Demo Program

‌We prepares two demo programs for NNM.

• Face detection

• Object detection

Download source code

We will put demo source code in same folder as bm1880-system-sdk.

Get source code and bmodel by below command:

$ cd bm1880
$ git clone https://github.com/BM1880-BIRD/bm1880-ai-demo-program.git

---

Face detection (MTCNN)

Device Requirement:

• NNM (inference)

• PC with Ubuntu 18.04 (capture image and display result)

• UVC camera on host PC

Step 1: Build demo program

Demo source code is located in bm1880-ai-demo/rtsp_fd_fr.

Build binary for NNM:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr
$ cmake . -DAARCH64_TOOLCHAIN_BIN=/XXXXXX/bm1880/bm1880-system-sdk/host-tools/gcc/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/ -DPROGRAM=edge -DPLATFORM=soc_bm1880_asic -Bbuild/fdfr_soc
$ make -C build/fdfr_soc

Set AARCH64_TOOLCHAIN_BIN to proper path.

build binary for PC:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr
$ cmake . -DPROGRAM=host -DPLATFORM=cmodel -Bbuild/fdfr_host
$ make -C build/fdfr_host

NNM program is rtsp_fd_fr/build/fdfr_soc/bin/test_libusb_fdfr.

PC program is rtsp_fd_fr/build/fdfr_host/bin/test_libusb_fdfr.

Step 2: Put NNM demo program and bmodel into image

To modify rootfs folder, we need to unpack emmc.tar.gz, and then unpack rootfs.tar.gz.

$ cd bm1880/bm1880-system-sdk/install/soc_bm1880_asic_nnm/bm1880_emmc_dl_v1p1
$ tar xvf emmc.tar.gz
$ cd emmc_nnmc_pkg
$ tar xvf rootfs.tar.gz

Copy test_libusb_fdfr and necessary bmodels to rootfs folder, and then pack emmc.tar.gz.

$ cp -a ../../../../../bm1880-ai-demo-program/rtsp_fd_fr/build/fdfr_soc/bin/test_libusb_fdfr rootfs/
$ rm -rf rootfs.tar.gz
$ sudo tar zcvf rootfs.tar.gz rootfs
$ rm -rf rootfs
$ cd ..
$ rm -rf emmc.tar.gz
$ tar zcvf emmc.tar.gz emmc_nnmc_pkg
$ rm -rf emmc_nnmc_pkg

Step 3: Re-download image to NNM

Download image to NNM again for updating rootfs

$ cd bm1880/bm1880-system-sdk/install/soc_bm1880_asic_nnm/bm1880_emmc_dl_v1p1
$ sudo python bm1880_emmc_download.py emmc.tar.gz

Re-plug NNM to start downloading. After finished, you can check in NNM's serial console by this:

$ ls /test_libusb_fdfr

/test_libusb_fdfr

Step 4: Run demo program (NNM)

In NNM's serial console, input commands below to start demo program:

Add a new configuration file:

$ vi /fdfr.conf

repository: /data/repository
# detector: one of {tinyssh, mtcnn}
detector: tinyssh
# detector_models: list of paths separated by |
detector_models: /system/data/bmodel/tiny_ssh.bmodel|/system/data/bmodel/det3.bmodel
extractor: bmface
extractor_models: /system/data/bmodel/bmface.bmodel
similarity_threshold: 0.6
facepose_model: /system/data/bmodel/light_rpoly_lmp_iter_75000_w_scale.bmodel

Start program:

$ ldconfig
$ cd /system/data
$ ./load_jpu.sh
$ ./load_npu_bm1880.sh
$ mdev -s
$ cd /
$ ./test_libusb_fdfr ./fdfr.conf

Step 5: Run demo program (PC)

In PC's terminal, input commands below to start demo program:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr/build/fdfr_host/bin
$ sudo ./test_libusb_fdfr

There will be a window showing the results of face detection.

---

Object detection (YOLOv3)

Device Requirement:

• NNM (inference)

• PC with Ubuntu 18.04 (capture image and display result)

• UVC camera on host PC

Step 1: Build demo program

Demo source code is located in bm1880-ai-demo/rtsp_fd_fr.

Build binary for NNM:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr
$ cmake . -DAARCH64_TOOLCHAIN_BIN=/XXXXXX/bm1880/bm1880-system-sdk/host-tools/gcc/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin/ -DPROGRAMS="yolo_nnm;libusb_inference_server" -DPLATFORM=soc_bm1880_asic -Bbuild/objdet_soc
$ make -C build/objdet_soc

Set AARCH64_TOOLCHAIN_BIN to proper path.

build binary for PC:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr
$ cmake . -DPROGRAMS="yolo_host" -DPLATFORM=cmodel -Bbuild/objdet_host
$ make -C build/objdet_host

NNM program is rtsp_fd_fr/build/objdet_soc/bin/yolo_nnm.

PC program is rtsp_fd_fr/build/objdet_host/bin/yolo_host.

Step 2: Put NNM demo program and bmodel into image

To modify rootfs folder, we need to unpack emmc.tar.gz, and then unpack rootfs.tar.gz.

$ cd bm1880/bm1880-system-sdk/install/soc_bm1880_asic_nnm/bm1880_emmc_dl_v1p1
$ tar xvf emmc.tar.gz
$ cd emmc_nnmc_pkg
$ tar xvf rootfs.tar.gz

Copy yolo_nnm and necessary bmodels to rootfs folder, and then pack emmc.tar.gz.

$ cp -a ../../../../../bm1880-ai-demo-program/rtsp_fd_fr/build/objdet_soc/bin/yolo_nnm rootfs/
$ cp -a ../../../../../bm1880-ai-demo-program/darknet-yolov2-object-classification-v2/darknet/models/bmnet/yolov3_1_3_608_608.bmodel rootfs/system/data/bmodel/
$ rm -rf rootfs.tar.gz
$ sudo tar zcvf rootfs.tar.gz rootfs
$ rm -rf rootfs
$ cd ..
$ rm -rf emmc.tar.gz
$ tar zcvf emmc.tar.gz emmc_nnmc_pkg
$ rm -rf emmc_nnmc_pkg

Step 3: Re-download image to NNM

Download image to NNM again for updating rootfs

$ cd bm1880/bm1880-system-sdk/install/soc_bm1880_asic_nnm/bm1880_emmc_dl_v1p1
$ sudo python bm1880_emmc_download.py emmc.tar.gz

Re-plug NNM to start downloading. After finished, you can check in NNM's serial console by this:

$ ls /yolo_nnm

/yolo_nnm

Step 4: Run demo program (NNM)

In NNM's serial console, input commands below to start demo program:

EX. yolo_nnm <bmodel_path> <bmodel_name> <yolov2 | yolov3>

Start program:

$ ldconfig
$ cd /system/data
$ ./load_jpu.sh
$ ./load_npu_bm1880.sh
$ mdev -s
$ cd /
$ ./yolo_nnm /system/data/bmodel yolov3_1_3_608_608.bmodel yolov3

Step 5: Run demo program (PC)

In PC's terminal, input commands below to start demo program:

$ cd bm1880/bm1880-ai-demo-program/rtsp_fd_fr/build/objdet_host/bin
$ sudo ./yolo_host

There will be a window showing the results of face detection.

USB communication between Host and BM1880

Please refer to

https://github.com/BM1880-BIRD/bm1880-ai-demo-program/tree/master/usb_api

https://github.com/BM1880-BIRD/bm1880-ai-demo-program/tree/master/darknet_nnm_sample

Troubleshooting

Add later