Meidapipe 3D手势姿态跟踪算法，手机端实时检测 ,多个手势同时捕捉

Ubuntu安装MediaPipe
- 1. github下载Meidapipe代码
- 2. [安装Bazel](https://docs.bazel.build/versions/master/install-ubuntu.html)
- 3. Install OpenCV and FFmpeg
- 4. 安装在Linux desktop运行需要的工具
- 5. 运行Hello World desktop example
安装Android SDK和NDK
- 1. 安装[Android Studio](https://developer.android.google.cn/studio/)，在Android studio下安装NDK
- 2. 配置SDK，NDK环境变量
在已有的Android项目中使用MediaPipe
- Step1. 构建MediaPipe AAR
- Step2：运行bazel构建binarypd文件
- Step3：在Android Studio中使用Mediapipe
- Step4：构建完成

Ubuntu安装MediaPipe

1. github下载Meidapipe代码

1
2
3
4

$ git clone https://github.com/google/mediapipe.git

# Change directory into MediaPipe root directory
$ cd mediapipe

2. 安装Bazel

提供了三种options,选择第一种，安装时可能会网速太慢导致安装失败
Step1：添加源(一次性操作)

1
2
3

sudo apt install curl
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list

Step2：安装更新Bazel

1 2	sudo apt update && sudo apt install bazel sudo apt update && sudo apt full-upgrade

Step3：安装JDK(在Android上运行需要，否则可不安装）

1
2
3
4
5

# Ubuntu 16.04 (LTS) uses OpenJDK 8 by default:
sudo apt install openjdk-8-jdk

# Ubuntu 18.04 (LTS) uses OpenJDK 11 by default:
sudo apt install openjdk-11-jdk

3. Install OpenCV and FFmpeg

1
2
3

$ sudo apt-get install libopencv-core-dev libopencv-highgui-dev \
libopencv-calib3d-dev libopencv-features2d-dev \
libopencv-imgproc-dev libopencv-video-dev

4. 安装在Linux desktop运行需要的工具

1	sudo apt-get install mesa-common-dev libegl1-mesa-dev libgles2-mesa-dev

5. 运行Hello World desktop example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

$ export GLOG_logtostderr=1

# if you are running on Linux desktop with CPU only
$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
mediapipe/examples/desktop/hello_world:hello_world

# If you are running on Linux desktop with GPU support enabled (via mesa drivers)
$ bazel run --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 \
mediapipe/examples/desktop/hello_world:hello_world

# Should print:
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!
# Hello World!

安装Android SDK和NDK

1. 安装Android Studio，在Android studio下安装NDK

2. 配置SDK，NDK环境变量

1
2
3

# 这个也是一次行操作，关闭终端后需重新配置
export ANDROID_HOME=<path to the Android SDK>
export ANDROID_NDK_HOME=<path to the Android NDK>

例如我的环境变量是

1
2
3
4
5
6
7

export ANDROID_HOME=/home/zhw/Android/Sdk
export ANDROID_NDK_HOME=/home/zhw/Android/Sdk/ndk/21.1.6352462

#如果想设置全局环境变量
$ sudo vim /etc/profile
#把上面两行export复制到profile最下面，再执行以下操作生效
$ source /etc/profile

在已有的Android项目中使用MediaPipe

Step1. 构建MediaPipe AAR

Create a mediapipe_aar() target.
新建aar_exmaple和BUILD
在mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/BUILD添加

1
2
3
4
5
6

load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")

mediapipe_aar(
name = "mp_face_detection_aar",
calculators = ["//mediapipe/graphs/face_detection:mobile_calculators"],
)

运行bazel构建AAR

1
2
3
4
5
6

bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a \
//mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mp_face_detection_aar

# It should print:
# Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mp_face_detection_aar up-to-date:
# bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mp_face_detection_aar.aar

Step2：运行bazel构建binarypd文件

1	bazel build -c opt mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu:binary_graph

Step3：在Android Studio中使用Mediapipe

先放一张最终项目结构图，可按照这个结构复制所需文件

新建FaceDetection项目
将以下文件复制到项目中

1
2
3
4

cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example/mp_face_detection_aar.aar /path/to/your/app/libs/
cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu/facedetectiongpu.binarypb /path/to/your/app/src/main/assets/
cp mediapipe/models/face_detection_front.tflite /path/to/your/app/src/main/assets/
cp mediapipe/models/face_detection_front_labelmap.txt /path/to/your/app/src/main/assets/

mp_face_detection_aar.aar --------> app/lib/mp_face_detection_aar.aar
facedetectiongpu.binarypb -------->app/src/main/assets/facedetectiongpu.binarypb
face_detection_front.tflite -------->app/src/main/assets/face_detection_front.tflite
face_detection_front_labelmap.txt -------->app/src/main/assets/face_detection_front_labelmap.txt

下载OpenCV-android-sdk，复制opencv jni库的项目中

1	cp -R ~/Downloads/OpenCV-android-sdk/sdk/native/libs/arm* /path/to/your/app/src/main/jniLibs/

再把mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu下的res，AndroidManifest.xml，MainActivity.java放到项目中，修改AndroidManifest.xml的package="com.google.mediapipe.apps.facedetectioncpu">为你自己的包，其他报错按提示修改

在app的build.gradle中添加依赖库，这里我安装的时候参考了csdn上的一篇博客，我直接用的他的dependencies，结果有一个依赖错了，导致我的项目运行失败，花了两天时间才解决，所以请严格安装官网安装，此博客只作为一个参考。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

dependencies {
implementation fileTree(dir: 'libs', include: ['*.jar', '*.aar'])
implementation 'androidx.appcompat:appcompat:1.0.2'
implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
testImplementation 'junit:junit:4.12'
androidTestImplementation 'androidx.test.ext:junit:1.1.0'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.1'
// MediaPipe deps
implementation 'com.google.flogger:flogger:0.3.1'
implementation 'com.google.flogger:flogger-system-backend:0.3.1'
implementation 'com.google.code.findbugs:jsr305:3.0.2'
implementation 'com.google.guava:guava:27.0.1-android'
implementation 'com.google.guava:guava:27.0.1-android'
implementation 'com.google.protobuf:protobuf-java:3.11.4''
// CameraX core library
def camerax_version = "1.0.0-alpha06"
implementation "androidx.camera:camera-core:$camerax_version"
implementation "androidx.camera:camera-camera2:$camerax_version"
}

Step4：构建完成

接着就可以插上你的手机运行下试试吧