AOSP builds with ninja

引言

AOSP master platform builds now build with ninja instead of GNU make.

The build/core makefiles and Android.mk files are read by kati

(https://github.com/google/kati/blob/master/README.md), which will

interpret them and write out a OUT_DIR/build_(TARGET_PRODUCT).ninja

file, and then the execution of the build rules will be handled by

ninja (https://martine.github.io/ninja/).

Building with kati and ninja provides some immediate benefits:

1. Faster startup time for new builds. Running kati and then ninja

results in builds starting about 25% faster than GNU make.

2. Much faster incremental builds. kati caches the mtimes of files

and directories it read, what environment variables it used, etc., and

only re-reads all the makefiles if something has changed. This

includes globs like $(call all-files-under). Incremental builds with

a warm disk cache will start within 5 seconds.

3. Better build command output. Ninja buffers the output of each

command and prints it without mixing output from multiple commands,

meaning error messages will be printed as a group after a build

failure, and not mixed with other concurrent build output. Ninja also

prints a configurable status line that defaults to printing the

percentage of build commands that have been run, the number that have

been run, and the number that need to run, giving a reasonable

approximation of the progress through the build.

4. Incremental builds should be more reliable. Ninja considers

outputs whose command line has changed to be dirty, which will catch

changes to global cflags, etc. and rebuild anything that has changed.

The use of ninja should be transparent, it is wrapped by the top level

Makefile, so all make-based commands should continue to work.

These changes are a precursor to a much larger change that will

replace all the Android.mk files with data files using a format that

provides much better error checking, even faster builds, and 100%

reliable incremental builds.

Blueprint and Soong work together to read Blueprints files and

generate a build.ninja file. Blueprint is a framework to implement

your own build logic in a high level langauge (Go). It reads

Blueprints files and writes build.ninja files, but all the logic to

determine what build rules to put in the build.ninja file is delegated

to custom build logic, which for Android is Soong.

There will be more documentation and announcements for Blueprint/Soong

when they become relevant.

kati will be removed once we no longer need legacy support for

any makefiles.

The equivalent behavior to mm -B is not supported in ninja. If you

want to rebuild everything, you can rm out/.ninja_log. If you want to

rebuild just one directory, touch all the source files in that

directory.

Ninja uses smart output when running in a terminal, which overwrites

the status message with the next message. Try mm | cat to show all

the messages, or mm showcommands to see the actual command that was

run.

概念简介

Ninja

is a small build system with a focus on speed.

https://ninja-build.org/

See the manual – https://ninja-build.org/manual.html or

doc/manual.asciidoc included in the distribution – for background

and more details.

Kati

is an experimental GNU make clone. The main goal of this tool is to speed-up incremental build of Android.

Currently, kati does not offer a faster build by itself. It instead converts your Makefile to a ninja file.

Soong

is the replacement for the old Android make-based build system. It replaces Android.mk files with Android.bp files, which are JSON-like simple declarative descriptions of modules to build.

Blueprint

is a meta-build system that reads in Blueprints files that describe modules that need to be built, and produces a Ninja manifest describing the commands that need to be run and their dependencies. Where most build systems use built-in rules or a domain-specific language to describe the logic for converting module descriptions to build rules, Blueprint delegates this to per-project build logic written in Go. For large, heterogenous projects this allows the inherent complexity of the build logic to be maintained in a high-level language, while still allowing simple changes to individual modules by modifying easy to understand Blueprints files.

Android中的Ninja文件

从Android 7开始，编译时默认使用Ninja。 但是，Android项目里是没有.ninja文件的。

遵循Ninja的设计哲学，编译时，会先把Makefile通过kati转换成.ninja文件，然后使用ninja命令进行编译。 这些.ninja文件，都产生在out/目录下，共有两类。

一类是build-xxx.ninja文件，通常非常大，几十到几百MB。 对make全编译，命名是build-.ninja。

对mm、mmm等单模块编译来说，命名会更复杂些，大概是build--

用ninja编译

在产生全编译的Ninja文件后，可以绕过Makefile，单独使用ninja进行编译。

全编译(7.0版本)，相当于make：

ninja -f out/build-aosp_arm64.ninja

在8.0以上，应该用out/combined-aosp_arm64.ninja来执行全编译。

单独编译模块，比如Settings，相当于make Settings：

ninja -f out/build-aosp_arm64.ninja Settings

用ninja进行单模块编译的好处，除了更快以外，还不用生成单模块的Ninja文件。

常用工具

编译后在out目录下会生成一些可执行程序：

1. /out/soong/.bootstrap/bin

bpglob gotestmain gotestrunner loadplugins minibp soong_build soong_env

2. out/soong/host/linux-x86/bin:

aapt2 androidmk bcc bpmodify dalvikvm32 dex2oatd dexoptanalyzer hidl-gen llvm-tblgen multiproduct_kati profman soong_javac_filter tune2fs zip2zip

aidl aprotoc bcc_strip_attr clang-tblgen dalvikvm64 dexdump2 e2fsck ld.mc microfactory oatdump resize2fs soong_ui tzdatacheck

aidl-cpp assemble_vintf bpfmt cpp-define-generator-data dex2oat dexlist fileslist llvm-rs-cc minigzip patchoat simg2img soong_zip versioner

虽然编译过程中的相关很多，不过在开发过程中可能需要手动执行的命令却不多。

一个是格式化工具bpfmt。 与gofmt类似，可以格式化Blueprint文件。 (其实，代码基本上都是从gofmt复制而来。)

例如，格式化当前目录及其递归子目录下的所有Android.bp：

bpfmt -w .

另一个是androidmk，负责转换Android.mk为Android.bp。 其实，现阶段没有必要学会写Android.bp，通过写Android.mk来转换也行。

androidmk Android.mk > Android.bp