DB2中Dynamic SQL Variation以及Varlock (V lock，Internal V lock)详解

简介：

Dynamic SQL Variation 可以认为是Db2内部的一种变量，存放在Dynamic SQL Cache(package cache的一部分)中，每个Variation对应一条编译的动态SQL语句，也就是说，每当Db2编译了一条动态SQL，SQL cache中就会多一个variation，在variation上的加的锁即是Variation Lock(简称V lock 或 VarLock)。为了确保variation有效，SQL执行期间需要在其对应的variation上加锁，详细的加锁方案如下：
如果待执行SQL已在SQL Cache中，应用会在其对应的variation上加一个S锁，这个S锁可以确保SQL执行期间的有效性（比如，删除一张表会使依赖这张表的所有variation失效）。
如果待执行SQL不在SQL Cache中，那么需要编译该SQL，编译期间为了把variation加载(loading)到cache中，会在编译开始前对该variation加一个X类型的锁，这个锁被称之为V loading lock，等编译完成之后，就会释放该锁。X类型V loading lock的目地是为了确保只有这一个应用编译该SQL（编译SQL的代价非常高，只需要编译一次即可，之后不管是同一个应用还是其他应用，要执行相同的SQL就不必再重新编译）。如果应用A拿到了X类型的 V Loading lock，在进行编译期间，有应用B也想编译，那应用B也会尝试获取同一个V loading lock，不过是S类型的。

db2pd工具

为了对Dynamic SQL Variations有一个直观的理解，可以使用db2pd的dynamic选项查看内存中有哪些Dynamic SQL Variation:

测试用到的命令如下，测试之前请确保数据库是未激活状态：

$ db2 +o terminate
$ db2 +o connect to sample
$ db2pd -d sample -dyn
$ db2 +o "list tables"
$ db2 +o "select * from t1 fetch first 1 rows only"
$ db2 +o "describe table t2"
$ db2  "insert into t2 values(10,'aaa')"
$ db2pd -d sample -dyn
$ db2  "insert into t2 values(10,'aaa')"
$ db2pd -d sample -dyn

1. 刚连接库, 未发出过动态SQL，发现没有Dynamic SQL Variations

inst105@node01:~$ db2 +o terminate
inst105@node01:~$ db2 +o connect to sample
inst105@node01:~$ db2pd -d sample -dynDatabase Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:03 -- Date 2019-04-19-05.54.19.052052Dynamic Cache:
Current Memory Used           199632
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          0
Number of Statement Inserts   0
Number of Statement Deletes   0
Number of Variation Inserts   0
Number of Statements          0Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text Dynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt BlkDynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies

2. 发出4条语句之后，有4个Dynamic SQL Statements，可以根据AnchID和StmtUID找到对应的SQL语句（可以看到list tables和describe table命令也被Db2转化为动态SQL执行了）：

inst105@node01:~$ db2 +o "list tables"
inst105@node01:~$ db2 +o "select * from t1 fetch first 1 rows only"
inst105@node01:~$ db2 +o "describe table t2"
inst105@node01:~$ db2 "insert into t2 values(10,'aaa')"
DB20000I  The SQL command completed successfully.
inst105@node01:~$ db2pd -d sample -dynDatabase Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:24 -- Date 2019-04-19-05.54.40.171977Dynamic Cache:
Current Memory Used           643435
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          4
Number of Statement Inserts   8
Number of Statement Deletes   4
Number of Variation Inserts   4
Number of Statements          4Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text
0x00007F60ACF51EA0 116    1          1          1          1          1          SELECT TABNAME, TABSCHEMA, TYPE, CREATE_TIME FROM SYSCAT.TABLES WHERE TABSCHEMA = USER ORDER BY TABSCHEMA, TABNAME
0x00007F60AFD994E0 854    1          1          1          1          1          insert into t2 values(10,'aaa')
0x00007F60ACF5ABA0 904    1          1          1          1          1          select * from t1 fetch first 1 rows only
0x00007F60ACF5FD60 988    1          1          1          1          1          SELECT COLNAME, TYPESCHEMA, TYPENAME, LENGTH, SCALE, NULLS FROM SYSCAT.COLUMNS WHERE TABSCHEMA = CURRENT SCHEMA AND TABNAME = 'T2' ORDER BY COLNODynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007F60ACF52060 116    1          1          CS  5    B
0x00007F60AFD99640 854    1          1          CS  5    B
0x00007F60ACF5AD00 904    1          1          CS  5    B
0x00007F60AFD90080 988    1          1          CS  5    BDynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007F60ACF523E0 116    1          1          1          1          6   01000000010000000100800ED6 Y   2019-04-19-05.54.23.368394 16888      0
0x00007F60AFD999C0 854    1          1          1          1          4   01000000010000000100C06AD6 Y   2019-04-19-05.54.36.155828 5336       1
0x00007F60ACF5B080 904    1          1          1          1          6   010000000100000001000071D6 Y   2019-04-19-05.54.27.488288 5560       1
0x00007F60AFD90340 988    1          1          1          1          6   01000000010000000100807BD6 Y   2019-04-19-05.54.31.645792 18432      1

3. 再执行一次相同的SQL，发现还是用的原来的 Variation，只是NumRef增加了1.

inst105@node01:~$ db2  "insert into t2 values(10,'aaa')"
DB20000I  The SQL command completed successfully.
inst105@node01:~$ db2pd -d sample -dynDatabase Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:32 -- Date 2019-04-19-05.54.48.034430Dynamic Cache:
Current Memory Used           643435
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          5
Number of Statement Inserts   8
Number of Statement Deletes   4
Number of Variation Inserts   4
Number of Statements          4Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text
0x00007F60ACF51EA0 116    1          1          1          1          1          SELECT TABNAME, TABSCHEMA, TYPE, CREATE_TIME FROM SYSCAT.TABLES WHERE TABSCHEMA = USER ORDER BY TABSCHEMA, TABNAME
0x00007F60AFD994E0 854    1          1          1          2          2          insert into t2 values(10,'aaa')
0x00007F60ACF5ABA0 904    1          1          1          1          1          select * from t1 fetch first 1 rows only
0x00007F60ACF5FD60 988    1          1          1          1          1          SELECT COLNAME, TYPESCHEMA, TYPENAME, LENGTH, SCALE, NULLS FROM SYSCAT.COLUMNS WHERE TABSCHEMA = CURRENT SCHEMA AND TABNAME = 'T2' ORDER BY COLNODynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007F60ACF52060 116    1          1          CS  5    B
0x00007F60AFD99640 854    1          1          CS  5    B
0x00007F60ACF5AD00 904    1          1          CS  5    B
0x00007F60AFD90080 988    1          1          CS  5    BDynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007F60ACF523E0 116    1          1          1          1          6   01000000010000000100800ED6 Y   2019-04-19-05.54.23.368394 16888      0
0x00007F60AFD999C0 854    1          1          1          2          4   01000000010000000100C06AD6 Y   2019-04-19-05.54.36.155828 5336       1
0x00007F60ACF5B080 904    1          1          1          1          6   010000000100000001000071D6 Y   2019-04-19-05.54.27.488288 5560       1
0x00007F60AFD90340 988    1          1          1          1          6   01000000010000000100807BD6 Y   2019-04-19-05.54.31.645792 18432      1

4. 如果有其他的应用，执行相同的SQL，也不会有新的Dynamic SQL Variations。

Trace工具

为了进一步验证上面的说法，可以抓取trace，做一个简单的分析。脚本如下：

db2stop force && db2start
db2 connect to sample
sleep 2db2 -x 'values MON_GET_APPLICATION_HANDLE()' | awk '{print $1}' > currentAgentID
appHdl=`cat currentAgentID`
db2trc on -t -apphdl $appHdl -f db2trace.dmp for i in 1 2 3 4 5
dodb2 +o "select * from t1 fetch first 1 rows only"sleep 2
donedb2trc off
db2trc flw -t db2trace.dmp  db2trace.flw
db2trc fmt db2trace.dmp db2trace.fmt
db2pd -d sample -dynamic > db2pd.dyn.out
db2 +o terminate
echo "Script ends"

输出中可以看到SQL执行了5次（NumRef）：

inst105@node01:~/20190419$ cat db2pd.dyn.out Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:18 -- Date 2019-04-19-06.22.34.333059Dynamic Cache:
Current Memory Used           568245
Total Heap Size               5236572
Cache Overflow Flag           0
Number of References          6
Number of Statement Inserts   2
Number of Statement Deletes   0
Number of Variation Inserts   2
Number of Statements          2Dynamic SQL Statements:
Address            AnchID StmtUID    NumEnv     NumVar     NumRef     NumExe     Text
0x00007FEE60F51040 651    1          1          1          1          1          values MON_GET_APPLICATION_HANDLE()
0x00007FEE60F56680 904    1          1          1          5          5          select * from t1 fetch first 1 rows onlyDynamic SQL Environments:
Address            AnchID StmtUID    EnvID      Iso QOpt Blk
0x00007FEE60F511A0 651    1          1          CS  5    B
0x00007FEE60F567E0 904    1          1          CS  5    BDynamic SQL Variations:
Address            AnchID StmtUID    EnvID      VarID      NumRef     Typ Lockname                   Val Insert Time                Sect Size  Num Copies
0x00007FEE60F51520 651    1          1          1          1          6   010000000100000001006051D6 Y   2019-04-19-06.22.21.009028 6840       0
0x00007FEE60F56B60 904    1          1          1          5          6   010000000100000001000071D6 Y   2019-04-19-06.22.22.333892 5560       1

查看Trace中锁的名子，loading = 1表示是V loading lock，可以看到V loading lock的lockname和其他基本一样（只差了一位）：

inst105@node01:~/20190419$ grep lockname  db2trace.fmt | grep -i VARIATIONlockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )lockname 010000000100000001000071D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 0, )

如果进一步分析trace文件,还可以验证很多结论，比如，加的V loading lock是X类型的：

1428    entry DB2 UDB lock manager sqlplrq fnc (1.3.35.22.0)pid 10054 tid 140663370802944 cpid 10075 node 0 sec 0 nsec 3299000eduid 22 eduname db2agentbytes 152Data1   (PD_TYPE_SQLP_LOCK_INFO,144) SQLP_LOCK_INFO:lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )pLRB (nil) prevIntent NON curIntent NON intent ..X duration 1rlInFlags 0x00000001 rlOutFlags 0x00000000 rlTimeout 0xFFFFFFFE cursorBitmap 0x40000000 rrIIDin 0 rrIIDout 0 priority 0 pEHLState 0000000000000000rlUserData.UNKNOWN 0000 0000 0000 0000                        ........dataPtr (nil)1429    exit DB2 UDB lock manager sqlplrq fnc (2.3.35.22.0)pid 10054 tid 140663370802944 cpid 10075 node 0 sec 0 nsec 3302000rc = 0bytes 288Data1   (PD_TYPE_SQLP_LOCK_INFO,144) SQLP_LOCK_INFO:lockname 010000000100000001000171D6 SQLP_VARIATION (anchor,stmt,env,var={904,1,1,1}, loading = 1, )pLRB 0x7fee44354f00 prevIntent NON curIntent ..X intent ..X duration 1rlInFlags 0x00000001 rlOutFlags 0x00000000 rlTimeout 0xFFFFFFFE cursorBitmap 0x40000000 rrIIDin 0 rrIIDout 0 priority 2 pEHLState 0000000000000000rlUserData.UNKNOWN 0000 0000 0000 0000                        ........dataPtr (nil)Data2   (PD_TYPE_SQLP_LRB,128) SQLP_LRB:state L (0x04) next (nil) rsInfoIdx 0prev (nil) tranChainPrev (nil)status G (0x01) mode ..X dur 1 convMode NON tran_handl 3holdcount 0 wantrrIID 0  rrIID 0  lrbFlag 0x0 chainNum 0 lsoFeedback 0tran_chain 0x7fee44354200 head_ptr 0x7fee44354e80 awb_ptr (nil)cursorBitmap 0x40000000 hashResult 1517 wantAttributes 00 priority 2 pad2 00

又比如：
第一次跑SQL，对V lock操作顺序如下：加X类型的V loading lock (010000000100000001000171D6)，加S类型的V lock(010000000100000001000071D6)，释放X类型的V loading lock，释放S类型的V lock。
之后的4次SQL，因此SQL Cache中已经有variation, 所以再没有需要过V loading lock，只是简单的加V lock，释放V lock

再比如：
compile SQL的代价真的很高，从开始1436到结束的17737：

inst105@node01:~/20190419$ grep 'sqlra_compile_var entry' db2trace.flw
1436 0.003305000 | | | | | | sqlra_compile_var entry [eduid 22 eduname db2agent]
inst105@node01:~/20190419$ grep 'sqlra_compile_var exit' db2trace.flw
17737 0.182564000 | | | | | | sqlra_compile_var exit
而运行5次SQL才一共39039：
inst105@node01:~/20190419$ tail -n 2 db2trace.flw
39038 9.400739000 | | sqlccipcrecv data [probe 55]
39039 9.400740000 | | | sqloSSemP entry [eduid 22 eduname db2agent]

再比如：Db2是先获得了X类型的V loading lock（Trace中1428条目），然后才开始编译（Trace中1436条目），也就是说编码的前置条件是已经获得了X类型的V loading lock。