1.数据文件及介绍

1.1 Sqlite数据

1.2 序列化数据的存储

• NuDB 可用在各个平台上
• RocksDB 不可用在Windows平台上

2.数据结构

2.1 Sqlite表结构及说明

2.1.1 Ledgers

2.1.2 Validations

2.1.3 AccountTransactions

2.1.4 Transactions

2.1.5 NodeIdentity

2.1.6 PublisherManifests

2.1.7 ValidatorManifests

2.2 序列化数据

2.2.1 Ripple的数据序列化

以LedgerInfo为例：

void addRaw (LedgerInfo const& info, Serializer& s)
{s.add32 (info.seq);s.add64 (info.drops.drops ());s.add256 (info.parentHash);s.add256 (info.txHash);s.add256 (info.accountHash);s.add32 (info.parentCloseTime.time_since_epoch().count());s.add32 (info.closeTime.time_since_epoch().count());s.add8 (info.closeTimeResolution.count());s.add8 (info.closeFlags);
}int Serializer::add16 (std::uint16_t i)
{int ret = mData.size ();mData.push_back (static_cast<unsigned char> (i >> 8));mData.push_back (static_cast<unsigned char> (i & 0xff));return ret;
}

2.2.2 存到NuDB的数据及序列化

<big>需要序列化的数据分三种类型</big>

/** The types of node objects. */
enum NodeObjectType
{hotUNKNOWN = 0,hotLEDGER = 1,//hotTRANSACTION = 2        // Not usedhotACCOUNT_NODE = 3,hotTRANSACTION_NODE = 4
};

• LedgerInfo (对应hotLEDGER)
  • LedgerSeq,LedgerHash等，具体可参见Ripple官网
  • 区块之间是通过LedgerHash与PrevHash（上一个区块哈希）来产生顺序关联的
  • LedgerInfo的序列化代码：

  // Save the ledger header in the hashed object store
  {Serializer s (128);s.add32 (HashPrefix::ledgerMaster);addRaw(ledger->info(), s);app.getNodeStore ().store (hotLEDGER, std::move (s.modData ()), ledger->info().hash);
  }
  

• StateMap (包括各种SLE信息)
  • SLE是STLedgerEntry的简写，是Ripple自定义的一种数据结构
  • SLE种类:Account、Escrow、Fee、Amendment、PayChannel等
  • 其中有一个比较重要的SLE是skipList，它包含前面256个区块的ledgerHash，这个SLE在无交易的情况下占用空间最大，每个区块hash占32字节，256个就是8k大小
  • StateMap序列化代码：

  void
  Ledger::rawInsert(std::shared_ptr<SLE> const& sle)
  {Serializer ss;sle->add(ss);auto item = std::make_shared<SHAMapItem const>(sle->key(),std::move(ss));// VFALCO NOTE addGiveItem should take ownershipif (! stateMap_->addGiveItem(std::move(item), false, false))LogicError("Ledger::rawInsert: key already exists");
  }
  

• TxMap (交易信息)
  • 包含交易信息txnData与交易的元数据metaData(包含交易影响的结构等)
  • 最终txnData与metaDat合到一块存储，也叫metaData
  • TxMap的序列化代码

  void
  Ledger::rawTxInsert (uint256 const& key,std::shared_ptr<Serializer const> const& txn, std::shared_ptr<Serializer const> const& metaData)
  {assert (metaData);// low-level - just add to tableSerializer s(txn->getDataLength () +metaData->getDataLength () + 16);s.addVL (txn->peekData ());s.addVL (metaData->peekData ());auto item = std::make_shared<SHAMapItem const> (key, std::move(s));if (! txMap().addGiveItem(std::move(item), true, true))LogicError("duplicate_tx: " + to_string(key));
  }
  

<big>说明:</big>

1. StateMap与TxMap都是SHAMap类型的结构
2. SHAMap既是基数树(Radix Tree)同时也是默克尔树(Merkle Tree)
3. SHAMap中包含 SHAMapTreeNode 与 SHAMapInnerNode 两种节点类型
   • SHAMapTreeNode SHAMap的非叶子节点
   • SHAMapInnerNode SHAMap的叶子节点
4. StateMap中的叶子节点对应hotACCOUNT_NODE，txMap中的叶子节点对应hotTRANSACTION_NODE类型
5. SHAMap分支构造及查找算法代码：
   最开始都是从root_节点找起，根据hash选择分支：

// Which branch would contain the specified hash
int SHAMapNodeID::selectBranch (uint256 const& hash) const
{int branch = * (hash.begin () + (mDepth / 2));if (mDepth & 1)branch &= 0xf;elsebranch >>= 4;assert ((branch >= 0) && (branch < 16));return branch;
}

如果对应分支没有节点，则直接插入

// easy case, we end on an inner node
auto inner = std::static_pointer_cast<SHAMapInnerNode>(node);
int branch = nodeID.selectBranch (tag);
assert (inner->isEmptyBranch (branch));
auto newNode = std::make_shared<SHAMapTreeNode> (item, type, seq_);
inner->setChild (branch, newNode);

否则，构造一个InnerNode,并连同原来的叶子节点一起作为新InnerNode的叶子节点

auto leaf = std::static_pointer_cast<SHAMapTreeNode>(node);
auto inner = std::make_shared<SHAMapInnerNodeV2>(seq_);
inner->setChildren(leaf, std::make_shared<SHAMapTreeNode>(item, type, seq_));
assert(!stack.empty());
auto parent = unshareNode(std::static_pointer_cast<SHAMapInnerNodeV2>(stack.top().first),stack.top().second);
stack.top().first = parent;
node = inner;

setChildren

void
SHAMapInnerNodeV2::setChildren(std::shared_ptr<SHAMapTreeNode> const& child1,std::shared_ptr<SHAMapTreeNode> const& child2)
{assert(child1->peekItem()->key() != child2->peekItem()->key());auto k1 = child1->peekItem()->key().begin();auto k2 = child2->peekItem()->key().begin();auto k = common_.begin();for (depth_ = 0; *k1 == *k2; ++depth_, ++k1, ++k2, ++k)*k = *k1;unsigned b1;unsigned b2;if ((*k1 & 0xF0) == (*k2 & 0xF0)){*k = *k1 & 0xF0;b1 = *k1 & 0x0F;b2 = *k2 & 0x0F;depth_ = 2*depth_ + 1;}else{b1 = *k1 >> 4;b2 = *k2 >> 4;depth_ = 2*depth_;}mChildren[b1] = child1;mIsBranch |= 1 << b1;mChildren[b2] = child2;mIsBranch |= 1 << b2;
}

序列化数据存储相关类图：

image.png

序列化相关调用堆栈：

LedgerConsensusImp<Traits>::beginAccept ->
LedgerConsensusImp<Traits>::accept(855) ->
OpenView::apply ->
Ledger::rawReplace/Ledger::rawInsert(序列化) ->
SHAMap::addGiveItem/SHAMap::updateGiveItem

2.3 序列化数据存储到NuDB过程

1. NodeObject的hash(uint256)元素取前4个字节得到key
2. NodeObject 的 data经过lz4压缩算法压缩得到dataSize与dataCompressed
3. dataSize,key,dataCompressed写入到文件

3.从DB中查找交易、区块、账户信息

3.1 查找区块信息

根据区块序号查找获得区块信息，如 ledger_data 命令：

1. 直接从Ledger表读取LedgerHash或者读取SkipList获取LedgerHash（只对当前最大区块前256个区块或256整数倍的区块有效）
2. 读取Ledger表，查找LedgerHash对应的记录信息并构造LedgerInfo
3. 使用LedgerInfo及从配置文件中读取的db,nudb配置构造Ledger对象
4. Ledger构造函数中
   • 根据LedgerInfo中的初始化txMap
   • 根据accountHash初始化stateMap
   • 初始化Map过程中需要读取NuDB文件获取到root_节点信息
5. 区块读取完毕

{"result": {"ledger": {"accepted": true,"account_hash": "EA4088D9B6FF34DA6102E4F6BCEC96DF860CF8006E66B544EEAD784635887514","close_flags": 0,"close_time": 561191060,"close_time_human": "2017-Oct-13 06:24:20","close_time_resolution": 20,"closed": true,"hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784","ledger_hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784","ledger_index": "8","parent_close_time": 561191044,"parent_hash": "C1EFB978755E1011376C4DEFA7B61BA3834C4F218B85DC4718F0C77A0CB446B2","seqNum": "8","totalCoins": "99999999999999990","total_coins": "99999999999999990","transaction_hash": "35098D46F21556B3496DC8409CD1F51AEA9568B6935D43736EB046278747769B"},"ledger_hash": "28980C1C7C407CE32FAE8A94984AEB1B69837F3F7F532F02D8773DAC7F218784","ledger_index": 8,"state": [{"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q","Balance": "10000000000","Flags": 0,"LedgerEntryType": "AccountRoot","OwnerCount": 0,"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC","PreviousTxnLgrSeq": 8,"Sequence": 1,"index": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99"},{"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","Balance": "99999989999999990","Flags": 0,"LedgerEntryType": "AccountRoot","OwnerCount": 0,"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC","PreviousTxnLgrSeq": 8,"Sequence": 2,"index": "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8"},{"Flags": 0,"Hashes": ["AB868A6CFEEC779C2FF845C0AF00A642259986AF40C01976A7F842B6918936C7","EF422714FD6900B2F5D69CA543FC3C67091E4178F514894CCDF11E30570FA90C","07EAC861B72B4F29F4825F695A203BAAD414E4A02DDAE7DB5C5F8B0B8A5ADFAE","847EA2FBAAB54AE6D084AC6A42DF1582907E0BC11C52A4E7D36F5A6424E832BA","69C031109212CB336A6E5A0D8BAB03E7A185711C48376EAFAC6B0458261D0CAF","1D3B71E935B8922E7FAD2D3E18EC36FD6884E1DBC6DE965A2BCFC27057DCAF68","C1EFB978755E1011376C4DEFA7B61BA3834C4F218B85DC4718F0C77A0CB446B2"],"LastLedgerSequence": 7,"LedgerEntryType": "LedgerHashes","index": "B4979A36CDC7F3D3D5C31A4EAE2AC7D7209DDA877588B9AFC66799692AB0D66B"}],"status": "success","validated": true}
}

<big>注：</big>

1. transaction_hash 不为空说明此Ledger上存在交易
2. state字段为此Ledger中包含的SLE信息

3.2 查找帐户信息

给出帐户地址，查找帐户信息，如account_info命令

1. 查找参数列表中有无ledger_index参数，如果有，去获取对应的Ledger，参考3.1，如果没有，去获取最新共识过的Ledger
2. 用1中获取到的Ledger去读取account参数对应的SLE节点信息，读取过程通过Ledger中的stateMap结构去查找SLE的key对应的叶子节点
3. 根据Ledger信息与得到的SLE构造返回结果

{"result": {"account_data": {"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q","Balance": "10000000000","Flags": 0,"LedgerEntryType": "AccountRoot","OwnerCount": 0,"PreviousTxnID": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC","PreviousTxnLgrSeq": 8,"Sequence": 1,"index": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99"},"ledger_hash": "50EE9E422B7591BD0B7EE7126FAFF8437F785D137DEB60E3BD98294CBA1175D1","ledger_index": 14,"status": "success","validated": true}
}

3.3 查找交易信息

给出交易Hash，查找交易详情的过程,如tx命令

1. 根据交易Hash，从Transaction表中读取交易信息得到Transaction对象，其中，RawTxn字段内容可以构造原始交易信息，LedgerSeq字段标识交易落在哪个区块上，Status字段标识交易的共识状态，V表示共识通过
2. 根据Transaction表读取到的LedgerSeq去读取Ledger信息，参考3.1
3. 根据读取到的Ledger信息去查对应txMap中Hash对应的Node，Node的中包含txnData与metaData
4. 将1中得到的Transaction对象与3中得到的metaData构造json，返回json，结束

{"result": {"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","Amount": "10000000000","Destination": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q","Fee": "10","Flags": 2147483648,"Sequence": 1,"SigningPubKey": "0330E7FC9D56BB25D6893BA3F317AE5BCF33B3291BD63DB32654A313222F7FD020","TransactionType": "Payment","TxnSignature": "30440220249D1FA7DBB71A0EAE07AF84289136FB8F4B16B774E255A8E2B41F08EB7EEA5302201566B52AD408760EC940E8930A1402CCF0F3CCBCBDDAF356D9D399EDFA4F89B1","date": 561191060,"hash": "22F555EFF4F67BFF08C3AFBF00C830EFFCAE33C8C57F5DF1D471618E1AB3F4CC","inLedger": 8,"ledger_index": 8,"meta": {"AffectedNodes": [{"CreatedNode": {"LedgerEntryType": "AccountRoot","LedgerIndex": "079B5765FF6A6AD78F2C72D3CF6A96C6F862A5FE550567BB8CE3B31223D36A99","NewFields": {"Account": "rBuLBiHmssAMHWQMnEN7nXQXaVj7vhAv6Q","Balance": "10000000000","Sequence": 1}}},{"ModifiedNode": {"FinalFields": {"Account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","Balance": "99999989999999990","Flags": 0,"OwnerCount": 0,"Sequence": 2},"LedgerEntryType": "AccountRoot","LedgerIndex": "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8","PreviousFields": {"Balance": "100000000000000000","Sequence": 1}}}],"TransactionIndex": 0,"TransactionResult": "tesSUCCESS","delivered_amount": "10000000000"},"status": "success","validated": true}
}

<big>注：</big>

其实Transaction表中也存有metaData字段，不知道这里为什么要读文件去取

3.4 查找帐户交易信息

查找一个帐户下的交易，可限定交易数量，最大一次查找200个交易

{"method": "account_tx","params": [{"account": "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","ledger_index_max": 1,"ledger_index_min": 1000,"limit": 20}]
}

这个过程完全从SqliteDB中进行，未涉及NuDB

1. 从AccountTransactions表查找交易
2. 根据查询结果构造交易信息
3. 每个交易添加 validated:true，条件是给出的ledger_index_min 与ledger_index_max范围是当前共识过区块范围的子集

SELECT AccountTransactions.LedgerSeq,AccountTransactions.TxnSeq,Status,RawTxn,TxnMetaFROM AccountTransactions INNER JOIN TransactionsON Transactions.TransID = AccountTransactions.TransIDAND AccountTransactions.Account = '%s' WHEREAccountTransactions.LedgerSeq BETWEEN '%u' AND '%u'ORDER BY AccountTransactions.LedgerSeq DESC,AccountTransactions.TxnSeq DESCLIMIT %u;

返回结果

{"id" : 1,"result" : {"account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","ledger_index_max" : 958,"ledger_index_min" : 1,"status" : "success","transactions" : [{"meta" : {"AffectedNodes" : [{"ModifiedNode" : {"FinalFields" : {"Account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","Balance" : "99999989999999990","Flags" : 0,"OwnerCount" : 0,"Sequence" : 2},"LedgerEntryType" : "AccountRoot","LedgerIndex" : "2B6AC232AA4C4BE41BF49D2459FA4A0347E1B543A4C92FCEE0821C0201E2E9A8","PreviousFields" : {"Balance" : "100000000000000000","Sequence" : 1}}},{"CreatedNode" : {"LedgerEntryType" : "AccountRoot","LedgerIndex" : "81843E2DE3A90BADB1CA75B3C3781CFC72BDFD1584CA893240F317A7003FC93F","NewFields" : {"Account" : "rwqbtoxtmwEzCatTocFW8TcP3DYU18GtGg","Balance" : "10000000000","Sequence" : 1}}}],"TransactionIndex" : 0,"TransactionResult" : "tesSUCCESS","delivered_amount" : "10000000000"},"tx" : {"Account" : "rHb9CJAWyB4rj91VRWn96DkukG4bwdtyTh","Amount" : "10000000000","Destination" : "rwqbtoxtmwEzCatTocFW8TcP3DYU18GtGg","Fee" : "10","Flags" : 2147483648,"Sequence" : 1,"SigningPubKey" : "0330E7FC9D56BB25D6893BA3F317AE5BCF33B3291BD63DB32654A313222F7FD020","TransactionType" : "Payment","TxnSignature" : "30450221008BC728BB5F58FE91E109BBEF1B3F098E17E49EF91EEF12CC7EB065483CF056700220450C9382096F3A2EEC8F054B47C48A91035985429C3F7594D9176D881B7AB7D4","date" : 560931910,"hash" : "3395C1AC406DB2974B72C1811008F8959D953802B88CB22E5B0231D6B0F8CE54","inLedger" : 20,"ledger_index" : 20},"validated" : true}]}

转自简书作者：SwordShield
博主QQ: 122209017