Blockchain kids know bitcoin, or Ethereum, but not many hyperledgers. In the world view of blockchain, there have always been public chain, alliance chain and private chain, but the latter two have not aroused as much response as the public chain, and are despised by many supporters of the public chain.
However, we should not take an either-or approach to this issue. No objective entity in the world is black or white, with different perspectives and conclusions. Alliance chain (private chain) is a means and channel to conduct business activities between existing centralized business groups (within groups). B2B business is difficult to transfer to the public chain, not only because of performance but also because of trade secrets and other issues. I believe that the future is the coexistence of public chain and alliance chain (private chain), each has its own use scenarios, just like wan and LAN, each has its own advantages and disadvantages, each plays its own role.
Finish the background and drag up the main story of this article: Hyperledger Fabric, a consortium project spearheaded by IBM that was handed over to the Linux Foundation as an open source project in late 2015. Hyperledger Foundation has many big name members, such as IBM, Intel, Cisco and so on. Fabric is the most famous of the many blockchain projects incubated at the foundation, and when we say Hyperledger, we basically mean Fabric. In the following sections, I will explain the Fabric architecture, including multi-chain, multi-channel, ledger design, etc., as well as the chain code writing and deployment process. Finally, I will talk about the pit encountered and some shortcomings of Fabric.
The Fabric structure
The Fabric architecture has evolved over two versions. The original version 0.6 can only be used for commercial verification, but cannot be applied to real scenarios. The main reason is that the structure is simple, and basically all the functions are concentrated in the peer node, which has natural shortcomings in scalability, security and isolation. Therefore, in the later official version 1.0, the functions of the peer node were separated, the consensus service was separated from the peer node, and pluggable consensus service was independently provided for the Orderer node. One of the more important changes is the addition of multi-channel function, which can realize multi-service isolation, which is a qualitative leap from version 0.6.
Multichain and multichannel
A chain in a Fabric contains a logical structure of chaincode, ledger, and channel, which isolates organizations and transactions. It meets the basic requirements for different people to access different data in different business scenarios. Usually we say multi – chain in the operation and maintenance level is multi – channel. A peer node can access multiple channels and join multiple links to participate in different services. As shown in the figure:
In the figure, (P1, PN), (P1, P2, P3) and (P2, PN) constitute three mutually independent chains. The peer node only needs to maintain the ledger information of the chain it has joined and cannot sense the existence of other chains. This mode has many similarities with real business scenarios. Different businesses have different participants. If you do not participate in the business, you should not see any information related to the business. The multi-channel feature is the killer feature introduced by Fabric in the field of commercial blockchain. Of course, it is not perfect. Although peer nodes cannot see transactions of unrelated channels, for orderer nodes, transactions of all channels can be seen.
Books structure
A ledger is simply a series of ordered, immutable records of state transitions. State transitions are the result of chaincode execution, in which each transaction commits a series of key-value pairs to the ledger by adding, deleting, or changing them. An orderly series of transactions are bundled into blocks, which connect the ledger into a blockchain. At the same time, a state database maintains the current state of the ledger, so it is also called the world state. In 1.0 Fabric, each channel has its own ledger, and each peer maintains a ledger of the channels it joins, including the transaction log (ledger database), the state database, and the history database.
The ledger state database actually stores the most recent values of all the key-value pairs that have ever appeared in a transaction. The latest values of all data are stored in the state database for efficient execution of the chain code call. Logically, a stateful database is simply a snapshot of an ordered transaction log, so it can be recreated from the transaction log at any time. The status database will be automatically restored or reconstructed when the peer node is started. The node will not accept new transactions until it is complete. The state database can be LevelDB or CouchDB. LevelDB is the default built-in database, and CouchDB is an additional third-party database. Just like LevelDB, CouchDB can store arbitrary binary data, and as a JSON file database, CouchDB has additional support for JSON rich text queries. If the key-value pairs of chain codes are stored in JSON, You can take advantage of CouchDB’s rich text query capabilities.
Fabric’s ledger structure also has an optional history state database, which is used to query historical changes of a key. Note that the history database does not store the specific value of the key, but only records that a key changed once in a transaction within a block. In subsequent queries, the actual changed values are queried according to the change history. This approach reduces data storage and increases the complexity of query logic, both of which have advantages and disadvantages.
The ledger database is based on the file system, storing blocks in file blocks, and then storing blocks and offsets corresponding to block transactions in LevelDB, which uses LevelDB as an index to the ledger database. Block storage in the form of files can be a nightmare without an index that can be quickly located. Currently supported indexes are:
- Block number
- Block the hash
- Transaction ID Index transaction
- Block transaction number
- Transaction ID index block
- Transaction ID Indicates the transaction verification code
Kafka consensus
The consensus mechanism based on Kafka implementation is one of the consensus algorithms provided in Fabric1.0. The PBFT provided in version 0.6 was temporarily cancelled because the transaction performance was not up to the requirements. Secondly, in the context of the federation chain for which Fabric is oriented, the need for fault tolerance is not very strong because the nodes have access control. Instead, the concurrency performance is the most important. Hence the Kafka consensus.
A consensus cluster consists of multiple Orderer nodes (OSNs) and a Kafka cluster. Orderers do not communicate directly with each other, they only communicate with the Kafka cluster. In orderer’s implementation, channels are isolated in Kafka as topics. Within each Orderer, producers and consumers are created for each channel corresponding to the Topic in the Kafka cluster. The producer sends the transactions received by the Orderer node to the Kafka cluster for sorting, and at the same time as the production, the consumer also consumes the sorted transactions.
So how do you identify which block a transaction belongs to? Block clustering in a Fabric is determined by two conditions, block volume and block time interval. When the configured volume reaches the threshold, the caking operation is triggered regardless of whether the interval is reached. On the other hand, if the set interval threshold is triggered, any transaction will trigger a caking operation, meaning that there will be no empty blocks in the fabric. The kafka producer in the Orderer node sends a TTC-X (Time to cut Block X) message to the Kafka cluster. When the Kafka consumer in any Orderer node receives the TTC-X message from any node, the kafka producer in the Orderer node sends a TTC-X (Time to cut Block X) message to the Kafka cluster. All the previously received transactions are packaged and stored locally in Orderer, and then distributed to the peer nodes. The above process can be described by the following figure:
Chain code
The Fabric architecture described above is more for Fabric platform operations engineers, but for more application developers, chain codes are the most important. Chaincode is a smart contract provided by Hyperledger Fabric that is the medium through which upper-layer applications interact with the underlying blockchain platform. At this stage, Fabric provides chain codes written in Go, Java, and other languages, but the pattern is similar regardless of which language chain codes are written.
All chain codes inherit from two interfaces, init and Invoke. The init interface is used to initialize the contract and is executed only once during the lifetime of the chain code. The remaining Invoke interface is the only entry to write the business logic. Although there is only one entry, different business logic can be freely distinguished according to different parameter passing, which is very flexible. For example, if the application developer specifies that the first parameter of the Invoke interface is the contract method name and the remaining Invoke parameter list is the parameter passed to the method, then the Invoke interface method body can be split based on the method name.
So what do you get in a contract? I divide the data available to the contract interface into three categories:
- Obtain input parameters. This is easy to understand, we need to know the input of this call before we can process the logic and deduce the output;
- Interact with the state and history databases. At the contract level, we can regard the bottom layer of blockchain as a key-value database. The contract is the increase, deletion, change and check of the key value in the database.
- Interactions with other contracts. During the execution of a contract, data can be exchanged with other contracts to achieve a cross-chain effect. With this form of data retrieval, it is possible to split loosely connected business logic into multiple contracts and only invoke it across contracts when necessary, much like the microservices architecture advocated today.
There is another very important rule for coding chains: don’t have any localization and random logic. By localization, I don’t mean language localization, but implementation environment localization. Due to the decentralized architecture of blockchain, the business logic is not only executed at a node, but at all the consensus nodes. If the chain code output is related to localized data, the results may be different and consensus cannot be reached. For example, time stamps, random functions, etc. These methods are off-limits to chain coding, so use them with caution unless you know what you’re doing!
Chain code deployment
In the early days of Fabric, I was confused about the relationships between peers, channels, and chain codes until I had deployed the platform from start to finish. To put it simply, a peer is an independent computer node, whether it is a physical machine or a virtual machine, or an independent entity. Before joining any channel, the peer cannot perform any service because it has no service carrier. As a business carrier, a channel is a purely logical concept that can exist independently of a peer, but therefore has no meaning to exist. Finally, chain code is business, and business runs in channels. Even if different channels run the same chain code, because of different carriers, they can be considered as two different businesses. Peer is ground, channel is road, and chain code is car. Only by complementing each other can a complete set of blockchain business system be constructed. After the above analysis, the deployment of chain code is as simple as “putting an elephant in the refrigerator and taking a few steps” :
- Create a business carrier channel;
- Bind the channel to the peer node;
- Instantiate the chain code on the channel.
Logically, it’s as simple as that. Of course, in the actual operation process, it must be more complicated, interested readers can compare with the official document from beginning to end, and then read this article, I believe you will have a clearer understanding.
Pit and shortage
Blockchain is a new technology, Hyperledger Fabric is the implementation of the updated blockchain, in the process of our investigation and practice, basically in addition to the official documents can not find other information for our reference, we only feasible way to learn is: source! Through the code to understand the internal implementation mechanism, unknown error, directly look at the source backwards, the use of the road is really difficult. In the process of use, we also found a lot of pits, some pits can be bypassed by other means, and others can only be regarded as the deficiency of Fabric.
- Channel management: In the design of Fabric, channels are the concept of ledger. At this stage, there is only creation, but no deletion function. Of course, it is understandable that an important feature of blockchain is that it cannot be tampered with. It would be great if the whole chain could be deleted directly. However, in the case of Kafka consensus, if the data is not handled properly, the data will be deleted directly from Kafka, and orderer has no logic to handle this abnormal deletion, so it will constantly retry and crash the whole process when it reaches the retry limit. Because one channel error affects the operation of the whole system, this is not a good design;
- No complete data management solution: In our usage scenario, the growth of data is fast, and if we use CouchDB as the underlying data engine, the data explodes geometrically. The only solution is to deploy nodes in the cloud, provide sustainable cloud hard drives, and use LevelDB instead of CouchDB to avoid fuzzy queries.
Study suggest
Before because of the lack of learning materials, we can only learn through the manual code. At this point in time, Fabric1.0 has been released for some time, and there are already many learning materials available online, which are basically free and open source. More focused articles on Hyperledger Fabric will be published later, so stay tuned.
References:
Hyperledger Fabric official document 2017: Hyperledger – Fabric. Readthedocs. IO/en/latest/I…
A Kafka – -based Ordering Service for Fabric 2017:docs.google.com/document/d/…
The authors introduce
Ziyou, the underlying architect of blockchain. At the beginning of 2016, I got involved in blockchain and devoted myself to it full time. Now I am working in a fortune 500 enterprise to do blockchain underlying research and build BAAS platform. Proficient in blockchain underlying storage, consensus and other technologies, professional orientation of alliance chain system.