V0.3.0 was released in 4.23, replacing our ReadEx & WriteEx & SplitEx with AsyncRead & AsyncWrite; Simplifies the Kad/DHT implementation logic.

Modify the

ReadEx & WriteEx & SplitEx:

We initially tried to define our own IO operating-related traits with async-traits so that we could write code purely with async/await.

ReadEx, for example, looks something like this:

#[async_trait]
pub trait ReadEx {
    async fn read(&mut self, buf: &mut [u8]) -> Result<usize, io::Error>;
}

Using the traits we defined does give us some benefits, rather than having to write state machine-like code. The same logic can be implemented in a more understandable way; It also introduces some problems:

  1. It is difficult to separate read from write

    Let’s first look at why we need read/write separation, useasync/awaitWay compared topollThe way to lose some control of the ability to be unable in oneFutureDo more than one thing at a time, such as reading and writing here:
let mut socket = ... ; Fn poll(socket: Pin<&mut socket >, cx: Context) {match socket.poll_read(cx) {Poll::Pending => {// There is an opportunity to process read and write socket.poll_write(cx)} Poll::Ready = > {... } } } // async/await async fn handle_socket(socket: &mut Socket) { socket.read().await; // There is no way to do this when the read is not ready, // Let read_fut = socket.read(); // let read_fut = socket.read(); // let write_fut = socket.write(); // select(read_fut, write_fut).await; }

For this reason, we need to handle the reads and writes separately. While reading and writing is handled in separate coroutines, the code logic is also clearer.

To achieve read-write separation, of course, you need the underlying Runtime IO support. The two Runtime camps provide different implementations:

  • async-std 以 CloneTo achieve the purpose of separation
  • tokioIs with the aid ofBiLockSeparation of read and write

The Clone method is of course very good, libp2P-rs is divided into many layers, the lower layer to the upper layer of IO has its own logic in the inside, so there will be various reasons to make it difficult to implement Clone, even if we pay some cost to implement Clone that means bound to a runtime.

The drawback of ReadEx is that it has no way to release a lock while the underlying IO is Pending. This can lead to deadlocks between read and write coroutines.

For this reason, SplitEx was defined, and the IO abstractions provided up each layer needed to implement SplitEx. This solved the problem, but it was less elegant and added work.

2. There is no good backward compatibility with Rust asynchronous programming, which has no officially defined standard for read and write traits. AsyncRead and AsyncWrite in the Futures library can be said to be de facto standards. Applications/systems are based on the existing AsyncRead & AsyncWrite to build, want to switch to the existing application/system libp2p – rs need to make some changes, Rust in the code are mostly in the form of a generic type parameters with the where condition limit, Rust’s modification of this qualification is often a full-body exercise.

Alternatively, we could wrapper AsyncRead on top of ReadEx, but this would require additional performance overhead, which is not desirable.

3. Some extensions in the Futures library cannot be reused

Kad:

Some changes have also been made to the implementation of the KAD-DHT protocol. More specifically, the re-provide/publish feature has been removed from Kad. We believe that the re-provide/publish logic belongs to the application that uses Kad, not Kad itself. This, in turn, simplifies the Provider/Record data structure and adds two gc_xxx methods to the RecordStore Trait to perform GC on the Provider/Record, respectively. This is done by tracking the Provider/Record timestamp received from the network. Note that GC is performed only on provider/ Record received from the network.

Netwarps is composed of domestic senior cloud computing and distributed technology development team, the team has very rich experience in the financial, power, communication and Internet industry. Netwarps has set up research and development centers in Shenzhen and Beijing, with a team size of more than 30, most of which are technical personnel with more than 10 years of development experience, respectively from the Internet, finance, cloud computing, blockchain and scientific research institutions and other professional fields.


Netwarps focuses on the development and application of secure storage technology products, including decentralized File System (DFS) and decentralized computing Platform (DCP). Netwarps is committed to providing distributed storage and computing platform based on decentralized network technology, which has the technical characteristics of high availability, low power consumption and low network. Applicable to the Internet of Things, industrial Internet and other scenarios.


Public account: Netwarps