SegWit has been activated on Bitcoin. As of today, all SegWit ready nodes on the Bitcoin network are already executing the new rules, marking bitcoin’s largest protocol upgrade to date.
But activation did not come easily and did not happen overnight.
This article is a review of the long road to SegWit.
The problem
Bitcoin transactions consist of two main components. One is “underlying transaction data,” which includes the locations (addresses) to which bitcoins are moved and to which they are moved, as well as other data. The second part, called a “witness,” contains code that encrypts signature data to prove that the owner of a Bitcoin actually wants to use it.
It’s this signature data that brings some complexity to bitcoin transactions. In a so-called “malleable vulnerability,” anyone can change bitcoin signatures slightly after they are created without invalidating them. This means that the entire transaction occurring, and more specifically the transaction number (TXID), can be changed by the repeater or miner who trades.
Data from a malleable attack on the Bitcoin network in 2015. The red line roughly represents extended transactions on the network
This in itself is not necessarily a big problem. Transactions are still valid, bitcoins are still transferred from one place to another, and all under exactly the same conditions. However, it does make creating new transactions more cumbersome because of the presence of unconfirmed transactions: new transactions need to know the transaction number they rely on. That, in turn, makes it harder to build certain second-tier protocols, such as two-way payment channels, on top of Bitcoin.
idea
The idea of addressing malleability vulnerabilities by “separating” signature data from other transaction data dates back several years.
Back in 2012, Bitcoin core contributors Russell O’Connor, Matt Corallo, Luke Dashjr and Gregory Maxwell, along with Bitcointalk moderators’ Theymos’ discussed this issue on the IRC Bitcoin development channel, but at the time they couldn’t find a way to successfully separate signatures from the bitcoin network.
Russell O’Connor, Gregory Maxwell, Luke Dashjr and Theymos discussed malleable vulnerabilities at IRC in 2012
A year later, in August 2013, the issue resurfaced as bitcoin core contributors Peter Todd and Gregory Maxwell had a similar discussion on IRC. But by then, both men were making progress against malleability according to their own ideas. “I mean scriptsig [whole thing apart].” “I would even suggest using scriptsigs-free deals as [trade ids],” Maxwell wrote.
A month later, Maxwell discussed malleability with renowned cryptographer Dr. Adam Back at IRC. This time, Back recommends omitting the signature when calculating the transaction ID. Although, Maxwell commented, “Separating the signature from the TXID might help, but this is a deep hard fork change… And it’s actually hard to guarantee safety.”
Side chain
Blockstream originally proposed a side-chain extension for the Bitcoin blockchain
Blockstream originally proposed a side-chain extension for the Bitcoin blockchain
In August 2014, Adam Back, Gregory Maxwell, entrepreneur and investor Austin Hill, and several core Bitcoin developers, including Dr. Pieter Wuille, founded Blockstream, a bitcoin technology company. The company focuses on side-chain technology, or alternative blockchains that can be effectively linked to Bitcoin.
By early 2015, Engineers at Blockstream decided to add a new feature to the company’s side-chain prototype product, Elements, which was announced in June of that year. This capability will ultimately address scalability issues on the side chain by separating the underlying transaction data from the witness data into different data structures.
The name of this new feature is, of course, Isolated Witness.
Block size dispute
The debate over the size of blockchain has been going on for some time: technically, since October 2010; To be more specific, since October 2013; Finally, it broke out into the public eye in the spring of 2015:
Former Bitcoin Core lead developer Gavin Andresen and Bitcoinj lead developer Mike Hearn argued that Bitcoin’s 1 Megabyte size limit should be increased through a hard fork. A hard fork is a protocol change that is incompatible with the existing bitcoin system and requires an upgrade of nearly the entire Bitcoin ecosystem. This is not an easy task — there is no consensus in the Bitcoin community for such a change.
However, in the summer of 2015, Andresen and Hearn announced that they would use the Bitcoin XT software client to advance their plans. The controversial move has put the bitcoin development community and industry in a state of emergency.
To resolve this disagreement, and to help find a possible solution to the block size controversy, the Bitcoin community quickly organized two conferences (or workshops) in the second half of 2015: the Montreal Bitcoin Scaling Conference and the Hong Kong Bitcoin Scaling Conference.
One of the most promising capacity expansion proposals presented at the Montreal meeting was the Lightning Network, a complex tier 2 capacity expansion solution detailed only a few months ago in a white paper published by Joseph Poon and Thaddeus Dryja. The only problem: this solution requires a fix for malleability.
Soft branch
Lombrozo (CodeShark), Wladimir van der Laan (Wumpus), Luke Dashjr (Luke-Jr), and Dr. Pieter Wuille (SiPA) discuss the possibility of SegWit as a soft fork at IRC
At that time, most people still believed that segregated witness could not be implemented on bitcoin’s main chain without a hard fork.
In addition to Bitcoin Core contributor (and Bitcoin Konts maintainer) Luke Dashjr.
In October 2015, between two Bitcoin expansion conferences, Bitcoin Core contributors Eric Lombrozo, Pieter Wuille, Wladimir Van der Laan, and Luke Dashjr discussed a possible new model for a soft fork at IRC. In this discussion, Dashjr pointed out that the mechanism proposed at the meeting does not apply to all potential soft forks, such as SegWit soft forks.
Interestingly, what Dashjr thought was obvious — deploying SegWit as a soft fork — had never been thought of that way before. Even Dashjr himself seemed initially unaware of the implications of this possibility.
In order to deploy SegWit as a soft fork, the witness data must be placed in a new, separate structure of the Bitcoin block. And all this “anchor” of witness data (the “Merkle roots”) has to move to a somewhat unconventional part of the Bitcoin block: coinbase transactions that reward miners with new coins.
While this approach was somewhat unusual, in the days and weeks that followed, Bitcoin Core contributors realized that this approach provided an additional benefit. By placing witness data in a separate structure on the bitcoin block, the size of the bitcoin block increases, but these changes are not noticed by the unupgraded nodes. This can actually increase the size of bitcoin blocks without increasing the size of bitcoin’s existing blocks.
In the weeks leading up to the second Bitcoin expansion symposium, some core Bitcoin contributors thought they might finally have found a solution to the debate over block size limits that would at least temporarily. Isolated Witness will effectively increase block limits in a backward compatible manner, while fixing long-standing malleability vulnerabilities, enabling more advanced scale-up solutions such as the Lightning Network.
A win-win-win solution — or so they thought.
show
Segregated Witness — as a soft fork — was first proposed by Pieter Wuille at the Bitcoin Scaling Conference in Hong Kong in December 2015. Many were hearing the proposal for the first time, and initially it seemed to receive a warm welcome.
Shortly after the second “Bitcoin Scale Up” meeting, Gregory Maxwell presented a plan called the “scale up Roadmap” with SegWit at its core. This roadmap was quickly embraced by the Bitcoin Core development team as well as other developers and users in the Bitcoin ecosystem.
critical
Despite the initial excitement surrounding the proposal, the quarantine witness has come in for a lot of criticism.
Concerns about the proposed upgrade vary. Former Bitcoin core contributor Jeff Garzik — who founded his own development company Bloq — doesn’t think SegWit is an adequate short-term scale-up solution. Meanwhile, Bitcoin XT lead developer Mike Hearn wasn’t entirely convinced by the proposal either: he dismissed the solution as an “accounting fraud” and withdrew from Bitcoin development altogether shortly after.
Jonathan Toomim, developer of Bitcoin Classic, an alternative software client, called the proposal “ugly and clumsy” and suggested it might be best tried as a hard fork. Even Bitcoin Core contributor Peter Todd has his own concerns, particularly related to mining.
However, most of these issues were considered solvable, untrustworthy, or worthy of tradeoffs by the Bitcoin Core development team. This is where the soft fork upgrade began.
The development of
While a version of Segregated Witness has been implemented on Elements, much of the code for the main version of Bitcoin has yet to be written, not only because it needs to be implemented as a soft fork, but also because SegWit on Bitcoin will include a number of new features that don’t exist in Elements: For example, the “witness discount” required to increase block size, the new compatibility of peer-to-peer networks, etc.
The specific Bitcoin improvement plan BIP141 was written by Pieter Wuille, Ciphrex CEO Eric Lombrozo, and independent Bitcoin Core contributor Johnson Lau. By early January 2016, as the expansion debate continued to heat up, these and other Bitcoin Core contributors launched an initial dedicated test network for the protocol upgrade, called SegNet. Two weeks later, the test network was made public for testing by the wider Bitcoin development community. By March, SegNet had been upgraded to a test version that supported lightning networks.
In the coming months, development has been ongoing, incorporating feedback from the Bitcoin development community, fixing bugs, improving the code base, and rolling out a new version of SegNet.
Isolated Witness’s GitHub page, where development and other issues are publicly visible for anyone to follow and contribute to
At the same time, Bitcoin contributors have been talking to more people in the bitcoin industry, and the number of companies and projects that have committed to supporting Segregated Witness has grown over time.
By June of that year, Quarantine Witness had added 4,743 lines of code (including test code) and proposed removing or modifying 554 existing Bitcoin Core lines. After a more in-depth review of the code by several contributors, Bitcoin Core’s lead maintainer Wladimir van der Laan incorporated the code into Core’s “main branch” by the end of June.
The meeting
The development of SegWit comes at a time when the debate over the size of bitcoin blocks is heating up again in the bitcoin community. This time with Bitcoin Classic leading the way, some Bitcoin companies and miners seem determined to use a hard fork to increase the block size limit to 2 megabytes.
In Hong Kong, several Bitcoin Core contributors, pool operators and other members of the Bitcoin industry held an emergency meeting to discuss expansion.
The meeting led to an agreement known as the Bitcoin Roundtable consensus (or Hong Kong Consensus). The Bitcoin Core contributors at the conference are working with the entire Bitcoin development community to develop a secure hard fork based on segregated Witness improvements. In turn, the miners agreed to run the isolation Witness in production after Bitcoin Core released a version containing the aforementioned hard fork code. That crisis seems to have been averted — though it will soon become clear that not everyone is happy with the outcome.
A few months later, more Bitcoin Core contributors and mine pool operators held a meeting in California. After the meeting, the Core contributors who attended were confident that the isolation witness would be activated by the miners.
release
Approximately six months after the original planned schedule — originally scheduled for release in April — In October 2016, Isolation Witness was officially launched in Bitcoin Core version 0.13.1. Protocol upgrades are also implemented in several other Bitcoin clients, such as Bitcoin Knots and Bcoin.
Using an activation method (BIP9) called “VersionBits” (BIP9), which is designed to minimize network outages, 95% of miners (via hash rate) must show support to activate SegWit on the Bitcoin network. From November 15, bitcoin miners can show their support for the upgrade. Users are also encouraged to upgrade the client. So far, many users do appear to have upgraded.
As of August 2017, the vast majority of the Bitcoin network consists of isolated witness ready nodes (source: luke.dashjr.org)
Based on meetings with mine pool operators and the general belief that SegWit will be a boon for Bitcoin, many expect the soft fork to be activated soon.
political
But that’s not the case. As it turned out, several participants at the Hong Kong Roundtable disagreed about what they actually signed.
In particular, Jihan Wu, bitmain’s co-CEO, said he was only willing to activate SegWit if the Bitcoin Core development team also implemented a hard fork to increase block size limits. Other mining pools, including F2Pool, HaoBTC and Bitcoin.com, also do not have signal support for soft forks.
Bitmain (and affiliate AntPool) require a hard fork to raise the block size limit and in return will support SegWit activation
In addition, a new mining pool has emerged in China: ViaBTC. Due to its close relationship with bitcontinent, ViaBTC has enough hash rate to prevent SegWit activation alone. Its operator, Yang Haibo, has positioned himself as a staunch critic of the proposed upgrade.
Activation of SegWit seems a long way off.
UASF
Profile photo of anonymous bitcoin and litecoin developer Shaolinfry
In February 2017, three months after SegWit’s official launch, a new opportunity emerged.
Anonymous developer “Shaolinfry”, who has contributed to Litecoin in the past, has put forward a new proposal on the Bitcoin development mailing list and the popular Bitcointalk.org forum: a “user-activated soft fork” or “UASF”.
The hash rate activation mechanism that has become the standard for soft forks was never a “vote,” Shaolinfry said in his email. “The signal method is widely misunderstood to mean that the hash rate is voting on a proposal, and it is difficult to correct this misunderstanding in the wider community,” he writes.
Shaolinfry suggests an alternative: user-activated soft fork (UASF). Unlike hash rate activation, user-activated soft forks use “designated day activation,” where nodes start enforcing new rules at a predetermined time in the future. “As long as the UASF is implemented by a majority of economic players, this should force the majority of miners to follow (or activate) the soft fork.”
The idea immediately made waves on Bitcoin forums and social media. And when former BTCC coo and outspoken SegWit advocate Miao Yongquan set up an award fund for the development of UASF software, it seemed that this suggestion might become a reality.
Patent technology
In the first week of April 2017, Gregory Maxwell broke a bombshell via the Bitcoin mailing list.
Maxwell claims to have reverse-engineered a manufacturer’s ASIC mining chips and found that they included patented AsicBoost technology. More importantly, Maxwell revealed that the secret use of this technology is incompatible with deploying SegWit through a soft fork. “This incompatibility largely explains some of the puzzling behavior of some people in the mining ecosystem,” he said.
While Maxwell’s email did not name specific ASIC manufacturers, Bitmain acknowledged that it added proprietary technology to its mining chips — though it denied using them on bitcoin’s main network.
Either way, for some users, the discovery reinforces the desire of some to enable an isolated witness soft fork on the Bitcoin network. Moreover, as hash rate activation becomes more and more impossible, user activation of soft forks becomes more and more likely to solve this problem.
BIP148 proposal
After coming up with the general idea for a UASF, Shaolinfry and a dozen other members of the Bitcoin community started a UASF channel on Slack, the core bitcoin community.
The channel became a center for discussion and organization about the initiative. The activation date was originally selected to be October 1, then changed to August 1 in case the hash rate was too low. Shaolinfry wrote a specific bitcoin improvement: BIP148. Rodolfo Novak, the founder of Open Dim, has also set up an information website to promote the idea.
The original plan was for exchanges and other businesses to support UASF. If these companies support this proposal and enforce the soft fork, the desired economic majority will be within reach.
But the UASF has not gained the support its supporters had hoped for. While many companies and some developers seem to support BIP148, none of the major exchanges or other businesses have declared their support, and some have even spoken out against the initiative.
By mid-April, Gregory Maxwell wrote on the Bitcoin development mailing list that he didn’t think BIP148 was a good idea either. Maxwell, one of the most respected and influential core contributors to Bitcoin, had a major impact on the rejection of the initiative: this version of UASF seemed to have lost all momentum.
Instead, some are using an alternative to UASF: BIP149.
Counterfeit currency
Many shanzhai coins are based on bitcoin-based codebase. This means that SegWit code developed for Bitcoin is largely compatible with these alternative cryptocurrencies. Not surprisingly, some shanzhai coins decided to implement SegWit. Back in January 2017, Groestlcoin was the first to activate quarantine witness.
But other currencies are also struggling. Litecoin, Vertcoin and Viacoin all seem to be attracted to bitcoin’s political game. These currencies rely heavily on the same miners as Bitcoin, and most of them don’t signal support for upgrades.
This is said to be due to technical issues or other reasons, but, as Romano, Viacoin’s lead developer, said: “It seems more likely that they don’t want to activate quarantine Witness on Shanzhai Coins because it would give them less reason to delay the activation of quarantine Witness on Bitcoin.”
By April 2017, this attitude had led Litecoin founder Charlie Lee to advocate a user-activated soft fork on Litecoin. His initiative received an enthusiastic response from Litecoin users; Soon, the Litecoin miners, Lee, and other members of the Litecoin ecosystem arranged an online meeting that resulted in the Litecoin Global Roundtable resolution. In exchange for some promises from Lee, the miners agreed to activate SegWit. ShaolinFry and others consider this UASF effort a success.
Within a week of The Isolation Witness being activated on Litecoin, an unidentified person made a bold move. He or she sends $1 million worth of litecoin to an address protected by quarantine witnesses, challenging anyone to steal the funds. So far, the bounty remains unchanged, further boosting confidence in the technology.
New York agreement
Meanwhile, the debate over block size rages on. As another client software that increases the block size of Bitcoin through a hard fork, Bitcoin Unlimited is very popular in the Bitcoin mining community. With the backing of Bitmain’s Mr. Wu in particular, the project seems to be heading for a potentially (and arguably) hard fork.
This potential threat, and the possibility of a “split” in the Bitcoin blockchain, was DCG founder and CEO Barry Silbert’s reason for holding a meeting ahead of the 2017 New York Consensus. The conference was originally announced on a private email list for Bitcoin entrepreneurs and other prominent industry members, and brought together a large segment of the Bitcoin industry, including miners — though, apparently, none of the Bitcoin Core contributors.
The result of that meeting is often called the New York Consensus. Participants agreed on what they thought was a compromise between those who wanted to use a hard fork and those who preferred to increase the block size of Bitcoin with SegWit. According to the idea originally proposed by RSK founder Sergio Demian Lerner, SegWit would activate under certain conditions, and there would also be a hard fork that would double bitcoin’s “basic block size limit.”
The New York Consensus and its two specific action points
But not everyone in the Bitcoin ecosystem supports the protocol, and one particular problem stands out. The conditions for SegWit activation under the New York Consensus are largely incompatible with those proposed by the Bitcoin Core development team, the code that has been widely adopted by Bitcoin users.
An intolerant minority
Image released by Miao Yongquan in support of BIP148 UASF
While many people switched from supporting BIP148 UASF to BIP149, not everyone abandoned the first UASF proposal entirely.
When Shaolinfry proposed the concept, he assumed that it would be supported by the majority of the economy; otherwise, it should be discontinued before the appointed date. But some users on UASF’s Slack channel had a different idea. Some of them — including Developer Luke Dashjr of Bitcoin Core and Bitcoin Knots — are considering activating a soft fork, regardless of how the rest of the Bitcoin ecosystem reacts. Even if they are in the minority, and even if they actually become a copycat coin, they will continue to upgrade UASF.
Around mid-May, Alphonse Pace linked the decision to what statistician and author Nassim Nicholas Taleb describes as a game theory concept: “an intolerable minority.” In short, the idea is based on the premise that even an economic minority should be able to motivate miners to activate the separation witness soft fork. Otherwise, they will lose their customer base.
Thanks to rumors about AsicBoost, the activation of segregated Witness on Litecoin, and dissatisfaction with the “New York Consensus” — this time, backed by game theory, support for BIP148 is growing on social media and message boards, again becoming a craze.
In addition, several articles discussed the potential for UASF growth, followed by numerous debates on social media, YouTube channels, and other discussion platforms. Eric Lombrozo, meanwhile, is throwing his weight behind the proposal, and miao’s UASF hats are all the rage. Inspired by the codename of the upcoming Electrum Wallet, August 1 has been dubbed “Bitcoin Independence Day.”
The only problem: BIP148 and New York Consensus’s activation methods are incompatible with those proposed by New York Consensus and the Bitcoin Core team.
Kludge
James Hillard, engineer at Bitmain Warranty, came to the rescue. Hilliard came up with a slightly more complex but clever solution to make everything compatible: the Isolated Witness activation proposed by the Bitcoin Core development team, BIP148 UASF and The New York Consensus activation mechanism. His BIP91 keeps Bitcoin intact — at least while isolated Witness is active.
As long as most miners activate BIP91 by August 1, all Bitcoin nodes should remain part of the same network. This is a relatively small time window, as the solution was only proposed in late May, but Jeff Garzik, the lead developer of the New York agreement, accepted the proposal and plans to release the software client resulting from the agreement in the weeks leading up to August 1. It works.
The activation
Status of information site xbt.eu when locked by BIP91
By mid-July, Bitcoin miners had missed their chance to activate quarantine Witness in time using a method proposed by the Bitcoin Core development team that was compatible with BIP148. Therefore, the market seems to be nervous about the potential “split” between BIP148 and non-BIP148 chains. In the space of a week, the value of bitcoin has fallen from around $2,500 to $1,900: its lowest level in more than a month.
Perhaps taken aback by these market moves, the Bitcoin mining community began signaling its support for BIP91 quickly, even before the New York agreement was agreed. On July 20, 10 days before the designated activation date for BIP148 (originally scheduled for August 1), BIP91 was locked. Two days later, it was activated.
With BIP91 locked, it was only a matter of time before isolation witnessed a formal lock. The quarantine witness was finally locked down on August 9 – and the point of no return had already been reached by August 8.
Bitcoin will be “officially” quarantined after a two-week grace period.
using
Isolation Witness logo designed by Albert Dros
Of course, the final step in quarantining witnesses is the actual user adoption. Because Quarantine Witness was just launched at the time of this article’s publication, it is impossible to know how many users are actually using the upgrade and how fast. Some critics, perhaps most notably Garzik, predict that widespread adoption could take a year or more. Others, including some wallet and library developers, thought they could use the feature in a few weeks, or that they were ready. Other technologies that rely on upgrades, such as Lightning Networks, as well as Merkelized Abstract Syntax Trees (MAST), atomic switching, faster hardware wallet transaction signatures, and TumbleBit in payment processor mode, are all in various stages of development.
It’s been a long road, but starting today, anyone who wants to use a segregated witness should be able to do so.
Source: https://bitcoinmagazine.com/articles/long-road-segwit-how-bitcoins-biggest-protocol-upgrade-became-reality/
By Aaron Van Wirdum