David Vorick is the CEO and co-founder of SIA, a blockchain technology that is attempting to reinvent cloud storage. About 18 months ago he, along with other members of his team, decided to launch Obelisk, a manufacturing company created to produce ASIC mining computers.
His journey in developing an ASIC hardware manufacturing company was one very impressive learning curve, and the story he shares about those many lessons learned along the way is simply fascinating. If you’ve got *any* interest whatsoever in cryptocurrency mining, we consider David’s article a must-read:
The State of Cryptocurrency Mining
For those new to the blog, I am the lead developer of Sia, a blockchain based cloud storage platform. About a year ago, myself and some members of the Sia team started Obelisk, a cryptocurrency ASIC manufacturing company. Our first ASICs are going to ship in about 8 weeks, and our journey with Obelisk has given us a lot of insight into the world of cryptocurrency mining.
One of the reasons we started Obelisk was because we felt that coin devs in general had a very poor view into the mining world, and that the best way to understand it would be to get our hands dirty and bring a miner to market.
Since starting Obelisk, we’ve learned a lot about the mining space, as relevant to GPUs, to ASICs, to FPGAs, to ASIC resistance, mining farms, electricity, and to a whole host of other subjects that coin developers should be more aware of. We aren’t able to share everything that we know, but we’ve pulled together information on a set of key topics that I think will be helpful to cryptocurrency designers and other members of the cryptocurrency community.
We’ve been pessimistic on ASIC resistance for a long time, and our journey into the hardware world solidly confirmed our position. Hardware is extremely flexible. General purpose computational devices like CPUs, GPUs, and even DRAM all make substantial compromises to their true potential in order to be useful for general computation. For basic hardware development, most algorithms can see substantial optimization just by taking away all of that generality and focusing on one specific thing.
The vast majority of ASIC-resistant algorithms were designed by software engineers making assumptions about the limitations of custom hardware. These assumptions tend to be incorrect.
Equihash is perhaps the easiest target, as a lot of people were quite confident in the equihash algorithm, and we’ve been saying for close to a year that we know how to make very effective equihash ASICs.
The key is to make sorting memory. A lot of algorithm designers don’t seem to realize that in an ASIC, you can merge the computational and storage pieces of a chip. When a GPU does equihash computations, it has to go all the way out to off-chip memory, bring data to the computational cores, manipulate the data, and then send the altered data all the way back out to the off-chip memory.
For equihash, the manipulations that you need to make to the data are simple enough that you can just merge the memory and computation together, meaning that you can do most of your manipulating in-place, substantially reducing the amount of energy used to move data back and forth, and also substantially decreasing the amount of time between adjustments to the data. This greatly increases efficiency and speed.
Needless to say, we weren’t the least bit surprised when Bitmain released powerful ASICs for equihash. The Bitmain ASICs are actually substantially less performant (5x to 10x) than our own internal study suggested they would be. There could be many reasons for this, but overall we think that it’s pretty reasonable to assume that more powerful equihash ASICs will be released in the coming months.
We also had loose designs for ethash (Ethereum’s algorithm). Admittedly, ethash was not as easily amenable to ASICs as equihash, but as we’ve seen from products on the market today, you can still do well enough to obsolete GPUs. Ethash is by far the most ASIC resistant algorithm we’ve looked at, most of the others have shortcuts that are even more significant than the shortcuts you can take with equihash.
At the end of the day, you will always be able to create custom hardware that can outperform general purpose hardware. I can’t stress enough that everyone I’ve talked to in favor of ASIC resistance has consistently and substantially underestimated the flexibility that hardware engineers have to design around specific problems, even under a constrained budget. For any algorithm, there will always be a path that custom hardware engineers can take to beat out general purpose hardware. It’s a fundamental limitation of general purpose hardware.
A lot of people believe that computing is broken up into 3 categories: CPU, GPU, and ASIC. While those are the categories that are generally visible to the public, in the chip world there’s really only one type of chip: an ASIC. Internally, Nvidia, Intel, and other companies refer to their products as ASICs. The categories as known to the public are really a statement about how flexible the ASIC is.
I would like to use a 1 to 10 scale to measure flexibility. At one side, a ‘1’, we’ll put an Intel CPU. And at the other side, a ‘10’, we’ll put a bitcoin ASIC. Designers have the ability to create chips that fall anywhere on this scale. As you move from a ‘1’ to a ‘10’, you lose substantial flexibility, but gain substantial performance. You also decrease the amount of design and development effort required as you sacrifice flexibility. On this scale, a GPU is a ‘2’.
Generally speaking, we don’t see products developed that fall anywhere between a GPU and a fully inflexibile ASIC because typically by the time you’ve given up enough flexibility to move away from a GPU, you’ve only got a very specific application in mind and you are willing to sacrifice every last bit of flexibility to maximize performance. It’s also a lot less costly to design fully inflexible ASICs, which is another reason you don’t see too many things in the middle.
Two examples of products between a GPU and an ASIC would be the Baikal miners and the Google TPU. These are chips which can cover a flexible range of applications at performances which are substantially better than a GPU. The Baikal case specifically is interesting, because it’s good enough to obsolete GPUs for a large number of coins, all using the same basic chip. These chips appear to be flexible enough to follow hardforks as well.
The strategy of hardforking ASICs off of a network is going to lose potency the more it happens, because chip designers do have the ability to make chips that are flexible, anywhere from slightly flexible to highly flexible, with each piece of flexibility costing only a bit of performance. The Monero devs have committed to keeping the same general structure for the PoW algorithm, and because of that commitment we believe that you could make a Monero miner capable of surviving hard forks with less than a 5x hit to performance.
Equihash is an algorithm that has three parameters. Zcash mining happens with one particular choice for these parameters, and any naive hardfork from Zcash to drop ASICs would likely involve changing one or more of these parameters. We were able to come up with a basic architecture for equiahsh ASICs that would be able to successfully follow a hardfork that chose any set of parameters. Meaning, a basic hardfork tweaking the algorithm parameters would not be enough to disrupt our chip, a more fundamental change would be needed. Despite this flexibility, we believe our ASIC would be able to see massive speedups and efficiency gains over GPUs. We never found funding for the equihash ASICs, and as a result our designs ended up on the shelf.
The ultimate conclusion here once again wraps back to the capabilities of ASICs. I think there are a lot of people out there who do not realize that flexible ASICs are possible, and expected that routinely doing small hardforks to disrupt any ASICs on the network would be sufficient. It may be sufficient sometimes, but just as algorithms can attempt to be ASIC resistant, ASICs can attempt to be hardfork resistant, especially when the changes are more minor.
Monero’s Secret ASICs
A few months ago, it was publicly exposed that ASICs had been developed in secret to mine Monero. My sources say that they had been mining on these secret ASICs since early 2017, and got almost a full year of secret mining in before discovery. The ROI on those secret ASICs was massive, and gave the group more than enough money to try again with other ASIC resistant coins.
It’s estimated that Monero’s secret ASICs made up more than 50% of the hashrate for almost a full year before discovery, and during that time, nobody noticed. During that time, a huge fraction of the Monero issuance was centralizing into the hands of a small group, and a 51% attack could have been executed at any time.
Monero’s hardfork appears to have been successful in shaking the ASICs. I don’t believe that the ASIC designers attempted to build flexibility into their ASICs, but now that Monero has announced a twice-annual PoW change, we may see another round of secret ASICs with more flexibility. The block reward for Monero is high enough that even if you think you have only a 30% chance of your ASIC surviving the PoW hardfork, it’s more than worthwhile to pursue a hardfork resistant ASIC.
My strong guess is that Monero is going to have another round of secret ASICs built, and that these ASICs will be more conservative and flexible, attempting to follow the hard forks that Monero puts out every 6 months.
Other Secret ASICs
We’ve heard rumors of plenty of other secret ASICs. People who own secret ASICs tend not to talk about them very much, but as of March 2018, we had heard of secret ASIC rumors specifically for both Equihash and Ethash, and then for many other smaller coins that don’t have any ASICs on them yet. We believe a full 3 different groups were actively mining on Zcash with different ASICs prior to the Bitmain Z9 announcement.
We know of mining farms that are willing to pay millions of dollars for exclusive access to designs for specific cryptocurrencies. Even low ranking cryptocurrencies have the potential to make millions in profits for someone with exclusive access to secret ASICs. As a result, an informal underground industry has been set up around secret mining. The heavy amount of secrecy involved means it’s disconnected and mostly operates off of rumor and previous relationships. But it’s nonetheless a very lucrative industry, and even when things happen like the Vertcoin hardfork, the setback to secret miners is dwarfed by the returns of the successes.
At this point, I think it’s safe to assume that every Proof-of-Work coin with a block reward of more than $20 million in the past year has at least one group of secret ASICs currently mining on it, or will have secret ASICs mining on it within a few months. The easiest way to detect this is GPU returns, however as ASICs continue to infiltrate every coin on the market, that will cease to be a reliable metric as there will be no GPU-only coin to use as a baseline, at least not one that is large enough to sustain all of the massive GPU farms that are out there.
The ASIC game has become such an advanced game because there is so much money on the table. Even small coins can be worth tens of millions of dollars, which is more than enough to justify a high-risk production run.
Manufacturers that sell ASICs to the public, like Bitmain, tend to be less exposed than consumers to things like ASIC hardforks. Using Sia as an example, we estimate it cost Bitmain less than $10 million to bring the A3 to market. Within 8 minutes of announcing the A3, Bitmain already had more than $20 million in sales for the hardware they spent $10 million designing and manufacturing. Before any of the miners had made any returns for customers, Bitmain had recovered their full initial investment and more.
In this case, a hardfork doesn’t hurt Bitmain. Bitmain made a profit off of Sia, and there’s nothing the developers can do about that. And it seems that was the case for the Monero miners that Bitmain announced as well. Bitmain didn’t even get to announce the miners until after Monero announced their hardfork, and still it seems that they sold enough obsolete hardware to customers to make back their costs and turn a hearty profit.
The mining game is weighted heavily in favor of the manufacturers. They get to control the hardware production, the supply, and they know more about the state of the industry than anyone else. The profitability of a miner largely depends on variables that the manufacturer controls without disclosure to anyone else.
In the case of Halong’s Decred miner, we saw them “sell out” of an unknown batch size of $10,000 miners. After that, it was observed that more than 50% of the mining rewards were collecting into a single address that was known to be associated with Halong, meaning that they did keep the majority of the hashrate and profits to themselves. Our investigation into the mining equipment strongly suggests to us that the total manufacturing cost of the equipment is less than $1,000, meaning that anyone who paid $10,000 for it was paying a massive profit premium to the manufacturer, giving them the ability to make 9 more units for themselves. Beyond this, the buyer has no idea how many were sold nor where the difficulty would be when the units shipped. The manufacturer does know whether or not the buyer is going to be able to make a return, but the buyer does not. The buyer is trusting the manufacturer entirely.
Read the full article here>>