Zero-knowledge solutions are recognized as some of the most promising emerging blockchain technologies, according to a16z crypto and other key web3 analysts, creating new opportunities for enhancing privacy and scalability for blockchains. However, as past breakthrough innovations have shown, the potential of zk technology can only be truly tapped by combining expertise across sectors.
All too often, hardware, software, and cryptographic experts are siloed, working separately to reach disparate goals related to their niche, a serious obstacle blocking the evolution of new technologies and slowing their path toward mass adoption.
“You get a bunch of really smart people all repeating work, and then you get a bunch of different implementations where none of them are perfectly optimized,” says Evan Marshall, a cofounder of Demox Labs, which is building the Leo Wallet and other privacy infrastructure on the zero-knowledge blockchain Aleo.
The ZPrize competition was founded in May 2022 to help unite those siloed communities around the goal of implementing zero-knowledge cryptography solutions that can quickly accelerate its growth from an academic curiosity to a practical solution for solving real-world problems.
That industry-wide effort led to groundbreaking work in improving zk proof generation and transaction speeds, while connecting experts across cryptographic, hardware, and software sectors — and it continues with ZPrize 2023 (learn more here).
“It’s a really rare opportunity where we can work together as an ecosystem, because most of the time, there isn’t a ton of overlap between different projects,” Marshall says.
When specialists operate in isolation, without adequate communication or collaboration, it can lead to inefficiencies, vulnerabilities, and missed opportunities.
Cryptographers might design a theoretically secure system, but the entire system may be too slow to use in practice if it isn’t optimized to work with software and hardware needs. Ethereum is one example: by maximimizing for security, in part to prove the trustworthiness of the blockchain in its early days, it suffered from unnecessarily slow transaction speeds (and high gas costs) as a result.
Similarly, software experts might create efficient and user-friendly applications, but without the input of cryptographers, these might be susceptible to various attacks. Consider “the Heartbleed Bug,” a serious vulnerability discovered in April 2014 that affected the OpenSSL cryptographic software library. While its cryptographic protocols were sound, a software implementation error allowed attackers to read memory of the server and potentially expose sensitive data.
On the other hand, hardware designers can develop cutting-edge devices, but without proper integration of cryptographic and software principles, these devices can become weak links for both speed and security — the Spectre and Meltdown hardware vulnerabilities exposed in 2018 as affecting Intel, ARM, and AMD processors — are prime case studies showcasing how the gap between hardware design and software/cryptographic considerations can lead to serious challenges.
Collaboration across these often separated sectors can help avoid the costs of silos and accelerate innovations in zk technology, as ZPrize 2022 showed while bringing together over 22 teams and nearly 600 competitor applications.
A core motivation of launching the Zprize competition was to break the “chicken-and-egg” dilemma: Most interesting applications can’t scale unless they have the efficiency optimizations for attracting a large user base, and yet there is no market demand to form that user base until those applications exist.
ZPrize 2022 helped accomplish that goal, says Weikeng Chen, a research partner for L2 Iterative Ventures and a 2022 PhD graduate of the UC Berkeley RISELab. Thanks to the innovative open-source solutions published last year, this year’s builders are able to start building zk applications from the get-go.
“People know immediately what are the right approaches and technologies to help solve the problem and so they can quickly go into building applications,” Chen says.
While builders know more about where to start, the ecosystem still has many options for experimentation, says Niall Emmart, a former hardware engineer at Nvidia who went full-time with his company Yrrid Software after winning two categories in ZPrize 2022.
“A problem we face in technology sometimes is that one particular cryptography platform gets assigned early on, and prescribed as the solution,” Emmart says. “Right now, there are three or four different platforms you can do computation on, and there are lots of people working to find the right solution.”
Experts working solely within their cryptographic silos are primarily focused on security, and understandably so, Emmart says. However, that also creates problems later when trying to make applications actually useful through various hardware solutions.
“Some of the problems with some of the earliest cryptographic work is that, on the algorithm side, it required a lot of space or a lot of compute capacity,” he says.
Emmart is part of the “parallel track” of people intensely focused on optimizing hardware for zk processing, which primarily come down to three options that cryptographic and software experts might not be as well versed in:
“The guys designing the algorithms and publishing papers for Marlin verifier, and other systems, are not necessarily experts in hardware … so they’re not thinking from the perspective of how they can map things best for GPUs or FPGAs or ASICs.”
Traditional central processing units (CPUs) are good at doing one thing really well and fast, but don’t have the compute space to quickly process the massive equations necessary to provide zk proofs and validate them. That’s because CPUs have capacity for 1 execution thread; GPUs, in contrast, have the capacity to run as many as 50,000 execution threads at the same time.
“The advantage of CPUs is that you can write sloppy software: Just write up some Java code, throw it through a compiler, and it will be really quick,” Emmart says.
“What you’ve done with GPUs is create an algorithm. You have to devise your computation so it can be parallelized 50,000 times. And that can be a really challenging thing for the software guy to have all the data in all the right places so it can compute at full speed.”
One focus of ZPrize 2022 was to write a fast-sorting algorithm that could help get data formatted in the right way so it can be properly used on the hardware side.
Emmart says the competitive aspect of the ZPrize forced hardware experts like himself to stretch the boundaries of what is possible, flexing their creativity in ways they might not if there wasn’t a competition on the line.
“With ZPrize, you don’t know how the other competitors are doing, so you end up killing yourself trying to figure out: “How do I get the last 5% of that computation?”
Marshall, the cofounder of Demox Labs, says ZPrize insights related to hardware are already informing their work on the software side. .
“A pretty surprising fact so far from last year’s ZPrize is that FPGAs really weren’t all that much better than GPUs,” he says, which means app developers can “still optimize for whichever hardware you want.”
ZPrize 2022 focused on building the base for zk applications by improving zk proof generation and speeds. Marshall is excited that ZPrize 2023 will now use that solid base to break new ground on new privacy-focused software applications.
“This year, we have much fewer requirements about which technologies people can use, so you can build on any sort of browser technology — you can use WebAssembly, you can use JavaScript, WebGPU,” Marshall says.
That should expand the diversity of zk-backed software that can be built, since those browser technologies are compatible with a greater variety of consumer devices, including laptops or smartphones.
That could lead to progress in building projects similar to the Leo Wallet, which has around 65,000 active monthly users and is a great example of real-world zk applications. “This isn’t for some benchmark analysis,” Marshall says. “This competition is really designed to see how we can improve the experience of programmable privacy.”
Marshall sees the value of ZPrize as a way for the zk community to sort out crucial questions facing the entire ecosystem, especially as future insights fuel further innovations in a space that is far from settled.
“ZPrize is a good place to come together and say, ‘Hey, this is the proving system that we should really devote resources to and keep optimizing,” Marshall says.
“There aren’t really any zk apps that have Product Market Fit yet. So I think a lot of projects are invested in really testing the user experience, so that it doesn't become a bottleneck for adoption.”
ZPrize is working to bring together experts from across the cryptography, hardware, and software verticals, collaboration that is critical to creating innovations that will drive the next wave of zk innovations.
The best part? These are still the early days for zk technology, which means there is significant opportunity for developers that want to build privacy-enabled applications that will form the backbone of a better web.
If you want to learn more, we encourage you to join the Discord and explore our ZPrize GitHub, which includes the open-source solutions published from past competitions.
Then, check out ZPrize 2023 to see how you can win rewards — or sponsor a future prize — to help push the entire zk ecosystem forward.
