Scientists at Sandia invent new yardstick for benchmarking performance
What does a quantum computer have in common with a top draft pick in sports? Both have attracted lots of attention from talent scouts.
Quantum computers, experimental machines that can perform some tasks faster than supercomputers, are constantly evaluated, much like young athletes, for their potential to someday become game-changing technology.
Scientist-scouts now have their first tool to rank a prospective technology’s ability to run realistic tasks, revealing its true potential and limitations.
A new kind of benchmark test, designed at Sandia, predicts how likely it is that a quantum processor will run a specific program without errors.
The so-called mirror-circuit method, published recently in Nature Physics, is faster and more accurate than conventional tests, helping scientists develop the technologies that are most likely to lead to the world’s first practical quantum computer, which could greatly accelerate research for medicine, chemistry, physics, agriculture and national security.
Until now, scientists have been measuring performance on obstacle courses of random operations.
But according to the new research, conventional benchmark tests underestimate many quantum computing errors. This can lead to unrealistic expectations of how powerful or useful a quantum machine is. Mirror-circuits offer a more accurate testing method, according to the paper.
A mirror circuit is a computer routine that performs a set of calculations and then reverses it.
“It is standard practice in the quantum computing community to use only random, disordered programs to measure performance, and our results show that this is not a good thing to do,” said computer scientist Timothy Proctor, a member of Sandia’s Quantum Performance Laboratory who participated in the research.
The new testing method also saves time, which will help researchers evaluate increasingly sophisticated machines. Most benchmark tests check for errors by running the same set of instructions on a quantum machine and a conventional computer. If there are no errors, the results should match.
However, because quantum computers perform certain calculations much faster than conventional computers, researchers can spend a long time waiting for the regular computers to finish.
With a mirror circuit, however, the output should always be the same as the input or some intentional modification. So instead of waiting, scientists can immediately check the quantum computer’s result.
The research was funded by DOE’s Office of Science and Sandia’s Laboratory Directed Research and Development program. Sandia is a leading member of the Quantum Systems Accelerator, a DOE national quantum research center.
New method reveals flaws in conventional performance ratings
Tim and his colleagues found that randomized tests miss or underestimate the compound effects of errors. When an error is compounded it grows worse as the program runs, like a wide receiver who runs the wrong route, straying farther and farther from where they are supposed to be as the play progresses.
By mimicking functional programs, Sandia found results often had larger discrepancies than randomized tests showed.
“Our benchmarking experiments revealed that the performance of current quantum computers is much more variable on structured programs” than was previously known, Tim said.
The mirror-circuit method also gives scientists greater insight into how to improve current quantum computers.
“By applying our method to current quantum computers, we were able to learn a lot about the errors that these particular devices suffer — because different types of errors affect different programs a different amount,” Tim said. “This is the first time these effects have been observed in many-qubit processors. Our method is the first tool for probing these error effects at scale.”