Frontiers of Materials That Learn: Proceedings of a Workshop (2026)
Chapter: 2 Physical Learning Implemented in Physical Systems
2
Physical Learning Implemented in Physical Systems
The first session’s moderator, Stefano Martiniani, an assistant professor at New York University, opened the period with some comments about why it is possible to learn in the first place. Speaking about an essay by the pioneering vision scientist Horace Barlow, he said that Barlow pointed to the patterns and regularity in sensory stimuli as being what drives unsupervised learning. “What he was saying essentially,” Martiniani said, “was that the reason why we can learn in the first place is because physical laws make the world regular. The brain’s ability to discover and model the structure of the world has to do with the fact that there are repeatable patterns, and because there are repeatable patterns, the brain can compress and, through this compression, it can learn to understand the world.”
Another way of looking at this, he continued, is captured by Shannon’s conditioning theorem, which is often summarized as “Conditioning reduces entropy.” The theorem says that the uncertainty in a random variable is never increased by knowledge of another random variable. In other words, any condition placed on the world will reduce its entropy. It will make it regular. “So,” Martiniani continued, “this is an information theoretic way of saying the same thing that Horace Barlow was saying—that the world cannot be a hot mess. There has to be correlations in the system in order for it to function and to be understood.”
He then offered a few details about what his lab does. It works to unify the laws that govern matter and learning, he said, essentially by understanding how physical principles can explain learning and also how an understanding of physical systems makes it possible to design better learning algorithms. In particular, one thing that his lab does is to develop frontier artificial intelligence algorithms
that can design functional materials, and they built a model called Open Materials Generation, which Martiniani described as currently the state-of-the-art model for discovering new materials.
His lab is also working to “put neurons back into neural networks,” he said, and they are doing this by designing neural networks on the basis of neuroscientific principles and identifying learning rules that are local. Using local learning rules in a learning system is a fundamentally different approach from how neural networks are currently trained, he noted.
The lab is also bringing these two approaches together and trying to use nonequilibrium statistical mechanics and their experience in designing machine learning models to comprehend why neural networks learn in the first place. When he talks about this work, people often tell him that it has nothing to do with machine learning, he said, so his lab has recently shown that if one takes these physical ideas, it is possible to construct algorithms that outperform some of the best algorithms that computer scientists have come up with. Specifically, he mentioned Contrastive Learning as Manifold Packing, which is a way of doing self-supervised learning that essentially packs neural manifolds in the representation space of the neural network (Zhang et al., 2025).
PHYSICAL LEARNING IN MECHANICAL NETWORK MATERIALS
Xiaoming Mao, a professor in the Department of Physics at the University of Michigan, began the session’s presentations with a discussion of recent work on physical learning in mechanical network materials. To provide context, she opened her talk by describing a biological organism that has inspired her research in that area.
“Let’s imagine an insect flying in the wind, and the wind condition is changing over time,” she said. “The morphology and the activation of the wing will determine what kind of strain develops in this wing as a result of its interaction with the wind, and this strain will then be picked up, will be detected by the neuron system and turned into neuron signals, which in turn will determine the morphology and the activation of the next moment.” In short, she said, the insect wings have a feedback loop that allows the insect to fly in a stable way in changing, complicated situations.
Scientists and engineers, Mao said, are now trying to develop autonomous material systems (metamaterials) with embedded intelligence and also neuromorphic metamaterials, which mimic biological systems such as the brain’s neural structure, and are able to do things similar to those biological systems. Mao then showed an example of one such system (Figure 2-1). In that example, electric circuits are embedded in a mechanical metamaterial that picks up inputs, such as a force field, and then computes the output, or strength field, that the metamaterial needs to produce in order to yield the desired action, such as flying.
SOURCES: Presented by Xiaoming Mao on October 2, 2025. “Roadmap on embodying mechano-intelligence and computing in functional materials and structures” (Figure 3c) from Alù et al. (2025), used under CC BY 4.0 license with no changes.
Learning and adaptation play important roles in both the insect wing and the mechanical metamaterial, Mao said. In the case of insect wings, the main driver of the adaptation is evolution, and many insects are able to fly immediately upon emerging from the pupal stage. “However,” she continued, “there are many other examples of biological functions that need learning. For example, we did not know how to use our hands when we were babies, and it takes a lot of practice and learning, and then they become amazing tools for us.”
The situation with engineered materials is similar, she continued. The current paradigm of metamaterials with embedded intelligence is to pre-code the functions that these materials perform and to use an external circuit to control the feedback loop that shapes the functionality. However, she said, at the same time there is a major opportunity to embed the learning process itself in physical materials so that the material can generalize and adapt to unpredictable new environments it has never seen and still function. Furthermore, she added, combining learning with intrinsic materials processes in this way has an additional benefit of maximizing efficiency.
Creating Material Systems That Learn
To create such a learning system, Mao said, the basic approach is to use a feedback loop where there is some sort of trial, the results of the trial are sensed and then analyzed, and then the system is modified to perform better on the next iteration of the trial. “There has been a lot of exciting recent work on this new front,” she said. An understanding is emerging from the collective work that there are a few ingredients you need to add in order to produce learning in physical materials, she continued. One part of this understanding is what she called the “local rule,” which
is that only local measurements are used to determine what should be changed and it is not necessary to connect every part of the material with some external computer to do the computations. Another part is “physical update,” which Mao said is very much related to neuroplasticity. In particular, she explained, “after you figure out what needs to change, some kind of adaptive or learning degree of freedom in the material has to actually physically change” to get the function to work.
There are a number of beautiful algorithmic schemes that have been proposed to achieve this sort of learning feedback loop in materials, she said, including contrastive learning (Movellan, 1991), equilibrium propagation (Scellier and Bengio, 2017), and coupled learning (Dillavou et al., 2022). She noted that the next talk would describe some experimental realizations of coupled learning.
In general, Mao continued, it is often assumed that these sorts of physical learning algorithms—contrastive learning, equilibrium propagation, and coupled learning—are different from backpropagation, which is one of the main algorithms used in machine learning today. Backpropagation is highly accurate and efficient and has enabled many learning algorithms, she said, “but at the same time, it depends on a global, not a local, computation. One needs to know all of the degrees of freedom at the same time to compute the gradient of the last function, and this would seem to be in contradiction of the local rule.
However, she continued, in recent work her team has shown how to turn backpropagation into a local rule and implement it in a mechanical system. This would be the topic of the remainder of her presentation.
Training Mechanical Networks with In Situ Backpropagation
Mao described two sets of related work. The first was the training of all-mechanical neural networks for task learning through in situ backpropagation (Li and Mao, 2024), and the second combined that method with topological in situ states to achieve robust learning (Li and Mao, 2025).
To explain the work, Mao first reviewed how backpropagation works. “Imagine having a very simple neural network,” she said. “You have some kind of input that is computed through the layers, and you get an output. To quantify its function, you can define this last function … that computes the distance of the actual output from the desired output you want.” Backpropagation works by finding ways to tune the tunable degrees of freedom so as to minimize a loss function through a steepest descent. There is a forward pass in which one takes input multiplied by the weights and then does nonlinear activation through all of the layers. “Eventually you’ll get output,” she continued, “and we can think of a very simple loss function, which is a mean-squared deviation from the desired output.”
There is also a backward pass that is used to find the loss function which depends on all of the adaptive degrees of freedom in the network. “In this case,” Mao
said, “these are the weights in this artificial neural network.” Using this approach led to major progress in neural networks and made it possible for large-scale computation to learn and optimize the loss function, she said. “In this particular formula,” she continued, “you can see that in backpropagation, you are able to find the gradient of the loss function with respect to your learning degrees of freedom layer by layer, from the output backwards to the input.”
This is a mature algorithm and has been used extensively in machine learning, she said, but it is not obvious how one might carry out the algorithm in a material because in every step of the computation all of the nodes of the neural network in a layer are involved at the same time. It is not clearly compatible with the local rule, she observed, but it turns out that there is indeed a way—an optimization algorithm called the adjoint method—to make backpropagation compatible with the local rule.
Next, she explained briefly how this works, using a simple mechanical system made of springs and masses (Figure 2-2). There is some kind of input force on some of the nodes, and another node is identified as the output. The displacement of this node in response to the input forces is measured, and that displacement serves as the output signal. One defines a loss function that depends on how the output node moves, and that is how the network computes, she said.
Explaining further, she said that the spring constants in this network are the learning degrees of freedom, or the weights of the neural network. The learning
NOTES: The black circles are nodes, and the straight lines connecting them are springs. F is the input force, and the node labeled “Target” is where the output is measured.
SOURCE: Presented by Xiaoming Mao on October 2, 2025. Figure by Dr. Shuaifeng Li from Dr. Xiaoming Mao’s group.
problem can be stated as a minimization problem, that is, the minimization of the loss function, L, which is a function of the displacement u of the output node (i.e., the target), which in turn is a function of all of the spring constants k in the network (Figure 2-3). At the same time, Mao continued, the system is subject to the mechanistic rule that the displacement field u and the force field F are related through a simple matrix D, which is derived just from force equilibriums and static mechanics.
The important quantity is the gradient of the loss function, ∆L, she reiterated. “It tells me how I need to change each of the individual spring constants such that the motion of the output node follows what I need and gives me the computation result.” The gradient can be written using the observables of the system as ∆L = (∂L/∂u)(∂u ∂k).
The first term, ∆L, is easy to compute, she said, because the loss function was defined as a function of the output displacement, but the second term, is difficult. It is a large matrix representing the displacement of all nodes in the network and the spring constants of all of the springs, and each node’s displacement depends on all of the springs at the same time. Thus, the optimization would seem to require a global computation, she said. However, she continued, the adjoint method allows one to rewrite the computation in terms of a very simple product that only has to do with this spring itself. It does not require knowing what happens to the spring’s neighbors.
The process involves two experiments or simulations, with the first informing the second. The first experiment or simulation, called the forward problem, is similar to backpropagation, Mao said. “You just give the input force and measure the elongation of all of the springs.” For the second experiment or simulation, called the adjoint problem (Figure 2-5), one measures the displacement of the output nodes from the first step and then uses that to compute the adjoint signal, which, Mao said, “is a very simple formula because we defined the loss function—we know what it is.”
Next, one uses the adjoint force as a signal in the mechanical network (Figure 2-4, right side) and carries out the backpropagation physically in that network.
SOURCES: Presented by Xiaoming Mao on October 2, 2025. From Li and Mao (2024). Figure by Dr. Shuaifeng Li from Dr. Xiaoming Mao’s group.
SOURCES: Presented by Xiaoming Mao on October 2, 2025. From Li and Mao (2024). Figure by Dr. Shuaifeng Li from Dr. Xiaoming Mao’s group.
“It’s a fully physical process,” Mao said, “so you can measure the elongation of the springs again, and then you do an element-wise multiplication of the elongation of each of the springs in these two experiments, and it tells you precisely what this gradient is.” Thus, by carrying out only the two experiments or two simulations, she continued, “you can find the gradient of how you can improve your network for it to learn a function.”
It is possible to carry out these two steps experimentally without using complicated equipment, Mao said. Her team used a simple three-dimensional printed network, exposed it to a force that would stretch it, and then visually recorded the results and used image analysis to quantify them. They designed a very simple loss function based on how they wanted the output node to move in response to a particular force. In the initial forward pass, the input node was subjected to an input force, and the team recorded the elongation of all of the springs; this can be carried out via simulation as well as experimentally, Mao noted. For the adjoint step, the team calculated the adjoint force, applied that force to the output node, and again measured the elongations of all of the springs. Then, using element-wise multiplication, they calculated the gradient. “This gradient tells me how much I need to change the spring constant of each of the springs in order to minimize the loss function,” she explained.
The next step would be to physically change the spring constants in the material network, and over a number of iterations the function would converge to the desired function. “If we are able to achieve it,” Mao said, “we will have an autonomous material that can learn by itself on the fly in real time.”
To this point, however, her team has not been able to carry out this last step experimentally because it would require changing the stiffness of a real physical
material, she said. However, there are many experimental methods that could potentially accomplish this and produce a learning mechanical network.
Continuing, she said that her team has carried out the in silico experiment, and in doing so they have shown that the process actually works. She showed a slide illustrating how the technique was used to solve a simple optimization problem where the goal was to get one node to move in a particular way in response to the movement of another node. But the same method can be used for much more complicated machine learning tasks, such as classification, she said.
As an example, she described how the technique could be used to classify different species of iris flower. The task is to learn how to classify the different irises correctly given data on characteristics such as petal width, petal length, sepal width, and sepal length. This can be turned into a mechanical problem on the type of learning network that Mao described as follows: The inputs are forces applied to four input-different nodes associated with the four different characteristics, and there are three output nodes for the three different types of irises being classified—in this case, Iris virginica, I. versicolor, and I. setosa—and the node that moves the most indicates which type of flower has been input. “We use the adjoint method to train this network,” Mao said, “and it works pretty efficiently.” The loss function decreases very quickly and, within a few rounds of training, the accuracy has become very high.
Not only can this method be used with a mechanical network, Mao added, but it can be adapted directly to such things as electric networks, acoustic metamaterials, and electromagnetic metasurfaces. “There are many systems, especially metamaterials, that have the same linear structure,” she said, and you can adapt this method of backpropagation.
Finally, she briefly described some recent work where this approach was used with topological metamaterials (Li and Mao, 2025). A topological material is a type of material whose interior behaves like an electrical insulator but whose surface acts as an electrical conductor. In this work, there is a topological metamaterial with a pair of topologically protected edge states, a spin-up and a spin-down state. They can apply this training algorithm to the topological metamaterial to tune the spring constants in this network so that the network recognizes different types of dynamic signals, she said, with one type of signal resulting in the wave propagating to the left, while another type of signal will send the wave to the right. “So we can do binary classification using pseudo-spin of topological states,” she concluded.
Looking to the Future
Mao closed by looking at the big picture of what should be done next. “In my view, this field of metamaterials, or materials that can learn, is really at the exciting intersection of a few emerging fields: neuromorphic computing, physical learning,
and physical neural networks,” she said. Numerous new papers come out every year on these topics, she said, and it is useful to think about the research as lying on a spectrum. On one end of the spectrum the goal is to make precise computing machines that have better efficiency than current digital platforms. This is the field of physical neural networks and neuromorphic computing, and people are actively trying many different types of platforms, including optical, electromagnetic, acoustic, and chemical. “This is an exciting field that holds the promise of orders-of-magnitude improvement of energy efficiency for learning compared to current digital architecture,” she said.
On the other end of the spectrum is research aimed at integrating the learning process with the materials’ functionality. “This is really a new paradigm for materials science and manufacturing,” Mao said, “because we’re now thinking about a new generation of materials that can update and adapt to new environments and still function in these new environments.” This also involves developing algorithms for broader contexts, she added, and there are many exciting parallels between the functioning of these materials and biological functions. “So we have a lot of exciting questions to explore here.”
Question-and-Answer Session
In the discussion session that followed her presentation, Mao was asked if there was any way to generalize the adjoint approach to include nonlinearity in a perturbative way. She answered that her team has not been able to achieve that yet, but it may be possible. “It may be worthwhile to search the literature to see how they deal with nonlinear problems,” she said. “At least you can expand around the ground state, and, like you said, do it perturbatively, and include higher-order terms.
In answer to another audience question, Mao said that her network process does not depend upon knowing the physics of the material. “We just do the measurement and compute a simple multiplication on each of the edges.” Following up on that, the audience member commented that the only external intelligence needed in the work was the computation of the derivative of the loss function with respect to the displacement of the node in order to figure out how much force to apply. “Because you know that derivative well,” he continued, “can you imagine some context in which a not-so-smart environment would nevertheless apply the right force? What does that derivative look like?” That might certainly be possible, Mao answered. “For the simple mean-squared loss function we have here, this is just a linear function, right? So, … I just need to measure the displacement in the forward pass and then plug it into this linear function. That tells me how much force I need to apply in the second step. There must be simple mechanisms to do that.” Something else that requires computation, she continued, is the multiplication for each of the springs. This is a simple
calculation, she said, but her team has not yet been able to figure out a physical mechanism to do that multiplication.
Another audience member commented on how accurately Mao’s network could classify different types of iris and asked about how the network performance scales with size. Mao replied that her team has not yet quantified that. She added that the network she used was not very large and that it worked well for several types of classification problems, but those problems were all relatively simple, and she does not know what size network would be needed to solve more complex problems.
Suhas Mahesh from Schmidt Sciences asked about how the sorts of learning networks that Mao talked about might fare in the future, given how much today’s neural networks can do. In particular, he said, neural networks have various functions such as skip connections and bath normalizations, two techniques that are used to speed up learning in a neural network, and “it’s not immediately obvious to me how a physical system could do all of that.” In terms of direct comparisons between the current physical networks and existing digital neural networks, she said, there are certainly problems that can be solved better using the classic neural networks, but some of that is balanced out by the energy efficiency that comes with using physical neural networks. However, instead of replacing digital algorithms that are already working well, she said, the real advantage of the physical systems that learn may lie in how they can be integrated with certain physical functionality, such as robotics or some kind of functions that are directly physical or chemical.
Martiniani commented that the adjoint method Mao described seems to be formally equivalent to predictive coding. “In that context, the problem when you train this network,” he said, “is that during your forward pass you have to relax the system, and their relaxation is low.” So if you were to do this computationally, this wouldn’t be able to scale, unlike the propagation, because you essentially are solving the e, and it has an exponential relaxation. … If there’s a characteristic time it will take for your system to adjust to the change, even if you could change the stiffness of the springs, it would take some time for the system to adjust, right?” Thus, the speed at which these networks could do their computations might be an issue.
Mao said that it may depend on the nonlinearity of the system. The computations they have been doing are linear, so it is not necessary to do minimization. “You just do the matrix multiplication,” she said. “We are not really doing a descent of the physical energy. If it’s a nonlinear system, then, like you said, you may need to do that, and we have not fully explored that yet. But the linear question is actually pretty fast.”
PHYSICAL LEARNING IN RESISTOR NETWORKS
The next speaker, Doug Durian, a professor of physics and astronomy at the University of Pennsylvania, described learning in a network of electrical resistors.
In particular, his work involves small networks of self-adjusting resistors. Each resistor can independently adjust its conductance in order for the network as a whole to learn a desired task that is defined in terms of training data. In short, the network carries out what Durian called “supervised contrastive learning” and, in doing so, is able to teach itself how to carry out tasks similar to those performed by artificial intelligence (AI). This is all done, he emphasized, without a computer. “So we’re doing AI without a computer,” he said. “This is a system that can do analog in-memory learning or analog in-memory computation.”
Neural Networks Versus the Brain
There are many reasons to develop these learning networks, Durian said, and one way to think about it is that he and others in the field are trying to develop systems that can carry out computations in a way that is more like the brain than like a computer or neural network. Computers are absolutely amazing, he said. They are blazingly fast and prodigiously precise. “By contrast,” he continued, “brains are very noisy and slow, and you just don’t know when neurons are going to fire.” There are all kinds of noise in the system, he said. In short, compared with electronic systems, brains are a mess. “Yet somehow out of this mess,” he said, “brains can actually learn much better than neural networks, and they’re incredibly energy efficient by comparison.” So the goal, he said, is to understand how brains do it and to apply that understanding to developing a new way to compute.
Durian then listed some specific differences between neural networks and brains. First, neural networks have to be trained; they cannot learn by themselves. In a neural network, the learning is imposed from the top down. One must have perfect knowledge of the system, and then the approach is to define a loss function and do gradient descent on that loss function. “So you have knowledge of the system and what you want it to do, and you impose the learning from outside,” he summarized. “You tell it what you want it to do.” By contrast, he continued, the brain learns from the bottom up. There is no model of the brain, no outside intelligence directing its learning. Instead, as Liu described in her opening remarks, the brain’s neurons adjust themselves with the local Hebbian learning rules. In short, the difference between the learning by neural nets versus the learning by brains is top-down versus bottom-up. With neural nets, the learning is imposed from the outside, while in brains learning is an emergent property.
Another difference, Durian said, is that information flow in artificial neural networks is generally just in one direction, from input to output—they are mainly “feed forward.” In brains, by contrast, “it’s this hairball mess of stuff that’s just wired up kind of at random, so it’s highly recurrent, and there’s no natural sense in which information flows from one place to the other.”
Contrastive Local Learning Networks
The general goal of Durian’s work, then, is to develop a physical system that mimics these aspects of how the brain learns. His group calls the systems that they are developing “contrastive local learning networks,” which he described with the remainder of his talk.
The work is done on networks of variable resistors (Figure 2-5). Some nodes are input nodes, where specific voltages are applied that represent the input data, and other nodes are designated as output nodes, where the output voltages are measured to provide the answer to whatever problem was being solved—determining whether a particular image was a cat or a dog, for instance. The goal is to adjust the conductances of the various edges in the network—that is, the legs containing the adjustable resistors—so that the output nodes produce the voltages corresponding to the correct answer. It is very similar to the mechanical networks that Mao described, with voltages and resistance in place of applied forces and spring constants.
Such a computational network would have some advantages, Durian said. For instance, as soon as the inputs are applied, the outputs appear immediately, and the computation is done. Also, the system is highly recurrent in the sense that the current flows in all directions. “There’s no sense in which there’s a flow of information in one particular direction,” he said. “It’s just kind of going everywhere.”
The system is contrastive in the sense that it learns by contrasting the behavior of the network under two sets of boundary conditions. The first set of boundary conditions, the free boundary conditions, consists of the relevant input voltages imposed on the input nodes along with the voltages on the output nodes that arise naturally from input voltages and the structure of the network before it has learned to provide correct answers. The second set of boundary conditions consists of the same input voltages plus voltages applied at the output nodes that correspond to the desired answer to the problem. These are called “clamped boundary conditions”
NOTE: Red dots represent inputs; blue dots represent outputs.
SOURCES: Presented by Doug Durian on October 2, 2025. From Stern et al. (2021). CC BY 4.0.
because the output nodes are not allowed to have the voltage they would normally have but are “clamped” to have the “correct” voltages. In this case, Durian noted, the currents that spring up throughout the network plus the voltage drops across the edges (i.e., the resistors in the network) are naturally different than they are under the free boundary conditions. By contrasting those two sets of local values, one can develop a learning rule that adjusts the edge conductances so that the output voltages get closer to the correct values.
There are various types of possible learning rules, Durian said, referring to a similar comment Mao had made in her presentation. Three of the best-known types of learning are contrastive Hebbian learning (Movellan, 1991), equilibrium propagation (Scellier and Bengio, 2017), and coupled learning (Stern et al., 2021), which, he noted, was developed by Liu and her postdoc Menachem Stern and collaborators. Durian’s work relies on coupled learning.
The traditional cost function that is minimized in learning networks is simply the square of the desired response minus the free response, for example, (desired response – free response)2; it is squared to make it a positive number. Durian’s group, however, uses a new contrast function, which is simply the power dissipation in the network under the clamped boundary condition minus the power dissipation under the free boundary condition. This does not have to be squared, he noted, because it is always a positive number. This is the case because the circuit will always try to minimize power dissipation, so for a given set of inputs the minimum power dissipation will be found under the free boundary conditions; when the output nodes are clamped at voltages other than the ones found under the free boundary conditions, power dissipation in the network will inevitably be greater than power dissipation under free boundary conditions.
Having defined this contrast function, which is physically positive, one then carries out a gradient descent on the function and obtains a learning rule that is purely local, Durian said. The conductance of a given edge, say edge J, is based on the voltage drop across edge J under clamped conditions minus the voltage drop across edge J under the free boundary condition. “Edge J figures out what it’s going to do based on what it feels,” he said. “It doesn’t need to know about anything else going on in the network. So it’s agnostic to the architecture of the network, and everything else. It does what it wants to do.”
Liu and Stern simulated this and showed that it works, Durian said. It was not obvious that it would work, he observed, because it was not obvious that the gradient descent on this contrast function would project well onto the gradient descent for the true loss function. Liu and Stern did the work in silico, Durian observed, which involved having to model the system and solve Kirchhoff’s laws for analyzing electrical circuits, which contain nonlinear elements. Because of that nonlinearity, the analysis does not scale well, he said. “This is an algorithm that’s not going to replace AI done at a computer. This is an algorithm that’s designed to be implemented in a lab.”
Carrying out the analysis via a physical resistor network has its own challenges, Durian said. “It’s great that it’s local in space, but it’s not local in time,” he said. “You’ve got two contrasting boundary conditions.” The naïve way to do the analysis would be to apply one boundary condition, measure the voltage drops across the different edges, and record them, and then apply the clamped boundary conditions, measure the voltage drops for that setting, then calculate the contrast function and do the calculations to figure out how to adjust the resistances on the different edges, and then finally change the resistances accordingly and do the whole process over and over again. Doing all of that time-consuming work would negate the benefits of using a learning physical network, he said, so his team found a way to speed things up.
“It’s what we call the twin network trick,” he said. They build twin networks on top of one another, with one dedicated to running the free boundary conditions and one dedicated to running the clamped boundary conditions (Figure 2-6). Because they sit one on top of the other, it is possible to build local circuitry connecting the corresponding components of the two networks. Initially, the corresponding edges in the two networks have conductances that are the same. However, because one of them has free boundary conditions and one of them has clamped boundary conditions, the voltage drops across the corresponding edges are different in the two networks. For each pair of corresponding edges, the connecting circuitry examines the voltage difference between them and either raises or lowers the conductance in the edge in the network with the clamped boundary conditions to bring the voltage drops closer together (Dillavou et al., 2022, 2024). Durian noted that the circuits were designed by his postdoc Sam Dillavou and built in his lab.
His team has built two generations of the twin networks so far, Durian said. The first generation involved digital potentiometers, or digipots, which are digital variable resistors. The resistance can be adjusted up or down by applying a pulse to one or the other of two pins on the device.
SOURCES: Presented by Doug Durian on October 2, 2025. “Nonlinear classification without a processor” (Figure 1) by Dillavou et al. (2023), used under CC BY 4.0 license with three figures shown.
With such a twin network it is possible to do various learning tasks, including classification tasks. For instance, Durian’s group carried out the same classification of iris species that Mao described, using petal length and width and sepal length and width to classify three different species of iris. In the work that Durian’s team did, they had photographs of 50 different irises from each of three species, of which 10 from each species were used for training and 40 for testing. They randomly chose four input nodes for the four measurements plus one input node for a ground, and they chose three output nodes at random for the three species. It did not matter which nodes were chosen, he said; the results were approximately the same for any selection. The network would reach about 95 percent accuracy after some 100 training steps, and, as a linear network, that was about the best it could do, he said. “So we’re not content to work strictly with linear networks,” he concluded.
Nonlinear Local Learning Networks
Moving beyond linear networks by having nonlinear edges would make it possible to do nonlinear tasks. “That’s where the real power of neural networks comes from, for example.” Durian said. “You want to be able to do nonlinear things.” To do this, his team collaborated with Marc Miskin from the University of Pennsylvania’s electrical engineering department, who worked with Dillavou to design a new generation of learning networks where transistors serve as the learning elements and the local memory is stored in terms of the voltage on a capacitor that is tied across the gates of the twin transistors (Figure 2-7).
“These guys have a nonlinear IV characteristic,” Durian said. “It’s a weird one, but it is nonlinear. There’s a linear regime, there’s a nonlinear regime, and what we do then is that we design some circuitry that sends a current to the capacitor that’s proportioned to the error that’s derived from the local learning rule. So, there’s a
NOTE: Gate voltage G is the learning degree of freedom.
SOURCES: Presented by Doug Durian on October 2, 2025. “Machine learning without a processor: Emergent learning in a nonlinear analog network” (Figure 1B) by Dillavou et al. (2024), used under CC BY 4.0 license with no changes.
local learning rule that will send current to the capacitor, and it could either add charge or take charge away from the capacitor, and it can learn.”
These networks learn all by themselves, he said, and they do it in an analog fashion rather than a digital fashion. To illustrate what such a network can do, he showed the results of a nonlinear regression task (Figure 2-8). Mapping output versus input, one can see that the initial answer has little in common with the training data, but the answer gets steadily better with time, first producing the mean of the training data, then the slope, and then the curvature, eventually matching the data as well as it can. “We can actually look at the dynamics of learning in these systems,” Durian said, “so it’s a great playground for measuring the dynamics of the emergence of learning.”
Next, he showed a video of the network learning over time to do a binary classification task, determining where the boundary lay between two types of dots scattered around a two-dimensional space with input and output as the two axes. Over time the boundary rose, rotated, and then curved until it settled in on a decision boundary that did a good job of separating the two classes. It was clearly a nonlinear task, Durian noted, because the decision boundary was curved. He noted also that the structure of the network was somewhat different from the earlier network. His team made it a lattice and also made it have periodic boundary conditions, which helped it learn. “It helps information—whatever information is defined as in the system—get smeared around the full network a little bit better by having these pre-added boundary conditions,” he said.
SOURCES: Presented by Doug Durian on October 2, 2025. “Machine learning without a processor: Emergent learning in a nonlinear analog network” (Figure 4A) by Dillavou et al. (2024), used under CC BY 4.0 license with no changes.
Next Steps
Looking to the future, Durian first addressed the issue of how far this approach might be effectively scaled up. “Why can’t we just make it a billion times bigger and, say, blow AI out of the water?” he asked. One conservative approach, he said, would be to use printed circuit boards, which would allow the group to go up to hundreds or maybe thousands of edges and racks. However, he added, thousands of edges are probably the limit on what they could do with printed circuit boards. Still, he added, this allows for much bigger networks than they can do with breadboards, and it also makes it possible for them to quickly turn around and refine designs. By contrast, the turnaround time for chips is much longer, and they are much more expensive than the printed circuit board approach. The team is using both approaches, depending on what they want to accomplish.
Some questions arise when thinking about scaling up these learning networks, he said. One question is whether the nonlinearity they have in the transistors is a good one. “The circuitry sometimes has a hard time finding how to take advantage of that nonlinearity,” he said. By contrast, he continued, there is the universal approximation theorem for artificial neural networks that says that there is a particular nonlinear activation function for how information is combined at a node that will make it possible to approximate any input–output relation. “So if you’re combining information on nodes, you know what nonlinearities you can work with,” he said. There is no universal approximation theorem for edges, however. “It’s a different problem,” he said. “We need a universal approximation theorem that will tell us what kind of nonlinearities to use. In the meantime, we can just try things out.”
There are also questions about the best network architecture. “If we continue with the square lattice and we sprinkle input nodes here and output nodes throughout, this particular output node isn’t going to know how to talk to this particular input node,” he said. “Information can kind of get diluted.” It may be necessary to use a sparser network and long-range connections or perhaps some complex, hierarchical network that resembles the brain. “These are big, big questions,” he said. “It’s not obvious.”
The effect of noise on these networks is another issue. These are physical systems, Durian said; they inherently have a certain amount of noise. How does that affect the system’s learning and performance? A related issue is the effect of physical imperfections in these networks. So far, he said, it seems that the networks are pretty robust to such imperfections as a broken edge or loose connection. The systems seem to learn around such things. “It’s like your brain,” he said. “It’s very robust to these manufacturing errors. Only occasionally does a manufacturing error for a particular edge make you have problems.”
One particularly attractive aspect of these learning networks, Durian said, is that they can be incredibly energy efficient. “We think if we could shrink them
down to chips, we could potentially achieve a factor of a million in energy efficiency over current AI approaches.” If so, it could potentially disrupt the current artificial neural network paradigm for doing AI, he said. But it is likely, too, that these learning networks find smaller-scale applications as well.
Question-and-Answer Session
In the discussion period, Durian first addressed a question from Varda Hagh about how quickly these networks learn. Durian said that it is difficult to address the issue of how fast they learn. “I don’t know how to experimentally address that because I don’t know how to experimentally do it on the exact loss function,” he said, but they clearly do learn. “Experimentally, and also in the simulation work that Nachi [Stern] has done, there’s clearly strong projection, because it does learn.” He added that a possible advantage of these learning networks is their potential to go in “random-ish” directions when finding an answer. “This could help you avoid getting stuck in local minima,” he said. “It’s a little bit like doing stochastic gradient descent.”
Workshop planning committee chair Lisa Manning of Syracuse University asked Durian if he had learned anything about the architecture of learning systems from his resistor and transistor networks that could point to some interesting directions for those working in mechanical or materials networks. One thing, Durian said, was that his team “made effectively all-to-all connections by some clever long-range jumping, and with those, we were actually able to do learning a little bit better.” But there is still much he would like to do that he has not yet figured out how to do. “I would love just to have random connections and then just let the system prune out things,” he said. “Let the architecture emerge, in a sense, by driving some conductances to zero.” But he does not even know what a good starting point would be for doing that, saying “so help us, please.”
Liu offered some additional details on the topic of the best architecture. “We’ve looked at this computationally a bit,” she said, “and my postdoc, Adam Kline, has shown that for the linear networks, the best you could do is with all-to-all between inputs and outputs and, in fact, any architecture for linear networks you can show just reduces to all-to-all between inputs and outputs.” However, she continued, for nonlinear networks it seems to be an open question as to which architectures are best. For electronic networks, all-to-all between layers is actually a very natural thing to implement, she said; what people have been doing with neural networks is actually what is also easy in electrical networks, although it is not necessarily so easy in mechanical networks and other kinds of networks.
Mahesh from Schmidt Sciences asked about how well Durian’s systems deal with noise, noting that modern neural networks have proven to be remarkably noise resistant. For instance, he said, DeepSeek’s latest models use only 8 bits of position for their parameters, although they still use 16 bits for the activations. So, he
asked, as the size of transistor networks increases, how will they handle noise? “Let’s say we want to actually show it for a billion parameters made in a foundry—what would the noise levels look like? Can we actually hit 8 bits of position at that level?”
Durian answered that since his networks are analog, not digital, he is not working with bits, and the noise question is somewhat different. “Do we need an equivalent, 1 part in 28 precision? I don’t know the answer to that.” To this point, he said, noise has not been a problem for his team, but that could change as they go to larger networks as the nodes get farther and farther apart and the voltage drops as you go across the system. It seems likely, he continued, that as they scale up they will need some way of boosting voltages so that each edge has a significant enough voltage drop across the system for the local electronics to sense. “As long as the local electronics can sense a nonzero voltage difference within whatever the noise level is, we should be good,” he said.
Brian Taylor of Cloud Capital asked how Durian’s network learning uses multiple training examples, as it seemed that the clamped boundary condition he used corresponded to a single training example. “It can or it can’t,” Durian responded. In some cases, every single member of the training set causes an update to the conductance. But there is also a “batching” approach in which the network sees all members of the training set before the edge is updated. “So, we can do it both ways,” he said.
LEARNING AND INFERENCE IN BIOMOLECULAR SYSTEMS
Erik Winfree, a professor of computer science, computation and neural systems, and bioengineering at the California Institute of Technology, turned the session’s topic from electronic systems that can learn to biomolecular systems that can learn and perform inference.
Information-Based Chemistry
To provide some context, Winfree explained that his interest lies not so much in developing molecular technologies that can learn, but rather in how to learn from what goes on inside a cell as a way to identify principles that apply to both biological cells and to future molecular technologies. Showing an illustration of a cell that was based on electron microscopy images, he observed that cells are packed full of different components. “We have proteins, RNA, DNA, all mixed in a bag,” he said, “and what’s not really shown here [in the illustration] is that it’s jumbling around. There’s Brownian motion, there’s thermal energy, things are moving, they’re sticking, they’re reacting.”
As a computer scientist, Winfree said, he is particularly interested in how the system inside a cell makes decisions. “How does it decide when to do one thing,
when to do another thing? How does it adapt to its environment—to learn, if you wish?” Is it possible to think about the processes taking place among all the molecules in a cell as algorithms of a certain sort? And if so, how are these kinds of algorithms programmed?
Perhaps, Winfree continued, these algorithms are programmed by learning, or perhaps they are programmed by other means. Either way, it would be valuable to understand the programming language for these systems. In this context, he continued, it is useful to think about a programming language as a way of articulating a design space for interesting behaviors. “A particular program gives you a behavior,” he explained, “and you want to be able to explore that space of all possible behaviors, and programming languages allow you to do that.”
Generally speaking, Winfree said, the design space of biology contains a wide variety of things that evolution has led to—but those things are not all that the design space contains. “If we’re thinking about the language and the design space more generally,” he said, “we’re not interested in what evolution came up with, except as inspiration. We’re interested in what’s possible, and what’s possible is much larger than what exists.” Specifically, he said, he is interested in the space of information-based chemistry, where the information in the molecules directs their behavior. Recent decades have seen a great deal of progress in engineering molecular systems—in particular, such things as protein design, RNA design, DNA nanotechnology, and the design of DNA systems—and this progress has made possible a great deal of experimental progress in the area of information-based chemistry. This is why, he said, that now is a good time for thinking about what the design space of information-based chemistry is.
Computation in Biomolecular Systems
The thesis of his talk, Winfree said, was that some of the concepts that were developed in the theory of neural computation, especially by John Hopfield and Geoffrey Hinton, provide a very valuable perspective for understanding the molecular world. And to offer some context for his talk, he offered a few details about what his laboratory is working on. “We’re interested in molecular programming with DNA nanotechnology,” he said. “Why DNA? Well, it’s just an example of information-based chemistry. It’s one that’s experimentally accessible. And we can look at different kinds of molecular systems.”
In particular, he said, there are three classes of systems in which the group studies information-based chemistry, and these three classes are related to the three phases of matter—gas phase, liquid phase, and crystal phase—and have three different prototypical types of molecular interactions. The first class, corresponding to the gas phase, can be thought of as a collection of small molecules that are dilute in solution, he said, “and every now and again they bump into each other and something happens, and then they go their way.” In practice, this could be realized by
a dilute liquid in a test tube with chemical reactions taking place among the small molecules. For example, if there were three small molecules, A, B, and C, then the set of reactions might be something like:
B + A → 2B
C + B → 2C
A + C → 2A
In the second phase, corresponding to a liquid phase, one has programmable condensates. “These are a phase that’s very dense,” he said. “The molecules are packed so tight that they’re always in contact with each other, with weak interactions so that they’re labile, they’re moving around, they’re making new contacts with different neighbors all the time.”
In the third phase, corresponding to the crystal phase, Winfree said, “we have another dense phase … where when molecules meet each other, they stick to each other sort of permanently, but their interactions are programmed.” In particular, he explained, which piece sticks to which other piece is programmed by some specific interface between the molecules.
Touching briefly on the connections that DNA nanotechnology has with neural networks, Winfree said that the deepest research in the area had been done by Lulu Qian, a bioengineer at the California Institute of Technology, who was attending the workshop. For the past 15 years, he said, Qian “has been developing, in the chemical reaction network space, ways of designing DNA molecules that interact according to the logic of certain models of neural networks.” In 2011 she built a four-neuron Hopfield neural network (Qian et al., 2011), and by 2018 she had built a 100-input winner-take-all network (Cherry and Qian, 2018). Just a week before the workshop, she published work in which a DNA neural network learned to classify patterns given as molecular examples (Cherry and Qian, 2025), and the day before the workshop she published a paper describing how to power DNA neural networks with heat, allowing them to run for a long time without additional energy input (Song and Qian, 2025). But, Winfree said, this talk would be on something different—some of the physical aspects of molecular systems.
Ising Systems and Their Generalizations
Winfree began with a discussion of Ising networks (Figure 2-9), which he described as one of the foundational concepts that led to a certain subset of neural network theory. An Ising system contains a number of nodes that can be in one of two states, on or off, or up or down, and there is an energy associated with each configuration of states. Two neighboring nodes can be in the same or opposite state, and there is an energy preference for being in the same state.
SOURCE: Presented by Erik Winfree on October 2, 2025. Courtesy of Erik Winfree and Cameron Chalk.
Hopfield’s model, which was adopted by Hinton, uses the same idea of a collection of nodes that are either on or off, but rather than having limited neighbors, all of the nodes are connected with one another, and rather than having the same energy for all neighboring interactions, there are varying weights ωij that give the energy for each pair of nodes so that every node is now different (Figure 2-10). Many modern neural network models are still based on this kind of energy formalism, Winfree said, although they are much larger, training is more efficient, and they are better in many other ways than these early models.
SOURCES: Presented by Erik Winfree on October 2, 2025. Courtesy of Erik Winfree and Cameron Chalk.
An important point here, Winfree said, is how the concept of energy was used to describe and analyze these systems. That is the standard physics approach, he noted. “But then in the neural networks, it was ‘energy’ in quotes. Mathematically, it was energy, it was the tools from physics for talking about energy, but there wasn’t an actual energy of the system.”
The second point Winfree made was similar to one made by Andrea Liu in her introductory remarks to the workshop. The Ising model has just one basic unit and two basic states, he said, “but you get lots and lots of them in a large network, and, poof, magnetism appears.” That is, a qualitatively new phenomenon emerges when there are enough copies of the same component. “More is different,” he said, quoting Nobel Prize–winning physicist Philip W. Anderson’s statement from a seminal 1972 paper in Science.
In Hopfield’s generalization of the Ising model, Winfree continued, different behavior emerged not from having more of a single type of thing but rather from having more types of things. “Every neuron is different,” he said. “Each neuron has its own set of weights, and so if you ask how many different kinds of things there are, well, there’s this neuron, there’s that neuron, there’s the other neuron. There’s many, many of them.” Scaling up the number of types of things in a system is another way of getting an emergent behavior such as intelligence.
Multicomponent Liquids
Winfree then transitioned to the topic of multicomponent liquids, one of the topics that he studies, and began by talking about how the ideas of Hopfield and Hinton transfer to this area. A multicomponent liquid has many different kinds of molecules with different attractions to other molecules, he explained, illustrating with an abstract model of a cube composed of individual parts of various colors representing the different molecules (Figure 2-11). The energy function describing this multicomponent liquid is given by a matrix with the various entries representing how strongly pairs of molecules stick to one another. This is a real physical energy of the liquid, he noted, not the “energy” that is found in a neural network. The N-by-N matrix representing the interactions between the molecules “allows us to program different kinds of behaviors into the system,” he said.
So, Winfree asked, how can these kinds of physical systems carry out the sorts of learning behavior, such as inference, that people are interested in? “Well,” he said, “what we’ll borrow from the neural network world is that equilibrium is a very interesting context for thinking about inference and learning,” and he proceeded to lay out the parallels between a physical system in equilibrium and probabilistic inference.
In equilibrium, he said, the standard arrangement is to have some state space in which each of the various states has an energy, there are neighboring states, and there are interactions between the neighboring states “that respect the energies so
SOURCE: Presented by Erik Winfree on October 2, 2025. Courtesy of Erik Winfree and Cameron Chalk.
the forward and backward rates are related to the difference in energies.” That is called a “detailed balance,” in which each elementary process is in balance with its reverse process, and what is important there is that when a system is at detailed balance, the probability of the states at equilibrium is described by a Boltzmann distribution, which is analytically very tractable.
The concept of probabilistic inference is very similar. “You have some probability space that you’re interested in,” Winfree said. “This is your knowledge of the world. Knowledge is represented in terms of probabilities.” In particular, there is a probability P(x) associated with each event or state x in the state space X; this probability can be thought of as being in parallel with the energy E(x) associated with each state in the equilibrium case. If there is a subset Y of the entire state space, then the probability of some event x happening given Y is the conditional probability P(x|Y) = P(x)/P(Y). In the statistical sense, Winfree said, this corresponds to “restricting your reachability from being able to reach everything in X to only being able to reach things in some subset Y,” and the conditional probability turns out to be the Boltzmann distribution on a restricted subspace—corresponding to the probability of states at equilibrium in the previous example. “You can think of that as inference,” he concluded.
Classic Models of Neural Networks
To make these arguments clearer, Winfree continued, he would quickly review some of the classic models of neural networks. Speaking briefly about the Hopfield model, he said that in the standard update for a neuron, if the weighted linear sum
of inputs is positive, then the value is flipped to +1; otherwise it is flipped to –1. “This update corresponds to choosing to flip a bit that decreases your energy,” he said, and therefore the dynamics involve starting at some point in the landscape and “rolling down the hill” to points of lower energy. “So you end up at these minima which are your memories.” But the main point, he said, is that a very simple Hebbian learning rule provides a way of sculpting this energy landscape such that the minima correspond to the examples you provided. The implication is that if one starts at some point where there is partial information—such as just a part of a photograph—then the dynamics of the system make it possible to recall the rest of the information—to fill out the rest of photograph, in this example. “So you have this point–attractor dynamic,” or, to use Hopfield’s term, an associative memory (Hopfield, 1982).
What Hinton did was to generalize the Hopfield model to a probabilistic framework where instead of going to and staying at a point attractor, one ends up exploring that probabilistic space continually, he said, “and so you can have different kinds of energy landscapes in a very meaningful way such as these continuous attractors where you can sort of explore, wander around, over a continuous space rather than a point space.” In this case, he said, the learning rule is a very simple contrastive approach in which the weight change depends on the state while clamped versus while moving freely.
High-dimensional probability distributions represent the world, Winfree said, as he displayed a set of digits 0 through 9 from the Modified National Institute of Standards and Technology (MNIST) dataset of handwritten digits which are used to train machine learning models and image-processing systems. “There’s all sorts of ranges of possible patterns, and what you want to do is capture that probability distribution,” he explained. “Learning will eventually give you a set of weights between the visible and the hidden units.” He chose to illustrate this with the MNIST digits, he said, because they help illustrate why, instead of wanting to get rid of randomness in the network behavior, researchers should embrace it. “Noise is actually what we’re trying to represent,” he said. “We’re trying to represent the probability distribution.”
Showing a video of shifting shapes, Winfree described it as an exploration of the state spaces in the collection of MNIST digits. The interesting thing about inference, he said, “is that inference happens naturally in this sort of exploration of an energy landscape when you clamp certain nodes…. If you freeze certain parts of the system and don’t update them and just let the other parts update, that’s your conditional probability distribution being sampled from, and in this case, that corresponds to recognizing that this was a 7.” One can clamp various things and get varying outcomes, he said. “For example, in this case we clamp class nodes so that it’s a 7, and we let everything else do its random thing, and eventually it starts reconstructing sort of samples of different kinds of 7s.”
An important point, Winfree said, is that inference is really omnidirectional. “You don’t explicitly build in the notion of an input and an output,” he explained. “Input and output aren’t really concepts here. There’s just what you clamp, what you don’t clamp. Information flows in all directions.”
Winfree next described how such a system is used for learning. The basic idea, he said, is that when the system is free it has some probability distribution over all the nodes, “and for the nodes that you’re going to be clamping, there’s a target distribution describing how the environment drives the system, and you want to make these two distributions match as well as possible.” The distance between the distributions can be measured by their relative entropy. When this is differentiated, he continued, “what falls out is this mean correlation between nodes so that your change in weights with time, if you’re doing gradient descent, is exactly this difference.” That is the classic result, he said. During training, the environment clamps visible units while the hidden units are sampled, enabling learning to adjust weights of hidden units without backpropagation (Ackley et al., 1985).
The amazing thing about this rule, he continued, is that it is agnostic to the direction of computation, to how many hidden species there are, and to the connectivity of the network. This is what makes it very powerful for looking at natural molecular systems, he said, because in working with those systems there is not necessarily an input and output direction, and there are not necessarily constraints on the architecture. “If everything interacts with everything, we’re still okay as far as the learning rule is concerned,” he said.
Molecular Systems
With that background, he quickly discussed the molecular systems that his team has been working with, beginning with dilute gas-phase chemical reaction networks (Poole et al., 2017, 2022). In such a network, various chemical species bump into one another, and some of them change through chemical reactions. It can be described by writing down the energies of the different molecules and their interactions and summing to get the energy of the full system. If the system satisfies the detailed balance condition, he said, then there is a “Boltzmann distribution which now is sort of partitioned into entropic terms and those that are due to the energies of the specific species.”
What is surprising about this, he continued, is that exactly the same type of argument that Hinton used in analyzing neural networks “can be derived with the same logical steps to show that you can reduce the relative entropy and do gradient descent, again by looking at the difference between the clamped and free-running averages of counts, now of individual molecules rather than correlations.” If there are N species, Winfree said, one has N energies to “play with,” and one needs a number of hidden species in order to represent the same kinds of probability dis-
tributions that Boltzmann machines do. However, he said, it can be shown that the full space of Boltzmann machine probabilities can be represented here as well. “It just takes more species and more reactions.”
He next made a similar argument for condensates, or what his team refers to as Boltzmann liquids (Chalk et al., 2024). The overall energy of the system depends on a chemical potential for each molecule as well as energies that depend on the interactions of molecules with neighboring molecules. And, as before, there is a Boltzmann distribution. Inference involves freezing part of the system so that those molecules are stuck in place—for example, if there are sticky domains on a surface that hold onto particular molecules—but with the rest of the molecules bouncing around. “That does conditional probability distribution inference,” Winfree said, “and it sort of infers that this is the arrangement that is likely according to the probability distribution for the liquid, given this input.” One can consider different kinds of inputs (i.e., freezing different parts of the system) and “the inference afterwards in the conditional probability distribution will be sort of the output of that,” he said. “This has the same kind of learning rule that derives from gradient descent on the relative entropy. So now it’s how many copies of species type i are next to species type j,” which determines the learning for the interaction energy between species i and j.
He skipped describing the analogous work his lab has done in the crystalline phase, pointing to a publication that describes it (Evans et al., 2024; see also Murugan et al., 2015). Then Winfree made a few overall points about the work. They do see interesting energy landscapes and multistability in the work, he said, and they can get “sort of” analogs of Hopfield network memories. All of this is done at equilibrium. The process is not using energy; it is just minimizing energy and dissipating heat. “The energy for the computation actually comes from the environment that provides the clamp,” he said.
He closed with a look to the future. There are many different kinds of molecular systems, he said, not just condensates and chemical reaction networks but also such things as polymer folding, self-assembly, nucleation, phase separation, molecular motors, molecular constructions, and systems that combine more than one of these. “If one is looking at the equilibrium behavior for these kinds of systems,” he said, “we expect that when you write out the energy function, you can follow the same kind of gradient descent on relative entropy to come up with learning rules for a wide range of molecular physical systems that are representing probability distributions” and then use them to do inference via being driven or clamped by their environment.
Question-and-Answer Session
During the following question-and-answer session, workshop planning committee chair Lisa Manning began by requesting more details on the update rule in
a system of condensates (the “liquid phase”) and asking how that rule would be implemented in a real system. Mathematically, Winfree answered. The rule is saying that the system is running two phases, one clamped and the other not clamped. When it is being clamped, he said, then the energy should be made stronger whenever two molecules are close to each other. At this point, he added, the rule is sort of a guideline for finding a physical mechanism for doing it, but the lab has not yet demonstrated any physical mechanisms for doing the learning. More specifically, he added that the update might be done indirectly by having molecules that are close together recruit some other kind of molecule that mediates that interaction or synthesizes a molecule that mediates the interaction. But it would be something purely local that involved strengthening an interaction whenever things are close to each other. Conversely, when molecules are not close to each other, one would need to undo their connection.
Manning then asked a follow-up question, assuming that there would be some sticky molecules on a surface on which there was a clamped condition. “How do I know, when that surface is being presented, that I have to do this sticky update rule and, when that surface isn’t being presented, then I should not be doing that sticky update rule?” she asked. “That seems hard to encode.”
Absolutely, Winfree answered. Murugan has a paper in which he looks at time differences, he said. “You clamp things and then you let them run free, and you sort of subtract that. You clamp them, add that. And then if you have some kind of temporal difference, you might be able to do that, but that really is an open question.”
One of the things that is most powerful about Hinton’s original formulation is that the visible/hidden distinction is not really a hard-coded thing, Winfree said, just as there is no hard-coded input and output. One can conceive of the rule in terms of clamping more and clamping less, he said, and that could be done in any biological system where some inputs might be driven by the environment while other ones are not driven by the environment, so they are not being clamped. A system in which things are getting more clamped does Hebbian-type updates, while a system where things are getting less clamped does anti-Hebbian updates, he explained. “That can happen for arbitrary amounts of clamping and variations in how much the environment is clamping,” he said. “I think of looking at that difference in terms of how much the environment is clamping as being the sort of internal signal that a physical system would have to pay attention to.”
Laurel Kroo from the Department of Polymer Science at the University of Massachusetts Amherst asked about Winfree’s use of the phrase “equilibrium is inference.” “Can you speak to the philosophy of Boltzmann liquids out of equilibrium and nonequilibrium conditions?” she asked. “If equilibrium is inference, what is nonequilibrium to these machines?”
Winfree said that he had included that phrase on his slide just to “provoke” the audience. “It’s not true, but it sticks in the mind,” he said. Many systems are
not equilibrium systems, and that is very important, he continued. For instance, the Hopfield network model is not an equilibrium model. All point attractors in the energy landscape do not have to be the same depth, he said; it is just that at zero temperature, you go to the nearest one. That kind of kinetic trapping is very important in many systems, he added. There are also phenomena such as nucleation that are inherently kinetic where it does not matter how deep the energy well is, but rather which energy well is easiest to obtain nucleation or some other phenomenon.
The bigger picture, Winfree said, is that thinking about the connection between neural networks and energy landscapes “is really the bottom line of understanding how energy landscapes can represent very complex information that a system has acquired,” with how to sculpt that landscape with respect to one’s experiences being the critical question. “But once you’ve got the landscape, … you can ask kinetic questions or equilibrium questions.”
Arvind Murugan from the University of Chicago commented that Lulu Qian’s work was somewhat at odds with what Winfree had described. Qian’s paper relied only on the Hebbian part in some sense, he said. “It’s not described by energies; it’s full of irreversible reactions, and it just works. So the question is: Should we take the theory seriously and then try to find a physical system that works, or are there just completely other principles where all you do is Hebbian, there’s no concept of energy, and it still works out?”
Of course there are systems that work, Winfree answered, and he commented that the larger context of Murugan’s question was what the purpose of theory is. “I would say when you have a theoretical perspective, it really guides your way of thinking about things, guides the kinds of questions you ask, and in my lab, guides the kinds of experiments one ends up doing,” Winfree said. “So thinking about energy landscapes is extremely productive in that sense.” He went on to say that there is a long history of learning algorithms that are not based on energy and are not based on gradient descent and that do work to a certain extent within their limits. “I don’t think Lulu is going to say that her molecular learning system is the be-all and end-all of molecular learning systems.”
Workshop planning committee member Andrea Liu asked which of the types of systems Winfree talked about had actually been implemented experimentally. Winfree answered that his group has implemented some examples of all of the kinds of systems experimentally. They have not published their condensate work, but he said that there are other groups that have done very beautiful things with DNA-based programmable multicomponent condensates. None of the systems that he described had the learning being done at a molecular level, so some of them used learning in silico to figure out such things as which tiles should stick to which other tiles. At this point, he continued, Qian’s experimental system is the only one he knows of that implements a complex neural network with learning done at the molecular level.
Winfree further explained that his group’s systems are programmed by sequence design. They design DNA molecules that will bring the desired molecules together in the desired ways, and they send the sequences to a company that produces the desired DNA and sends it back in a collection of tubes. “We mix the tubes together, we heat it up, we cool it down, and we look at what happens, and that’s the kind of experiment that we do,” he said.
DISCUSSION
Martiniani opened the discussion session by observing that the three presenters seemed to be doing very similar things, just talking about it in different ways. “Even the fundamental mathematical framework seems to be the same,” he said. Then he turned to Winfree specifically and asked how his work was different from machine learning models. It seems, he said, almost as if Winfree is taking what is understood from machine learning and neural systems and trying to engineer that in a different process. So, what is unique about implementing these algorithms in physical systems and how do they learn differently? “If all we’re doing is taking an algorithm and implementing it using a molecular network or a resistor network or a mechanical network, then that’s not so deep, right?” he said. “But if we can somehow leverage intrinsic properties of the systems to aid learning, then I think we’re making an advance.” He asked Winfree whether the substrate introduces unique characteristics of the learning process that make the learning achieved in these systems different from what artificial neural networks do.
Winfree pushed back on the assumption that the work with learning materials is essentially the same as what has been done with neural networks. First, he said, at the theoretical level there are significant differences in how one poses the tasks that are being learned. Is the goal to learn a full probability distribution, to just get conditional probabilities right, or perhaps not to learn a probability distribution but rather to learn a continuous input/output function? It is worth paying attention to these differences, he said. Second, in terms of implementation, the physical systems being used for learning are not doing exactly what the neural network models are doing. Researchers take the concepts and principles from the work on such things as neural networks and see how they apply in these physical systems, but the physical systems have their own inherent dynamics.
Martiniani followed up by asking whether creating and implementing these physical systems help. What extra do they provide? Do the physical systems, for instance, change how a model learns or what it learns, and is that helpful?
Winfree answered by first offering a few details about Qian’s work. The basic approach there was to take a neural network model and then design molecules to carry out different aspects of that model—for weighting a signal, for adding things
up, for comparing a signal to a threshold, and so on. “You can implement your learning tool that way, and it works and it’s amazing,” he said. But what he was talking about was taking the opposite approach, he said. “We started with ‘here’s what the molecules do, here’s their energy landscape, and how does that make decisions, how does that do inference naturally due to the physics of the situation?’” They are able to observe such things as the difference between when a molecule is free in solution and when a molecule joins a condensate, and they find that difference in energy to be a weighted linear sum with a threshold, so whether that step is physically favorable or physically unfavorable is inherently performing the computation of a weighted linear sum compared with a threshold. “I think this is a physical insight about the molecular system.” Winfree said. “It’s not necessarily technologically better. It might not work better. But it’s an insight about how molecules work that draws on the theory of neural computation, and I think that insight is actually the bottom line.”
Durian agreed with Winfree but phrased it somewhat differently. “How can you look at a physical system and decide what it is primed to learn?” he asked. For example, the human brain is primed to learn motor control and language. Memristor arrays are primed to carry out matrix multiplication. So, the question becomes, “What are the physical learning networks such as those described in the session primed to learn?” Or, alternatively, if there is some task one wants such a network to learn, how does it need to be built? Ultimately, Durian said, these physical learning networks are not going to be universal computational devices. So one must ask, “What are they good at? How can we figure that out?”
Liu commented that the field is now at the stage where researchers are asking questions and starting to answer them, rather than having all the answers. These physical systems have to learn by local rules, and those local rules are often imperfect in terms of how they project onto gradient descent, so what are the advantages and disadvantages of this? “We don’t even know that,” she said.
As an example of how learning physical systems differ from neural networks, she pointed to work by Lisa Manning and co-workers on the vertex model of epithelial tissues. It was shown that a certain developmental process called convergent extension could be modeled with gradient descent on a computer or with different biologically plausible local rules, and the predictions could be tested experimentally. “The one that actually fits the experiment the best is a biologically plausible local rule, and gradient descent is completely different,” she said. “There we can see that the local rule really is leading to different results than you would get from gradient descent.” The question, then, is what the advantages and disadvantages are of the different approaches. “Biology manages to do what it does quite robustly, and maybe that’s part of it,” she said.
One other thing, she added, is that when using local rules—even when it is done computationally—they are able to do reasonably complex things and never
need to introduce regularization. “We don’t need to, and perhaps that comes from the fact that the local rules are imperfect projections onto gradient descent,” she said.
Viola Vogel from ETH in Zürich asked Winfree a multipart question relating to equilibrium and nonequilibrium systems. Winfree began by saying that in DNA nanotechnology, much of it is in fact not in equilibrium. “It’s usually actually quite challenging to get things into equilibrium,” he said. “Once you work with molecules of this size with this much information, kinetic traps are all over the place.” Sometimes researchers use these kinetic traps to drive the system to a desired state. In other cases, he continued, they want the system to be in equilibrium. “So yes, there’s a whole range to explore there.”
Switching to DNA origami, which Vogel had mentioned in her question, he said that some other groups had shown how DNA origami can do shape changing at equilibrium by adding a number of intermediating strands called staple strands that make various shapes. Later they changed the ambient set of mediating strands, and over time replacement processes changed the previous shape into a new equilibrium structure. “That is fairly slow,” he said, “but, yes, the field is investigating that, and it’s very interesting.”
Finally, he challenged the idea that all of the interesting phenomena are found in the nonequilibrium states. “I think equilibrium, despite its age, is far more interesting than many people appreciate,” he said. For example, the classic 40-year-old Boltzmann machine model can do inference at equilibrium because it is using Boltzmann distributions, he said, indicating that equilibrium can have incredibly complex distributions and can do inference by going to equilibrium when clamped. “Biology surely exploits equilibrium every chance it can to avoid using information,” he said, “and fully exploiting the potentials of equilibrium subprocesses is part of the reason biology is so energy efficient.” He said that he is coming around to the position to “only use nonequilibrium when you absolutely have to.”
Durian followed that up by saying that in the physical systems there are actually two equilibrium processes to consider. “One is the physical equilibration, say, of the network,” he said. “If you apply voltage, how do the currents relax to equilibrium?” The other is relaxation to the equilibrium of the learning system, that is, the learning dynamics. There are two different equilibration processes to keep in mind, and in the network that Durian’s group builds, the relaxation to learning is slower than the relaxation to the physical network, although they can modify the speed of relaxation by, say, putting capacitors across the edges so that the equilibration of the currents happens on the same timescale that they are updating the edges. Surprisingly, he added, that does not degrade the learning quality or the dynamics significantly until the two timescales are made
comparable. But the main point, he concluded, is that there are two different equilibration processes to keep in mind.
Liu followed up on that by offering the example of proteins, which are pretty much always at thermal equilibrium, she said, “and have all these amazing functions, not always, but often, and they evolved at thermal equilibrium.” During evolution, she continued, the time for the protein conformations to equilibrate is extremely fast compared with the evolutionary timescale over which amino acid sequences change and therefore tune interactions along the protein. “In other words,” she said, “each point in the sequence of the protein has a choice of one out of 20 amino acids, and that choice is made by the evolutionary process. That’s how the tuning is done, and that gives you basically all the proteins and all their functions.” And the rate of physical equilibration that Durian was talking about was very, very fast compared with the evolutionary timescale, she said, so that is a case where the learning is basically taking place in equilibrium. More generally, she added, there are many biological processes that are done essentially in equilibrium.
Martiniani asked about the emergence of types and hierarchies in systems that learn. Are the individual units that researchers are using adaptable enough to be able to reproduce the kind of behavior that is observed naturally in intelligent systems? Neurons have many different subtypes, he noted; for instance, there are various kinds of excitatory and inhibitory neurons. Furthermore, he said, they can adjust their tuning curves so their individual behavior can change. “Do we have any evidence that that also happens in the physical system that we’re considering,” he asked, “or do we need to make changes to the fundamental units that we study so that we have this kind of adaptability?”
Durian said that, as an experimentalist, he does not have a lot to say about that issue yet, because the systems they work with are still very small. “But if we make them large enough,” he said, “then it would be wonderful to look at the result of learning and see how some effective network with some large-scale structure of connections emerged from that process.” Then, from that it might be possible to get better ideas about the sorts of wiring that should be used to initialize the system to help it learn.
Mao added that one of the most powerful concepts in statistical mechanics is renormalization, “and that is really targeting this sort of question—what kind of collective degrees of freedom really describe the system?” There are a number of questions that could be addressed in this area, he said, such as whether it is possible to combine these sorts of renormalization criticality questions with learning. “But I think that’s a very new question,” she concluded. “We don’t know a lot yet.”
Liu said that biological systems have many different types of neurons, while the systems that Durian, for example, is implementing rely on just one type of com-
ponent. It can be adjusted to many different conductances, but it’s still one type of component. Neural networks increase their power by getting bigger, she noted, but they do not scale particularly well. Biological systems, on the other hand, tend to add more types of components rather than simply more of the same type. “That’s something we need to think about,” she said. “What happens when you have many different sets of tunable degrees of freedom?”
Durian said he agreed that the systems could probably be made better with a mixture of different kinds of learning elements. On the other hand, he said, people who work with artificial neural networks have only one kind of activation function. Why? Could they not improve the power with different kinds? “I don’t know the answer to that,” he said, “but they seem to do well with one kind of activation function.”
Varda Hagh from the University of Illinois Urbana-Champaign referred to the idea of having multiple types of tunable degrees of freedom and asked if there might be a systematic way to map out for what types of learning these degrees of freedom are most useful. Liu said she did not see how to do that at present but agreed that this is the ultimate goal. “We need these methods,” she said. “This is why we need people to work on it, bring all different kinds of techniques to bear” because the field does not yet have the empirical knowledge to even know where to start in terms of developing these kinds of frameworks.
Daniel Suarez from the University of Maryland, Baltimore County, asked about the level of control on each node that is achievable in these systems and what happens if it is not possible to control all of the nodes. Winfree suggested that the question was whether it is okay if there is learning on some aspects of your network but not on other aspects of the network. “I think in many cases we can expect that that’s fine,” he said. “If I had a large molecular network and only some of the interactions were tunable, then the rest of the interactions would be sort of considered fixed.” If they are fixed in a random way initially, he said, then the system is in a regime that has been studied, known as reservoir computing, where there is some random set of interactions “but those are rich enough that by tuning a few extra layers of tunable parameters, you can still access a very wide range of behaviors, and so I think that’s quite common and likely to be okay.”
Durian added that it is possible to have dumb edges, and they do not degrade the behavior. The key thing is to have a large number of degrees of freedom compared with the number of constraints. “As long as you’ve got that, you’re good,” he said. “Dumb edges don’t hurt. They don’t help, either.”
REFERENCES
Ackley, D. H., G. E. Hinton, and T. J. Sejnowski. 1985. A learning algorithm for Boltzmann machines. Cognitive Science 9:147–165.
Alù, A., A. F. Arrieta, E. Del Dottore, M. Dickey, S. Ferracin, R. Harne, H. Hauser, Q. He, J. B. Hopkins, and L. P. Hyatt. 2025. Roadmap on embodying mechano-intelligence and computing in functional materials and structures. Smart Materials and Structures 34:063501. https://doi.org/10.1088/1361-665X/adb7aa.
Anderson, P. W. 1972. More is different: Broken symmetry and the nature of the hierarchical structure of science. Science 177:393–396.
Chalk, C., S. Buse, K. Shrinivas, A. Murugan, and E. Winfree. 2024. Learning and inference in a lattice model of multicomponent condensates. Leibniz International Proceedings in Informatics, Vol. 314. Available at https://www.dna.caltech.edu/Papers/Chalk-2024-Boltzmann-Liquids-DNA30.pdf.
Cherry, K. M., and L. Qian. 2018. Scaling up molecular pattern recognition with DNA-based winner-take-all neural networks. Nature 559:370–376.
Cherry, K. M., and L. Qian. 2025. Supervised learning in DNA neural networks. Nature 645(8081):639–647.
Dillavou, S., M. Stern, A. J. Liu, and D. J. Durian. 2022. Demonstration of decentralized physics-driven learning. Physical Review Applied 18:014040.
Dillavou, S., B. D. Beyer, M. Stern, M. Z. Miskin, A. J Liu, and D. J. Durian. 2023. Nonlinear classification without a processor. Presented at the Machine Learning with New Compute Paradigms on December 15, 2023. https://openreview.net/forum?id=4P65OwisG9.
Dillavou, S., B. D. Beyer, M. Stern, A. J. Liu, M. Z. Miskin, and D. J. Durian. 2024. Machine learning without a processor: Emergent learning in a nonlinear analog network. Proceedings of the National Academy of Sciences 121(28):e2319718121.
Evans, C. G., J. O’Brien, E. Winfree, and A. Murugan. 2024. Pattern recognition in the nucleation kinetics of non-equilibrium self-assembly. Nature 625(7995):500–507.
Hopfield, J. J. 1982. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences 79(8):2554–2558.
Li, S., and X. Mao. 2024. Training all-mechanical neural networks for task learning through in situ backpropagation. Nature Communications 15:10528.
Li, S., and X. Mao. 2025. Topological mechanical neural networks as classifiers through in situ backpropagation learning. arXiv. https://doi.org/10.48550/arXiv.2503.07796.
Movellan, J. R. 1991. Contrastive Hebbian learning in the continuous Hopfield model. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski, and G. E. Hinton (eds.), Connectionist models: Proceedings of the 1990 Summer School. San Francisco, CA: Morgan Kaufmann. Pp. 10–17.
Murugan, A., Z. Zeravcic, M. P. Brenner, and S. Leibler. 2015. Multifarious assembly mixtures: Systems allowing retrieval of diverse stored structures. Proceedings of the National Academy of Sciences 112(1):54–59.
Poole, W., A. Ortiz-Muñoz, A. Behara, N. S. Jones, T. Ouldridge, E. Winfree, and M. Gopalkrishnan. 2017. Chemical Boltzmann machines. In R. Brijder and L. Qian (eds.), DNA Ccomputing and Molecular Programming: DNA 2017. Lecture Notes in Computer Science, Vol. 10467. New York: Springer. https://link.springer.com/chapter/10.1007/978-3-319-66799-7_14.
Poole, W., T. Ouldridge, M. Gopalkrishnan, and E. Winfree. 2022. Detailed balanced chemical reaction networks as generalized Boltzmann machines. arXiv:2205.06313.
Qian, L., E. Winfree, and J. Bruck. 2011 Neural network computation with DNA strand displacement cascades. Nature 475(7356):368–372.
Scellier, B., and Y. Bengio. 2017. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience 11:00024.
Song, T., and L. Qian. 2025. Heat-rechargeable computation in DNA logic circuits and neural networks. Nature 646(8084):315–322.
Stern, M., D. Hexner, J. W. Rocks, and A. J. Liu. 2021. Supervised learning in physical networks: From machine learning to learning machines. Physical Review X 11:021045. https://doi.org/10.1103/PhysRevX.11.021045.
Zhang, G., D. J. Heeger, and S. Martiniani. 2025. Contrastive self-supervised learning as neural manifold packing. arXiv 2506.13737 [cs.LG]. https://arxiv.org/abs/2506.13717.
