Why Yudkowsky is wrong about "covalently bonded equivalents of biology"
confidence level: I am a physicist, not a biologist, so don’t take this the account of a domain level expert. But this is really basic stuff, and is very easy to verify.
Edit: I have added a few revisions and included a fact check of this post by an organic chemist. You can also read the comments on the EA forum to see Yudkowsky's response.
Recently I encountered a scientific claim about biology, made by Eliezer Yudkowsky. I searched around for the source of the claim, and found that he has been repeating versions of the claim for over a decade and a half, including in “the sequences” and his TED talk. In recent years, this claim has primarily been used as an argument for why an AGI attack would be extremely deadly. I believe this claim is factually incorrect.
The quotes:
I’m going to show the various versions of the claim I found below, with the relevant sentences bolded:
To plausibly argue that “humans” were intelligently designed, you’d have to lie about the design of the human retina, the architecture of the human brain, the proteins bound together by weak van der Waals forces instead of strong covalent bonds
-Yudkowsky discussing the flaws of evolutionary design, in “the sequences” blog post “dark side epistemology”.
It was obvious years before Nanosystems that molecular nanomachines would in fact be possible and have much higher power densities than biology. I could say, "Because proteins are held together by van der Waals forces that are much weaker than covalent bonds," to point to a reason how you could realize that after just reading Engines of Creation and before Nanosytems existed.
- Yudkowsky discussing AI interventions on the alignment forum.
A lot of the advantage of human technology is due to human technology figuring out how to use covalent bonds and metallic bonds, where biology sticks to ionic bonds and proteins held together by van der Waals forces (static cling, basically)
-Comment on a post discussing technology and AI.
Algae are tiny microns-wide solar-powered fully self-replicating factories that run on general assemblers, "ribosomes", that can replicate most other products of biology given digital instructions. This, even though the proteins are held together by van der Waals forces rather than covalent bonds, which is why algae are far less tough than diamond (as you can also make from carbon). It should not be very hard for a superintelligence to repurpose ribosomes to build better, more strongly bonded, more energy-dense tiny things that can then have a quite easy time killing everyone.
-Yudkowsky’s example scenario for how an AI could extinct humanity, on twitter
Can you build your own synthetic biology, synthetic cyborgs? Can you blow straight past that to covalently bonded equivalents of biology where instead of proteins that fold together and are held together by static cling, you have things that go down much sharper potential energy gradients and are bundled together, people have done advanced design work about this sort of thing.
-Yudkowksy’s Ted talk, again discussing AI capabilities, during the Q&A section.
I broadly endorse this reply and have mostly shifted to trying to talk about “covalently bonded” bacteria, since using the term “diamondoid” (tightly covalently bonded CHON) causes people to panic about the lack of currently known mechanosynthesis pathways for tetrahedral carbon lattices.
-Yudkowsky’s response to my recent article a few weeks ago, talking about how to refer to potential advanced nanotechnologies.
Summarising the claim
As you can see, Yudkowsky has repeated this claim several time over a time period spanning 15 years to just a few weeks ago, in very high profile contexts.
These quotes all make roughly the same argument, which I will sum up as follows:
Proteins are held together by weak van-der-waals forces, which are weak forces, akin to static cling.
In contrast, alternatives to biological proteins could utilize strong covalent bonds, and would therefore be much more powerful.
Although the claim is deployed in a few different contexts, I will focus this article on the context of molecular nanotechnology, where it forms part of an argument for the potential deadliness of rogue AGI.
Bond types
Let’s start off by defining four common types of bonds. These are not the only types of bonds (for example, metallic bonds are strong but will not be discussed here). Note that the following definitions are all chem 101 simplifications. The bond strengths vary depending on which atoms are bonded and a bunch of other factors, but I just want to give a general picture here.
Covalent bonds: Covalent bonds arise when atoms “share” electron pairs between them, allowing them to get closer to a complete “shell”. These bonds are very strong, typically being hundreds of kJ/mol, depending on the atoms involved.
Ionic bonds: Some atoms really want to get rid of electrons, and some atoms really want to receive extra electrons. If one of each meets, an electron will jump from one to the other, making each atom oppositely charged, so they stick together. Ionic bond strength can vary quite a bit, with variations between 170 and 1500 kj/mol.
Hydrogen bonds: These bonds occur because hydrogen only has one electron. If that one electron covalently bonds with another atom, then it’s stuck on one side of the hydrogen, so the other side is positively charged. At the same time, another nearby atom, like oxygen, might have unbonded electrons hanging out preferentially on one side. This brings the hydrogen and other atom together electrostatically, where they bond. Compared to the previous two, this is a comparatively weak bond, typically around 10 kJ/mol.
This is the name for a collection of very weak forces acting between atoms. Primarily these are a result of fluctuating polarisations on neighbouring atoms, and tend to be transient and weak. The strength varies depending on definitions, with one source telling me 0.4 to 4 kJ/mol, and wikipedia mentioning strengths as low as 0.04 kj/mol. They are generally defined by their weakness.
A definitional note: Some sources will include hydrogen bonds as a subset of Van der Waals forces, as they both count as intermolecular forces. It is more common to separate the two definitionally, such as by saying that hydrogen bonds are permanent dipoles, while Van der Waals forces are not. The main reason to do so is that hydrogen bonds are generally much stronger and more permanent than Van der Waals forces.
Edit: Talking merely about the types of bonds can be misleading when talking about material strength, as the amount of bonds also matters. A chemist told me that "long series of hydrogen bonds or even vdW forces (in things like UHMWPE fibers) can be stronger than single covalently bound strands". This is another problem with the quotes above.
Bonds in proteins
Now that we’ve got these definitions in place, let’s get to the question at hand. What forces are present in biology? The image below shows the primary structure of a protein chain.
What percentage of the bonds shown below correspond to each of the four bond types I listed above? I encourage you to take a guess now, before scrolling down.
The answer is as follows:
100% of the bonds are covalent.
Okay, I’ll admit, I have played a small trick here, and to explain why, I’ll have to explain a bit more about how proteins work.
To build a protein, a ribosome will read DNA to choose from a set of 22 available amino acids, and then stitch them together like beads on a string. Each amino acid is held together almost entirely by covalent bonds, and the peptide bonds between amino acids are also covalent in nature. This initial string is called the “primary structure”. The R1, R2 and R3 in the above picture represent “side chains”, extra bits that are unique to each amino acid and can have a significant effect on it’s behavior. These side chains are also primarily (or possibly entirely?) covalent in nature.
However, a long string is not a stable structure. As soon as this bead is spit out, it starts wiggling around and folding up until it’s reached a relatively stable 3d structure. The “protein folding problem” of predicting how this fold would occur was a super-hard problem, which was recently cracked using deep learning techniques by Deepminds Alphafold 2 program, which can now predict 3d structures from base structures with very high accuracy. Folding structure is often divided up into three or four regimes of structure as shown in the following image:
Okay. So we’ve established that primary structure is overwhelmingly covalent, let’s move on to the “secondary structure”. This is where the chain folds up into itself so that the “backbone” (the none side-chain part) of one section bonds with the backbone of the other. This usually takes the shape of either parallel sheets or helices, as shown in the image below.
Feel free to estimate once again, for the next image, how the bond types are divided up:
I did a rough count, and in the sheet pictured it’s roughly 90% covalent bonds. The other 10% are hydrogen bonds. It looks like a similar proportion applies in the helix, although the picture doesn’t show the other side.
Up at the tertiary level, everything becomes a lot less predictable: proteins can fold every which way, to make a huge variety of structures, which depend a lot on the individual amino acid side chains that make up the structure. The following image shows a few ways that the protein is pinned together at a higher level.
I’m not going to ask to predict the bonds this time, because I think it’s highly material dependent at this point.
We can see that in tertiary structure, side chains of the main protein can bond to each other in a ton of different ways. We can have ionic bonds, hydrogen bonds, Van der Waals interactions, and… yep, covalent bonds, in the form of disulfide links. These can act as sort of “pins”, locking folded structures in place.
This website has a nice 3d model that lets you play around and explore the different types of bonds in a complex protein.
So at long last, we have all four types of bonds available, and the proportions of each is just gonna depend on the exact sequence of amino acids used. However, for most if not all proteins, the order of prevalence will be covalent bonds>hydrogen bonds> everything else.
To summarize, proteins are held together by a combination of bond types, primarily covalent bonds and secondarily hydrogen bonds, with other forces playing some part, depending on the protein.
Edit: I think that the previous summary was a little muddled. I would summarize it now as follows:
The primary structure of protein is held together with covalent bonds, with the secondary backbone held together by hydrogen bonds. At the tertiary level, these chains are held together by a combination of many different bonds and forces, such as hydrophobic interactions, hydrogen bonds, Van der Waals forces, and covalent bond links, all of which contribute to it's 3D structure with differing importance depending on the material.
As a result, I am now certain that the statement “proteins are held together by van der Waals forces rather than covalent bonds” is false. It’s false even if you put hydrogen bonding in the “Van der Waals forces” category, which would be misleading in this context. Nobody who knew about the actual structure of proteins would use the phrase “covalently bonded alternatives to biology”. The entire field of organic chemistry arises from carbon’s thirst for covalent bonds.
Weirdly, I’ve popped open my copy of “engines of creation”, which Yudkowsky cites for his claims, and I see nothing about Van der Waals forces or covalent bonds. In fact, Drexler spends quite a while praising the flexibility and versatility of proteins: he views custom designed protein machines as the first step on the road to nanotech. He merely notes that biological material are not as durable as metals and diamond. Of course, Drexler is not a biologist so even if he had said it, it would still have been worth fact-checking.
“Covalent bonds” were not the motivation for molecular nanotech. Rather, it was about atomically precise manufacturing: that you could avoid the wiggly randomness of biology and instead place molecules atom by atom mechanically using a system of nanoscale manipulators. The hope was that you could build a universal assembler that could build anything, and not be limited to biological materials or biological assembly techniques. This would include extremely strong structures such as diamond or pure metals, but could also be anything else that MNT proponents can dream of. If you wanted a new car, you’d just send it off to a universal assembler, which would build the entire thing from scratch in your garage.
Of course, it’s easy to dream up such things, and much harder to actually build it. As I described in a previous post, attempts to actually build such a thing have completely stalled at the first hurdle. It may just be unworkable in practicality.
Gullibility filters
I think a key danger of these sort of mistakes is that they act as a gullibility filter.
Anybody with a chemistry or biology background who hears someone confidently utter the phrase “covalently bonded equivalents to biology” will immediately have their bullshit alarm triggered, and will probably dismiss everything else you say as well. This also goes for anyone with enough skeptical instinct to google claims to ensure they have the bare minimum of scientific backing.
This isn’t an isolated incident either, by the way. See my write-up on the errors in the math in the quantum physics sequences, or the errors in economics described here.
So the people who know things, and the people who actually google things, will disproportionately come to the conclusion that EA is talking nonsense and not join, whereas the people who blindly accept any scientific sounding word they hear will just walk right in. I think this is not good for the epistemic health of a community.
Strong biology?
Moving on, what do I think of the broader points about biology being “weak”? Well it’s not like biology has slept on the idea of “make strong things”:
Have you heard of wood? Horns? claws, shells, bones? They all seem pretty strong to me. There are also materials with extremely high strength to weight ratios, like spider silk. Some of these are made with hard proteins such as keratin, some are made with mineralized tissue, or a combination the two.
This is not to say that a fully diamondoid based nanobot, (if such a thing is even possible) wouldn’t be stronger than these examples. Protein is not entirely covalent, which introduces weaknesses. When put into extreme conditions such as high heat, they may lose their secondary and tertiary structure and unfold back to their primary covalently bonded structure, in a process called “denaturation”. Diamond can avoid this fate as all of it’s bonds are equally strong.
My point here is merely to say that this is not a case of diamond vs “static cling”, but diamond vs wood or bone. Don't underestimate biological ingenuity!
Flexibility
My other point is this: Are we sure we even want our proteins to be strictly covalently bonded?
In my previous post, I covered the woes of trying to stick two carbon atoms to a carbon surface. The issue was this: you have to pick the atoms up and drop them in place. But the only way to pick them up, at that scale, was to covalently bond them to your transport tip, then covalently bond the atoms and the tip to the surface, and then forcibly rip the transport atom from the surface to leave .The analogy I used was that it was like trying to lay bricks when your gloves are coated in superglue.
In this scenario, the fact that covalent bonds are strong is the problem. It would actually be fantastic if you could use weak forces here: you lightly stick the atoms to your transport tip, drop it into the covalent bond spot, and then easily pull the transport tip away.
Biology pretty much does this. For example, enzymes like the one pictured below hold molecules in place through noncovalent bonding, facilitate a chemical reaction between them, and then release them.
As part of the process, parts of the enzyme actually shift and mold their structures around the incoming molecules in order to better catalyse reactions. I’m not sure how easily you could replicate this using stiff strictly covalent structures.
Part of the reason that proteins are so flexible and versatile is exactly the fact that they aren’t fully welded together by covalent bonds, and thus can utilise multiple strengths of force as necessary for their purposes.
Back to the drawing board?
Imagine there was a boss and inventor who knew nothing about biology, and set out to build molecular machines. After fifty years of hard work, he rushes in and we get the following exchange:
Inventor: “EUREKA! I’ve invented a fully functioning self-replicating molecular machine. We can use this elaborate structure called DNA to encode strings of amino acids, and they will automatically fold into a virtually unlimited variety of 3 dimensional structures that can do a huge variety of microscale tasks, including replicating themselves using these incredibly complicated structures called “cells”. The possibilities are endless!”
Boss: “eh, I dunno, this design seems like weak shit. It’s not as tough as diamond and only like 90% of the bonds are covalent. I reckon you should abandon this design entirely and go back to the drawing board, it can’t be that hard to build something better”.
He may be right that something better is possible. But that doesn’t mean that something better can be built quickly. The DNA/RNA model has been refined with over 3 billion years of practical trial and error experimentation on a planetary scale. Evolution may be dumb and slow, but it’s also got a ridiculous head start.
I do not expect evolution to have created the best molecular machinery it is possible to ever make, given infinite time and resources. Evolution provides constraints on how changes can occur, so if you remove these constraints, you can, almost by definition, do better. But it seems to me like the smart thing to do, if you want to build effective self replicating nanomachines quickly, is to piggyback off the natural experimentation of biology, and make even cooler and better things within the DNA framework, rather than starting from scratch and trying to reinvent the wheel. I think this is true for humans, and I think it’s also true for non-godlike AI’s.
I am very excited to see what can be accomplished in biology in the years to come. We are now in the age of Alphafold 2 and CRISPR, and I expect at least some incredibly cool stuff to come out of that. In my previous post, I highlighted DNA robots and DNA origami as impressive advances that I will keep an eye on, as we learn it’s advantages and limitations .
I think there is still a lot of room to explore what biology tells us about the potential capabilities and dangers of molecular machines, biological or otherwise. I would be very interested to hear from any biologists who have expertise in these areas.
Conclusion:
I believe that the claims about protein bonding repeatedly made by Yudkowsky are factually incorrect. Proteins are primarily held together by covalent bonds, and secondarily by hydrogen bonds, with Van der Waals forces sometimes but not always contributing. It is true that molecular machines based on diamondoid or metal could be physically stronger that biology, but attempts to build such machines have completely stalled. Biology can still build very strong materials such as wood and bone, and the use of non-covalent bonds can actually be very useful for the flexibility and versatility of proteins such as enzymes.
Edit: a breakdown of the claims I have a problem with
I thought that the problems with the original quotes were self-evident, but it may be worth breaking down each quote in turn. The following are ranked from roughly most to least wrong.
human technology figuring out how to use covalent bonds and metallic bonds, where biology sticks to ionic bonds and proteins held together by van der Waals forces
This is just straight up, explicitly false. Biology does not "stick to ionic bonds and proteins". As I pointed out, biology is made up of covalent bonds at it's very core, and uses them all the time.
covalently bonded equivalents of biology where instead of proteins that fold together and are held together by static cling, you have things that go down much sharper potential energy gradients and are bundled together
The phrase "covalently bonded equivalents to biology" implicitly states that biology is not covalently bonded. This is false.
I have mostly shifted to trying to talk about “covalently bonded” bacteria
The context of this claim is that Yudkowsky is trying to come up with a new name for deadly Drexler-style nanomachines. He has chosen "covalently bonded bacteria", implying that "covalently bonded bacteria" and normal bacteria are different things. Except that's not true, because bacteria is completely full of covalent bonds.
Algae are tiny microns-wide solar-powered fully self-replicating factories that run on general assemblers, "ribosomes", that can replicate most other products of biology given digital instructions.
Okay, I just saw this one, but ribosomes are not "general" assemblers, and they cannot replicate "most other products of biology". They do literally one thing, and that is read instructions and link together amino acids to form proteins.
For the next two, let's establish the principle that if you say "X is held together by Y instead of Z", you are implicitly making the statement that "X is not held together by Z", or perhaps that "Z is irrevelant compared to Y when talking about how X is held together". Otherwise you would not have used the word instead of. Would you utter the phrase "animal bodies are held together by flesh instead of skeletons?"
proteins bound together by weak van der Waals forces instead of strong covalent bonds
This implicitly makes the statement "proteins are not held together by strong covalent bonds", which is false. Or it could be saying "strong covalent bonds are irrelevant compared to van der waals forces when talking about how proteins are held together", which is also false.
even though the proteins are held together by van der Waals forces rather than covalent bonds, which is why algae are far less tough than diamond
"rather than" means the same thing as "instead of", and therefore makes an implicitly false statement for the reason I said in the last quote.
proteins are held together by van der Waals forces that are much weaker than covalent bonds
Actually, this one is defensible, because it didn't use the phrase "instead of". I would still prefer more qualifying terms such "the weakest link". If this had been the only statement, I would not have written this post.
It is fairly easy to fix most of these statements. As an example, you could say something like "the 3d structure of protein contains weak links like hydrophic bonds that are easy to break apart, whereas Drexler style tech could be made from 100% densely packed strictly covalent bonds"
I appreciate the difficulty of science communication and the need for simplification, but I believe that if it is easy to avoid saying false things, you should do so.
Final edit:
An organic chemist going by "skillissuer" reached out, saying they broadly agree with the post, but had nitpicks. I thought it was interesting enough to just include what they wrote directly here:
I think that chemistry 101 classification of bonds is a tad useless here. Instead, you can go from first principles: there are things that happen when atomic orbitals overlap (covalent bonds, metallic and such), there are interactions that are mostly electrostatic in nature (ionic, dipole-dipole, quadrupole-quadrupole - important biologically as pi-stacking, also ion-quadrupole etc) and there are things that are a result of exchange interaction (van der Waals and steric repulsion). Hydrogen bonds would be a mix of dipole-dipole and van der Waals interaction. You don’t have to transfer electrons in order to have ionic interaction, most of the time in biologically relevant situations it’s proton transfer, or charges just were there previously. Hydrophobic interactions are almost entirely a solvent effect and aren’t a bond strictly speaking
In water, i’m pretty sure that proteins are mostly held by hydrogen bonds and hydrophobic interactions. EY is correct in that some proteins hold shape by mostly noncovalent interactions, but these are mostly hydrogen, ionic, hydrophobic interactions and the proteins that actually provide mechanical strength run in continuous covalent strands through entire length of them anyway (collagen, keratin). I don’t think that counting bonds and saying that something is 90% bound covalently is a meaningful metric, because long series of hydrogen bonds or even vdW forces (in things like UHMWPE fibers) can be stronger than single covalently bound strand, ie if you tried to pull out a single strand of kevlar or collagen from bulk material, above certain length you won’t pull it apart, you’d just break it because collective energy of hydrogen bonds will be greater than single covalent bond holding it together, that’s why these fibers are strong in the first place
There is another kind of flexibility that you haven’t mentioned: proteins are made out of single covalently bound strand, yes, but these aren’t straight C-C chains. Making and especially breaking C-C bonds in controlled way is hard, proteins can be just hydrolyzed at amide bonds. If protein breaks in some way, and in real world everything breaks, it can be recycled into aminoacids (+ any cofactors etc) and then put back in a pretty straightforward way; you can’t do this with diamondoids, when it breaks, it breaks hard, and you’re done unless you’re picking everything apart atom by atom which would be much harder and more energy intensive. As it happens you can buy bulk adamantane, but it’s just made in conditions where C-C bonds are weak (high temperature) and it’s preferentially formed because it’s most stable thermodynamically among its isomers (that are starting materials). Conversely, if you use weaker bonds, you can make pieces conform to some template, or to each other without breaking everything at once - this is basis of dynamic combinatorial chemistry. There’s also entire field of self-healing materials that is based almost entirely on these either noncovalent or reversible covalent bonds
As part of the process, parts of the enzyme actually shift and mold their structures around the incoming molecules in order to better catalyse reactions. I’m not sure how easily you could replicate this using stiff strictly covalent structures.
You actually don’t have to do that, and there are some small organocatalysts that are entirely covalently bonded and do the same job. However you can’t make them from from aminoacids, these don’t have secondary structure (too small) and are generally less active. The bare minimum is to provide a receptor for transition state, and you can make it work without drastic changes in conformation. You could make your catalyst as stiff as you like, and it’ll even make activity higher - but only if none of these stiff parts interfere with binding of substrates, and your options are limited. It’s often better to leave some wiggle room. Short peptides aren’t really stiff enough in ways that matter there and instead it’s secondary and tertiary structure that puts important bits in the right place