Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
Radical Numerics' new model Omni outperforms previous genomic language models on variant effect prediction, especially in non-coding regions, by applying alignment and post-training techniques borrowed from natural language AI. The company argues that the same models enabling generative design of DNA sequences are also best suited for biosafety detection, and that current sequence-matching tools are insufficient against AI-obfuscated pathogens. The key finding is that unsupervised pre-training on raw DNA, combined with structured mid-training, yields state-of-the-art results across multiple genomic tasks without requiring fine-tuning for each task.
Key points
Omni achieves state-of-the-art variant effect prediction across coding and non-coding genomic regions.
Alignment and post-training techniques from NLP were applied to make Omni useful out-of-the-box for scientists.
Radical Numerics operates with a dual mandate to advance both generative DNA design and biosafety defense.
Chain-of-thought prompting improved RNA aptamer design scores using progressively better example sequences.
Current biosafety checks rely on sequence matching that misses functionally equivalent but sequence-diverse threats.
Tools mentioned
Techniques
- long context modeling with convolutional architectures
- unsupervised pre-training on raw DNA sequences
- alignment and post-training for task-specific prompting
- chain-of-thought prompting for sequence design
- variant effect prediction using likelihood ratios of wild-type vs mutation
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
the design side's going to get more capable. The defensive side needs to try its best to get ahead. So I think inherently there is this arms race style dynamic that the defensive side has been
far far lagging. And so what we want to do is bring the defensive side [music] to par. Essentially, we felt it was important as a lab that a team that was both building the design capabilities is
actually also best suited for building the defense capabilities cuz they're basically the same models. A model that is good at generating turns out is also very good at discriminating or
predicting if a sequence is pathogenic or not. For us, we as a company thought it was very important to have a dual mandate. It's this idea of essentially being cognizant and and feeling
responsible for the capabilities that we're enabling on the design side. So if we're going to create models that can design [music] function into sequences, we believe and see a gap in companies
being able [music] to safeguard that technology. Welcome to Len Space. I'm Brandon. I build RNA the Therapeutics at Atomic AI. I'm joined by my co-host R.J. Honiki, uh
CTO and co-founder of Mirror OMIX. Today it's a pleasure to have with us Eric Gwyn at um CEO and co-founder of Radical Numeric. Eric uh started got his PhD in Chris Ray's group. Uh he spent a lot of
time thinking about how to do long context genomic models before uh long context or genomic models were cool. He was the you know first author and I think you know basically visionary
behind the EVO generative model one of the first generative genomics platforms uh developed EVO 2 which you know naturally led into um you know radical numericics. Uh yeah thank you for being
here. Did I miss anything? >> That sounds great. >> Yeah, >> welcome. Thank you. So, Eric, let's talk about uh Omni and the blog post that you
guys did about the benchmarking, but I want to hear first, okay, what is a genetic language model? Why do I care? What does it do? And then let's talk about the topline results from the the
blog post. So a genome language model or GLM is a large language model trained on DNA sequences. So very much like natural language and chatbots you see but not train on words or natural language but
on the raw fabric of life which is these sequence of letters that make up DNA. And we ourselves our our company and our team is known for creating the first uh generative uh genomics models which are
models trained on DNA not just to read but also write meaning able to generate new sequences of DNA. And we felt this was a an area that was overlooked and that if AI could read and write DNA, it
could change a lot, you know, scientific discovery and understanding of human health and how to treat it. Um, and so we felt there was a big opportunity to train AI on on the genome.
>> What kind of things can you potentially do with with a model like this? >> Great to start for for us. When we first started working on DNA models, we worked on this model called hyena DNA, which is
a large language model, but it used a convolution instead of attention. So, a little more technical details. Um, DNA has this property that well, it's very long, right? Um, at the time, these
large language models had limited constraints on uh context, right? Being able to fit long sequences. And so we were looking for a more efficient algorithm to be able to handle something
like DNA. And so we came up with this what we call the hyena operator use convolutions. Long story short, it let us process longer sequences. In this case up to a million and at the time was
the largest context for a language model. And what you what we did with it was essentially used it to read DNA predict function. So given a sequence of DNA string of characters, um we predict
its regulatory function. its effect on a genome. And you know this is interesting to scientists because a lot of the DNA in our bodies um you know perhaps people are less aware but actually we don't
know a lot that much about our genome. It's we know it obviously it encodes the information for making us us and how you know all the different complexities and potential diseases at the same time the
grammar sort of grammar rules about how the combination of those letters are sort of formed what they encode and how they encode function and traits is not fully understood and so the hope was
using these DNA models language models to be able to map some function from the raw DNA sequence and so our first generation models was able to show that yes, we can um train AI to be able to
read and understand to some degree um DNA sequences and especially um what we call the longer range interactions, meaning over sequences, you know, if you use chat bots for example, like if
you've heard the the the phrase a context rot, you know, the longer the input you put into a language model, um it starts to deteriorate. And so um being able to pick up long range
information and sort of patterns motifs, grammar over long sequences was what we were trying to accomplish. Um, and so we showcased in that first generation of hyena DNA that that was possible over a
million context. And then really what started the field now known as generative genomics was the model called EVO. And EVO we tried to showcase there was this idea of not just reading DNA
but being able to generate it. And so um we wanted to be accelerate essentially how biologists and scientists had learned from biology and in particular genomics. Um and we felt like this whole
field of of generative AI being applied to language great accelerated the you know obviously our understanding and and ability to manipulate the natural language. Um but here's this other
language DNA the genome that we don't understand and it's barely being applied with AI. in our at the time a few years ago models that we saw were really small short context so they could only pick up
small patterns and limited context and none of them generated DNA so they all just would read and we felt that the idea of generation was so powerful and transformative in natural language what
if we could bring that to biology and and DNA particular what can you accomplish that you can't do in a lab right so what does that unlock for you if you could do that very well
>> I think one of The um first things that we showcased that got folks sort of intrigued by the potential of this was um a crisper cast system. So it's an enzyme that's able to cut DNA itself.
And I think what was particularly enabled by the EVO models was the ability to generate over not just one modality or one type of sequence um but spanning multiple modalities and
spanning multiple scales. So um crisper cast it's it's a molecule made up of both RNA and uh D and proteins. And so at the time you hadn't really seen models that can generate multiple
modalities. you can they had protein language models that can generate proteins. Sometimes you had RNA models that could um generate RNA but you didn't have a single system to sort of
co-design. We showcased that um a single DNA model sort of the the foundation of of both of those right from from DNA you can get RNA and proteins that we can design a single system to generate and
also function in the real world. So we asked EVO, we showcased it a bunch of natural um crisper cast systems and essentially asked it can you can you make a new one and we're able to sample
from that that model that we trained and indeed we showcased that um Eva was able to discover a new crisper cast system and you know folks were were intrigued by it and cover of science magazine and
later get to give a TED talk about the work which is interesting because obviously the audience is very general and so trying to make you know what is crisper cast system what is DNA and how
do you generate it? Why why would you generate it? All sorts of fun topics. Um and I think the more even intriguing exciting thing that scientists eventually um just last year showcased
what you can do with a generative DNA model was to generate the first genome from scratch using AI. So um this is something not possible by humans, right? Humans usually you can think of it like
copy and paste parts of other genomes or other DNA, put it into something else. Um but they would just take out small motifs that you know they know the function um they understand the rules
but to build something from scratch in the ground up um had not been done before at the genome level and so um EVO turns out was able to generate a functional genome and in this case it
was what's known as a bacteria phase also known as a virus um and this this was a key turning point for I think the scientific community and for us as a company at radical numeric because we
felt this was such an indicative manifestation of the potential right to create a whole organism not existing in nature but also the potential harm that that means as well
right if you can control if you can manipulate the fabric of life itself control its function what kind of implications does that mean what what are you enabling into the world and so
we actually got a lot of feedback a lot of comments a lot of outreach from folks both excited and concerned about this capability this kind of capability and and just the trajectory, right? This
is the early stages, the first thing one can sort of project and imagine what this could lead to. And so we felt as a company important to it was important to not only push on the biological design
capabilities of these models, but also the ability to use them as defensive tools um for the potential of u misuse and and biological risk that emerges. Um and I think indeed a lot of companies a
lot of frontier labs are also being concerned about this emerging risk of AI models being capable of designing biological sequences and um you know at the same time it's being mostly attacked
from like a natural language standpoint like safeguards and things like claude you know if you talk about viruses it'll just like shut you down um which is great I think it's to some degree you
need you know safeguards at the natural language level but I think what you also need clearly ly is it safeguards um at the biological sequence level too. So you need models that not just can
understand language and you know the trajectory of of your chat but to understand the substrate itself is the next step and ultimate limit right if you can have models can understand you
know if the sequence is pathogenic or a virus that's the level of of of defensive capabilities that you want and then you know being able to push that out into surveillance systems um
national security of you know being able to monitor you know emerging sequences in the environment. This is what that kind of capability makes possible. And then we think bringing this frontier
technology to to that community as well is also important just as important um as using this for human health which is what we primarily focus on. >> There was this evolution that you
mentioned there's the hy hyena DNA and then there's EVO EVO 2 and now uh Omni. Can you just talk about a little bit about EVO and EVO 2 struggled to beat sort of specialized models across many
tasks whereas in the blog post you talk about how across a wide range of tasks that OMI is actually able to outperform them now. So can we just talk a little bit about that? So yeah, EVO was um
intriguing to folks in many ways. It showcased the the potential for applying to multiple um types of modalities, but it's still it still in many ways underperformed sort of the specialist
DNA models, especially on human genetics or genomics. And so although EVO was competitive, it it still wasn't state-of-the-art or or kind of pushing the needle. And so, you know, some parts
of the community thought like why use a a giant LLM when I can use these these smaller more specialized models. And so, what what we wanted to do with Omni was amongst many things, but one of the
first things was to showcase this idea of mid-training and post- training or or broadly alignment. So, the way we think about language models in bio in bio and Jonos in particular is that mostly
you've only seen base models trained. So they're pre-trained, but they're basically unaligned. So in, you know, in the analogous space for natural language, it's like you're doing all the
pre-training, but to make it actually useful in the real world and answer questions that users actually want and that is in the form factor they they find actually uh informative, there's a
bunch of alignment and post- training and mid-training done to get the models to, you know, be production ready and actually actually useful. And so we we felt EVO was just showcasing the
potential of that pre-training but Omni is a step of actually making it useful for folks like scientists right and so uh we spend a lot of time on alignment and and mid and post training which is
essentially showcasing tasks in the form that people generally would want to understand about genomics. you know, given a wild type and a and a mutation sequence, you know, help help me find
the causal variant, right? These types of questions and form factors for how you might want to analyze genomics um doesn't just emerge necessarily easily on its own from pre-training.
Pre-training is, you know, this next token predictions task or or infilling. And um basically I think of it as like the raw uh pattern making ability that you're you're teaching it is in the
pre-training but then taking those learned embeddings or features um and pointing at specific tasks or you know a bunch of tasks really and aligning it meaning have it show you the output in a
way that is meaningful to you takes a little bit of teasing and manipulating that so far like the front literature labs are the ones that drive that research. um in the natural language
community. And so we wanted to bring a lot of that research and more um to genomics. And so I think Omni is a is really just a preview to showcase that potential, right? And I think once you
do that even just a little bit, you know, we were surprised that it did start being um state-of-the-art and and pushing the boundaries, not just being affected of multiple tasks um just
broadly like EVO was, but actually pushing the the frontier of each of those areas of variant effects, prediction, um causal mutations for disease, it could it could start
actually being useful for human genomics. And so we're really excited to share that with folks. And it was just a preview in that sense because we were still actively training and
incorporating additional techniques into the model like additional modalities. But I think we were basically too excited and we wanted to get this in the hands of folks faster. And um you know
some of the feedback we got in in early interest. Um lots of hospital systems um nonprofits that have tons of genetic data and for example they know there's some kind of condition or symptom for
the patient but they can't figure out which parts of the DNA are causing it. So these got these V uses or variants of unknown significance that um we're extremely excited to apply these models
to and actually help diagnose a lot of these patients um is one example of a real use application. >> I'd like to talk more about the applications in a bit but I am curious
just from a technical standpoint like what does it look like and what does it mean to align a genomics model? I can imagine with large language model there's a sort of natural chain of
thought you know RL there's like a clear paradigm here. I'm curious what does it look like for a genomic model which is not maybe better describe as something like fine-tuning in a different
language. >> Yeah. >> Yeah. I mean I I think in in many ways one could describe it as fine-tuning but then introducing I'd say sort of the key
components are the right structure of the inputs. So feeding them in a certain sequence so that the model is aware that a certain task is being asked of it. Um so there's a mix of special tokens to
basically you can think of as like if you're going to do disease prediction for you know for disease A expect this special token right. Um just kind of like a way to prompt it. If you're
expected to do um design have another special token and then showcase the examples kind of like in a chain of thought manner. meaning showcase a sequence of desired outputs and the
trajectory of it. Um this is a little vague sort of intentionally cuz it's part of our part of our our secret sauce that we're still developing and I think over time we want to showcase more and
more of it but um in many ways it does mimic a lot of the natural language community. It's a lot of it is fine-tuning. Um, but really it's also carving out specific data sets that you
want it to focus on and then structuring the questions or tasks in specific ways as opposed to pre-training is really pre-training is really just feeding it in and um everything in really and just
doing next token prediction or mask infilling if you're doing mass language modeling and it has no sense of like this Q&A type structure where you have a question you know a prompt and then an
output and you can think of mid post training as starting to showcase given this type of input I expect this type of output whether it's score a prediction score or a design is largely the mid and
post training then post training also includes things like reinforcement learning too but I think the bigger steps are you know introducing structure of like question and answering do the
models have multiple heads that you are task specific or are you training one set of heads or whatever whatever that can answer multiple questions at the same time. Just change the input tokens
or whatever. >> Yeah, broadly it's a little you can I think we're flexible on this. The idea um yeah, sometimes you need you can use different heads but the idea for us is
to unify more so um and so I think early experiments we did have different heads but in some cases the different heads do better. in some cases the univi you know a single head a single model um does
better and so I think we're flexible on that but the I think broadly the direction that we are moving toward um is a single and the reason for a special motivation for that um is that we're
trying to unlock a lot of modality a lot of generalization and I think the more unifying um we're able to make these models I think that's when you see a more emerging capabilities um happened
and in in large part that's what motivated the the DNA work. We felt like a lot of models were specialized into other modalities a little more downstream from DNA. like RNA or
proteins or molecules. We largely felt DNA is the foundation and that from DNA you can learn a good deal of other modalities um ult potentially all of them and I think of other modalities as
sort of like additional context um that you're showing the model that's how I kind of view it uh philosophically in my in my head but yeah the the idea that single models unifying across modalities
scales is what the lab builds and builds toward Can you walk through a few of the tasks that that you talk about in the blog and and just explain and remembering to to
you know narrate for the listener only audience um but talk about some of these topline results and we can maybe dig into them a little bit. >> Sure. One of the areas that we thought
was really interesting for showcasing in this particular release for Omni was on understanding varants and their effects which are essentially in DNA a change in a position you know changing the letter
of one of the other three letters in your genome in in DNA sometimes can cause a disease and sometimes it doesn't do anything actually many times it doesn't do anything but there are
specific specific areas in your genome that if you have a different variant or a different letter there, it can cause disease. And in in many cases, uh because the combinations of of these
changes in your in the genome over three billion letters, right, is so vast that for clinicians and and scientists, we actually know only a very small portion of which variants are causal to disease
or not. And so there's these benchmarks from um folks who collect you know variants uh different hospital systems and and clinics um and some are known and some are are are still you know
unknown. Folks have created some benchmarks from ClinVar or Trait Gym to basically the idea is given a mutation in a DNA can you tell if it's going to cause a disease or not? Right? And so
this is a very good setup for DNA models because um they're probabilistic. And so when you do make a change, it basically can modify its confidence or the probability of predicting a next letter.
Uh and in this case, we've sort of leveraged that predictability of these models. Um they've essentially seen and been trained on so much DNA and particularly human DNA. they kind of
understand what's common in CIS and usually common or conserved across different other folks usually typically means healthier. And so if it's less common, you can think of it this way. If
it's less common, the model can pick it up and and and sort of predict that it's potentially pathogenic or or disease-causing. And so we we've taken some of these um benchmarks and when you
introduce a variant or a mutation, there's different types of mutations in in variants. So sometimes you can delete uh a letter altogether. You can just flip it. Um you can remove big portions
of the DNA. Um but these are generally single variants. Um and in this case this is where um previous DNA models really struggled especially on humans. Um and so we we showcased that not only
is it um you know competitive or capable for humans but in many cases it's state-of-the-art and actually most of these cases it's state-of-the-art. Um and I think the exciting part is that
areas where other models that were you know currently on the frontier they still were lagging behind quite a bit in terms of wear in the genome. So in the genome there's coding and non-coding
regions like protein areas uh protein regions meaning areas that are actually the coding the structure of a protein versus other areas that do other things like regulate what genes are expressed.
>> Yeah. And so um these non-coding regions are largely regulatory. They kind of control how much or when to use a certain gene or turn them on. Um and in many cases these non-coding regions,
these regulatory regions variants there mutations there um are much harder to predict if they cause disease or not. And I think what's exciting about the new generation of models we're building
with Omni is that that's where we shine. especially the models are able to pick up mutations and be able to distinguish if it's disease-causing in these non-coding and and especially long range
areas. Um, so I think this is particularly exciting to a lot of geneticists that have struggled to use traditional bionirmatic tools or statistical methods because they've
they've largely focused on coding regions which is only about you know one and a half 2% of the genome. And um turns out many if not most of the diseases are in these non-coding regions
and so there's been a real desire to build models that can actually pick up these variants of um of disease causing variants in these non regions. I have several questions. So this first um just
while we're here you know for the for the listeners um there's this column on this benchmark chart called uh Bzulli reference number four in the blog post. I think for maybe some historical
context could you talk about what this column represents and you know maybe this also helped give context for you know the omni column of the on the right.
>> Yeah [clears throat] great point. So what we show in this benchmark here is really taking some of the representative models or the strongest models um in the deep learning side and also in the
traditional methods. So we have EVO 2 is the latest previous genomic model that our team had worked on. And then Borzo is it's also a DNA model but a very different kind. Essentially it's a
supervised model that predicts from DNA um functional genomic tracks. So it's it it it too is inherently multimodal but it's not it's not a language model. So it doesn't predict like a next token
prediction. It goes from a DNA sequence directly to a functional genomic track like chromatic flexibility or gene expression. Um >> and these tracks have been annotated
extensively by people writing their dissertations and whatever. >> Yeah. Yeah. Right. So the big difference there is that it's a supervised task, right? So it requires labeled outputs
and in in our case these language models they do not require labeled outputs, right? So you're you're doing raw pre-training on unanotated sequences. So you know why is that desirable? Well,
there's a lot more data, a lot more genomic data that's not annotated. Actually, most of it pretty much in many ways all of it almost all of it is not annotated. And so being able to learn
from an unsupervised manner hugely desirable, right? Um for uh us we wanted to showcase the benefit of pre-training on on raw genomic sequences and it you know comparing it to state-of-the-art uh
models in other in other spaces in DNA. And so your prediction is when you say it's unsupervised, how does the unsupervised property work like like how do you convert the output of whatever
your model is to an actionable like ranker score whatever what I talked about before is uh you know to train the omni model it's pre-trained so it's unsupervised but when you're pointing it
at a specific task there there is a there is a supervised step. So it's just you know taking a smaller data set that is labeled but essentially what we're doing is using the likelihood scores. So
the raw outputs of the language model which basically you can think of it like a probability for predicting what the next letter is. We can essentially showcase the probability score the
likelihood score for um the mutation versus the wild type. So that seen in the reference genome versus the mutation in this particular case. And then we'll we'll have two scores. And then you can
think of it as like using a u a ratio of the two to showcase basically how different are you from how different is this mutation from um from a normal or baseline basically. And then that's you
can think of it as like a surprise factor um that the model is able to use and leverage and then we can use that to essentially um score an actual prediction for disease or not. Does
>> that make sense? >> Yeah. So, uh, omni auto reggressive is that or is it diffusion or something else that you can't tell me? [laughter] Uh,
>> yeah, we yeah, this time we we we're not describing the exact makeup, but um, >> EVO was an autogressive and I think it was the first large scale autogressive. >> And so I think
>> for us, we don't tie ourselves down to a specific training objective. We use every tool in the box toolbox essentially, you know. Okay, but for this specific benchmark, you're going
along and you're just using the the likelihood distribution of, you know, the tokens and some tokens are, you know, the model thinks these are unlikely and that is probably because
some evolutionary constraint like this doesn't show up often and because it doesn't show often across genomes, it is probably going to cause problems and you know people will not survive so on. And
so you think that is a basically your your ranking metric or something? Yeah, it's it's one interpretation of >> of what you know how the model's thinking about it and you know very
similar to in in natural language you can describe the same kind of paradigm >> and there is additional case in mid and post training to >> leverage more than that I suppose cuz we
can teach it specific structure and and benchmarks >> so that it can build on top of of what you just described which is like what's common in nature but also because it's a
specific task for disease um variant prediction then the model has additional >> um training introduced during mid training to showcase into to add additional learning power essentially
>> what are some examples of that again probably secret sauce to some extent but can you give just a gist of what that looks like what are the kinds of things you would throw in there
>> we we would actually throw in the score to itself so um you know that like like that I mentioned um a ratio that's a ratio of wild type versus mutation I that I would say that's more of a a
zero shot method where you don't even have to do any mid training and that's what EVO 2 is doing in particular in this column. So EVO 2 is it's not fine-tuned essentially. There's a EV
column which people are basically fine-tuning you know using them scores from EVO 2. um that's from Goodfire and that's also you know folks that we we greatly expect and and they kind of
showcase that these models are are able to be state-of-the-art when you can when you can fine-tune them as well not just zero shot and then our model is introducing sort of a step above that
not just fine-tuning but also introducing structure into by structure I mean the format of these benchmarks into the model itself that that Q&A style formatting during mid-training
which is what gives us an extra boost I see an extra boost, but I think the other benefit too um that you know we didn't emphasize too much in the blog, but I think it's really convenient for
practical use for scientists is that you don't you're doing this without taking the embeddings and then tapping on ahead and then you know doing some quick regression which is an extra step. It's
extra hurdle. Can you imagine catch PT if like every time you ask a question you had to like fine-tune it for a certain domain. Um, we've done it so that the model is flexible during mid
training to be trained on many tasks at once. And so the that fine-tuning that that last step of training the embeddings doesn't have to be done. It's out of the box at that point. You just
prompt in a certain format and it will um you know that certain format tells it which task you're going to do and then we'll output the answers in that in the desired format. Basically,
>> how careful were you in designing this post- training scheme to avoid kind of data leakage with the ClinVar Tra and so on like these data sets like how like how confident are you that there is no
data leakage either like accidental or something upstream and that >> you know >> because I would not be surprised if a lot of these sequences showed up also in
your training data even in a you know unsupervised sort of >> yeah The short answer is we're extremely cognizant of of the risk of data leakage and in extremely
hard to to not mislead or you know be careful >> and so we yeah we have bioinformmentians that are able to basically comb through the data and curate and ddup and align
sequences to make sure that things that are similar potentially to what's in the benchmark are not there and if they are there we we remove it and so yeah so we we actually have um steps to to um QC
the data quite extensively. >> Yeah. Cool. Yeah. Um I guess maybe before we move on, I I think it it's really cool seeing that there are these supervised methods which previously
several of these numbers were let's say within the error bars if not just straight up beating what came before them. And >> now you have significantly improved upon
that. >> Yeah, we're super excited. And I think for the longest time there's this area or this other method called CAD which has been state-of-the-art and
state-of-the-art for a reason which is what they sort of by design they'll take the best methods and kind of do it ensemble right so they'll take they'll take another even if the best method is
another previous model um they'll mix it with like an SVM and just like throw the kitchen sink at it and so you can see why it would be the best right and so that was the bar for us we're like
>> if they're going to throw the kitchen sink at it like we're not going to you cherry pick one model and say we're better than that. We need to beat what's possible, humanly possible now, like
across everything. And so, um, yeah, our our our researchers were were setting their their sights on that [laughter] to uh to see if they can actually improve performance across every method. I'd be
interested to see there there's a some discussion of chain of thought and that that broke my brain a little bit when I was first [laughter] >> first hearing about that. I'd be really
interested to hear about what that even means. >> Yeah. >> Um I I had to pour through the blog post to really understand that.
>> Yeah. Yeah. I think this is a really just a a taste of where we think um the design capabilities can move toward and and way more usable for folks. Um so chain of thought um stems from natural
language community. I I believe Jason Weey at OpenAI um showcased the first examples and and really what the breakthrough there was um showcasing that these language models performed
better when you just show your work essentially or showed the steps of how you came to a conclusion or an argument and and turns out even if the they were like simple steps but it just gave the
model a chance you know maybe it's sort of like philosophic who who knows exactly why it works but um essentially feeding more tokens in and giving it more scratch space to think. And so
people just think this is the um sort of the the beginning of reasoning u for these language models, this ability to kind of um get to an answer by thinking to itself by itself. And so it seemed
quite successful in language, very successful. And that's why you have um a lot of agents that just spend just tons of tokens, right? Just [laughter] show showing its work, right? um in in
many ways it stems from this chain of thought paradigm in in biology. We saw very little of that. We saw we started to see some of that in um protein design a little a little um and so we wanted to
push that and and showcase that you know well first explore is that possible in DNA like what does that even mean in genomics because you don't really have words that describe you know its
thinking um so how do how do you take that par that same paradigm and um introduce it to a DNA language model and so what we did was a simpler version in many ways
And we had this data set of RNA aptimer. So just think of it as these desired sequences with some kind of fitness score associated with them. So we took this large data set that had RNA input
and a fitness score. The fitness score go high is good, right? The simplest version. It's a big data set. And so what we wanted to showcase was it if we show the model progressively
better RNAs in a in a series of steps with its score, right? So you have like low scores first and then you gradually move up the chain. Can the model continue that trajectory on its own? And
then you know in the final step does it selfoptimize to a point where it's like the best score it can get? That was the experiment. Can can we do that? And so we took a data set a large data set of
aftermer we held out a portion of like the best performing ones and we show showed it only the the lower ones but then we ranked it right. So we showcased lower scores with the RNA apt and then
progressively got higher and then asked the model to just like continue with that pattern and it turns out it was able to recapitulate some of those higher scores that we had not shown it
yet. Um and we're actually in the process of validating the wet lab right now. So we we didn't get to show it here but we we wanted to know right actually can it not just do this in silicone
which it can. It showcased that it was able to continue this trajectory and create design plausible apt with higher fitness scores. And now uh we think this is a you know obviously if this works in
the lab we think this is a hugely hugely valuable paradigm that can be that can be pretty much applied to every other type of electrical sequence. That's what usually what you have. You have sequence
you have some kind of fitness score or desired output. And if we can get models to eventually learn that structure and you know basically just show a series of progressively stronger sequences, the
model can then predict the rest. That's a very powerful paradigm. >> Honestly a bit surprised about this um that specifically this task saw strong improvement. Maybe my personal bias is
coming in here, but RNA is uh somewhat notorious for not having good co-evolution data in terms of like constraining structures, right? for you know viral genomes um often times there
is strong evolutionary pressure but for mamalian usually RNA does not have evolutionary pressure and I think the community has um seen that very very clearly the predominant
>> pressure is like RNA will code um you know will will carry information coding information yeah >> so I I I mean I'm wondering like genomes carry lots of different information.
They code for proteins. They have regulatory elements and you know different types of genomes have different types of structure. So I I'm wondering where do you think the this
capability might have emerged in this language model? That's a good question and and honestly I'm not sure like we we're surprised too. one because the model is pre-trained on on DNA and it's
really just like mid-trained um on RNA very very you know in in a small way. Yeah, I think what your your intuition about the DNA having a lot of evolutionary effects or or information
is probably where and so I think this is hinting at the idea of why we think it's so important to pre-train on on DNA genomes and genomes first um and then sort of add additional modalities on top
>> because you get a lot of transfer and you want and the modality generalization is the thing that we're working toward. You know what I didn't talk about is a company for the company as well is this
idea of building toward general biological intelligence where we are unifying a lot of the different socalled like languages or modalities of biology. At the end of the day they stem from DNA
and I think people have not exploited that fact as much. it's usually really specialized domain specific model or mod modality specific models and and not leveraging a lot of the inherent shared
structure from other modalities and so one example of that is like you know when people talk about virtual cells for this a little bit tangent but um you know they they tend to focus on just RNA
and you know you know transcripts and and they want to generalize to describing an entire cell but a obviously a cell is a lot more things than that in my mind if you want to
learn a system, you want to learn from all the signals or sensors of that world or that system. Um, you know, if it's a cell, um, you want to fuse, you want to understand the DNA, you want to
understand the metabolomics, the epigenomics, the proteomics, and that's when you get closer to like quote unquote a virtual cell. And in my our minds, we don't even want to stop at
just the cell, but we want to we want to fuse all of these sensors across all of biology. Will it get us to a a super intelligence that understands every component of minutia biology? Who knows?
But I am confident that this type of paradigm will get us a hell of a lot further than we are now. Like that's like that's my bar is like can you make something far more useful than than now?
>> Right. [clears throat] >> So I I'm curious. Are you focusing on ukarotic cells? Are you focusing on like human genomes? Do have you gone so far as to do viral genomes? I mean there's a
lot of DNA viruses but it seems like plausible that RNA that there's a lot of RNA sequence virus sequences out there and I'm not sure fundamentally they would be much different in terms of
training. >> Um I'm you know I'm curious like what's the scope of that if you can talk about it. >> Yeah absolutely. We are interested in
all domains of life. So here we focused on humans in particular because we thought um this was an area that of of previous models EVO and EVO 2 were not as strong and sort of got a lot of um
feedback from folks asking you know what are these models useful useful for uh they can't understand human genomics because it's too complex of grammar and rules and DNA too it's too noisy it's
too there's too many repeat characters and all that stuff so we we wanted to showcase we think this is actually useful and it can be applied to humans and it's sort of the most complex of the
complex in some ways but I think there's opportunity to apply these models generally to to every form of of life. Um so we absolutely are interested in procariats and and and viral. Um viral
in particular we care about especially for bio defense and biocurity especially and um I think there are also lots of therapeutic applications that we can learn from microbial life of you know
maybe obviously for some folks uh in particular they mentioned that folks had use to generate uh the first AI genome a bacteria phase turns out you can use bacteria phasages potentially for AMR or
antimicrobial resistance you know if you have a superbug bacteria infection in the around the world there's about 2 million deaths from bacteria infections still the idea of using uh viruses
designed viruses to target specific bacteria has been done for a long time particularly in Eastern Europe and um there's a potential to make a new class of antimicrobials that are not like
antibiotics but very similar that can be used just like it um and so I think We're gravitating toward things that are high impact and the potential to save lives. Um, and so we we don't stop at
just one type of genome. I think we're interested in anything that's beneficial as you humans. >> Maybe going back to my question about um solex and RNA um predicting kind of a
chain of thought of RNA evolution. I I'm curious, was this model trained on RNA sequences or sequences which have some um might have evolutionary pressure on RNA structure? only during mid training
only so pre-training is all just genomes in in DNA >> and so the only time we introduced RNA was for this specific task where and only RNA from this data set so not even
outside >> I see it was so this really was something along the there is something encoding RNA structure in this model to some degree maybe
>> or either that or implicitly maybe >> yeah implicitly I would implicitly. Um I mean cuz >> that's cool. >> Yeah. Cuz I I'd say you know sequences
as we know from proteins like >> implicitly should learn structure from from the just sequence. Um >> yeah and so we don't we don't add in 2D or 3D information at this point but we
absolutely plan to. [laughter] >> Yeah. So, so um just so I understand first of all the chain of thought ideas this is a demonstration of it but the idea is that anything that you can get
sort of a a training set that has a sequentially better um measurement of some sort is maybe a a candidate for this technique. Yeah. >> And so can you just describe for this
particular experiment just so we can understand how we're mapping chain of thought to this data set and what the data sets have to look like. Can you just describe [clears throat] how is the
for this I know this this wasn't your data set but um how was the data collected in such a way that you could map accurately from sort of fitness or whatever to a particular phase of or
part of the data set. Yeah. So I so I'm less familiar with how the data was um actually sent us or generated from the experimental point viewpoint but they are validated from a wet lab in the real
world when it was collected. So it has some kind of fitness score uh I believe through >> I I can maybe provide a bit of context on this if you want. I mean, so the idea
here is you just generate a bunch of random sequences and then you take those sequences and um so you have like an optimer structure which is essentially like a switch with RNA which sort of
when something binds to it, it will do something like cleave um off a sequence and you can use an NGS like next generation sequencing readout very high throughput. So you create lots of these
different sequences. You I guess in this case they were targeting a a specific HIV um like protein or uh genome or something and if it binds you can you basically get the signal of like you get
more reads of that. And so the the more time the more sequences which are floating around kind of like the more fitness the more likely it is to bind. And then you take those and then you
mutate them again and you kind of iterate on this >> right? So you have this iterative experiment where you're progressively using the petri dish to basically
identify the most fit >> um things. So and then what you're doing here is you're basically doing this same experiment in silicone >> and they're actually doing very high
throughput like I think there's like 10 to the 11 or something sequence some like really high number of kind of sequences explored in parallel for this experiment.
>> But the key is is the biology of the experiment is actually doing the filtering right. >> Yes. Yes. Yeah. Right. So, and so you can imagine other types of experiments
where you could apply the same kind of idea where you you're doing this like progressive refinement of something which is very common in biological lab work and and so if you're capturing
those intermediate states then you can maybe feed them into the model and do that kind of thing. >> Absolutely. Yeah. So another example is um for the antimicrobial resistance. I
think that's something we're we're very interested in as well. And the ability to selectively target specific bacteria strains or kill bacteria strains yet that's measured basically a score zero
to one uh of how effectively that is done. And so showing progressively more effective phase genomes um and their associated scores for effectiveness fits that
paradigm very well. There's another um design task that we're working with a national lab to to do this with as well in a area that is quite different for us, but it's on um designing proteins to
extract rare earth minerals. Um so [laughter] >> um and so it turns out that you you don't just care about um uh proteins that could bind to something, but you
want it to be selective. You want to be bind to one type of rare earth mineral. Um and so you have scores associated with how much affinity or you know binding affinity for each type of
mineral. We want to essentially do the similar thought this similar exercise with uh rare earth minerals and show it a series of progressively you know desirable scores not just for binding to
this but like you know lower binding scores for other ones you can selectively do it. So I think there's the the creativity in which you can showcase sequence and design desirable
desirable sequence with you know some kind of fitness or functional output is a relatively intuitive way for people to design you know just prompt by just by prompt engineering essentially which I
think is uh is very exciting to to to explore more. >> Following up on the uh rare earth mineral extraction I I find that is an interesting use case for this. In fact,
maybe one where you probably wouldn't have a comparative advantage compared to some other techniques because it seems like your strengths are probably in larger scale design across like entire
organisms, but focusing on individual proteins that may be much more of a like a structural task. And I'm curious like going to talking about ClinVar, the argument here is that the model now is a
really good statistical representation of what type of mutations are common or not common. >> Yeah. And I think in order to do design as a if you want to design structures I
think you want to understand structure. If you want to understand disease I think that is I think more natural for many diseases is much more natural in terms of like a population genomic sort
of way. So I I'm I'm curious like where do you think your strongest competitive advantage is? Um you know in using this strategy and do you think that your model understands structure in addition
to function? >> Yeah. >> I'd say what drew us to this particular application of rare earths and and our strengths in general why we thought it
might be suited for it uh is two things. One is I think in in in areas where context matters. So yeah, you're right. So a protein design task where you're just designing structure not maybe not
naturally where we see ourselves um you know competitively advantaged but uh in the rare earth's case our hypothesis is that context matters and and what it matters here possibly is uh certain
microorganisms with proteins that you know have the function that we desire. we can potentially prompt and provide us context for this is the neighborhood in which you know tell Omni this is the
neighborhood of genomes or microorganisms that you should search for new proteins. So it's really a like sort of like a mining exercise and what we can showcase to our models that other
folks can't for like you know like protein structure models is that we can feed in um non-coding regions before that gene of interest or that protein of interest
>> and then ask for the model to provide variance essentially. So like mutate this protein but you know know that you're in this microorganism but show me different variants that you've seen in
nature or combine different things that you've seen in nature given this context. Um and I think that is one reason why we can make very evolutionary diverse sequences and potentially
phases. Folks have used EVO models to do this with toxin antitoxins in a sim very similar uh technique. they've prompted on things upstream from the proteins of interest and then asked the model to
kind of generate a bunch of plausible other ones. Um, and I and I think this case for rare earths that's super exciting because then we can come up with plausible variants and then test
them relatively simply for what they bind to and selectively bind to, which I think is really interesting for this this um this partner who cares for example about securing the supply chain
of rare earths for the US strategically. And so we thought that's absolutely worth something that um is worth supporting. >> So your point is not just you're
designing a protein but you're designing an organism which generates a protein and this protein has an action but >> or or is it it's that that the way to design this is you need to understand
how hit through the tree of life interactions of proteins with rare earths have or with certain minerals have occurred. Yeah, I'd say in this particular case for like the rare
earths, we we're not really interested in the in the organism like designing the whole organism, but I think the organism does tell us about what proteins are plausible and their
selectivity is not >> or the way they evolve, not just around the protein itself, but also the regulatory elements around it. Um, and it helps us narrow and provide
additional context. I guess I think of the genome or DNA broadly as sort of the imprint of the physical world into DNA. And so there's parts of the DNA that we want, but the things around those parts
that we want also tell us a little bit about the context of where it came from and how it came to be in its function. And so I just think of it, you know, like going back to that word, but
context. I think context matters um in many of these applications or at least in some of them. Yeah, I think context matters a lot in biology. Um, I think maybe one of our big bottlenecks is the
lack of context and how >> we as humans mostly approach biology in terms of a very engineering like let's isolate individual systems and a systems biology approach is very hard to get any
sort of meaningful quantitative predictive power. So maybe going off on a tangent, but I I mean I'm curious, you know, going to context and thinking about how context scales to an organism.
You're talking about right now 2 million length context, right? >> Yeah. >> You know, I think the human genome is roughly, you know, a thousand times
longer. [laughter] Um so uh or but even like a lot of, you know, say bacterial genomes if you're trying to engineer them are are quite a bit longer than that. So how do you leverage something
which has a long but still finite context compared in a to to do synthetic biology across like large organisms? >> Maybe actually maybe for context how did the Evo 2 bacteria phage design work
which was probably much more than 2 million for that as well. Yeah. So I could I can speak to the Evo bacteria phase just a little bit because it's actually a separate group that worked on
that, but um that context was actually pretty short actually. The reason why they started with phases is because it's amongst the the shortest genomes and so I believe it was
>> something around 6,000 base pairs. >> That's really Yeah. Yeah. [laughter] So extremely short are insane. They're incredibly efficient. >> They impact a lot in there.
>> Yeah. >> Yeah. Um but yeah, no I think your question about how how do you get longer context with something smaller um is a very key question that we I think
broadly the AI community is constantly trying to >> fix and be creative about and so I mean I think that's large part why we as a company are an AI research lab first
because we we think the innovation needs to be constantly pushed. It's not a space where we can just grab open source models and and expect that, you know, many of the tasks that we care about are
just going to be solved. Uh we we want to continually push the envelope. And so context is one of the key research areas that we we drive. And I would say that's probably how we got our name cuz we
worked on long context before it was a was it was a thing I guess like 2023 2022. >> Long context. you and your team, I mean your your collaborators have a long
history of these spa space state space models doing you know pushing context um like what seemed insane at the time. Yeah. >> I mean now I think routine of books the
big slabs but >> but at the time was like orders of magnitude longer. I um I don't know if you want to talk about that a bit and also I think one maybe one thing which I
think is really fascinating is how >> in for both this model and other ones you really go on a first principle way like diving into the architectures of how GPUs work and designing models which
are you know exploiting the architecture of GPU in addition to I mean it's not just oh we're we're building longer it's like what can we do special given the compute constraints we enough to push
the boundary. >> Yeah, absolutely. If you want to distract our researchers at Radical Nix, [laughter] uh this is this is how you nerd snipe
them. You talk about you bring up long context and kernels, GPUs, then they're like, "Wait, did someone say kernels?" [laughter] And then and they start trying to to figure out, you know, how
to make things fast. Yeah. Long context special place in my heart cuz that's what I focus on in my PhD at Stanford. And that's how we started thinking about DNA. And you know going back a little
bit for fun we were looking working on language models in general and then we noticed our models were good at long context and so that's the progression of like how we started working in the space
was like oh these models seem to be really efficient on long context um and then with Michael Py my lab mate he worked on the first design of Pyina this convolutional architecture and then we
started thinking like let's push this further let's see what new new applications open up if we really lean into long context and we asked what's the longest sequence out there and and
eventually unsurprisingly we we we landed on DNA we're like DNA's got to be the longest three billion base pairs and you know um and um we started thinking like okay what's being done there like
what kind of context lanes are people doing there were doing super short they're doing like one or two thousand base pairs or tokens at a time it's like a you know way smaller than what one
would want for DNA. >> I mean, most human transcripts are like 3K or so. So, that's not even like as you know, most that's not even what you need to represent like a protein or most
of the time. >> And so, it's clearly a need there and overlooked and we saw it as a a way to initially like let's see if we can do something that we had no idea if it was
going to work and and no idea who would want it. And so, we just started tinkering around and it turns out out of the box, relatively out of the box, it was it was doing pretty well at reading
DNA. And then it led to okay let's people keep asking about like DNA. They didn't really care about our language stuff as much and they would say like can you can you do longer? Can you know
what can you do with it? They always ask what can you do with it? We didn't know really. Um and then for some reason left on this idea of like well no one's writing DNA. Can we can we get it to
write DNA? And that was a really simple question but and in hindsight almost seems obvious that yeah you would want to write DNA and design it. But when I was first pitching the idea of EVO to
folks, I spent six months, which I guess in bio words is not that long, but I spent six months going around saying like, "Hey, if I generate DNA, like would you find that useful? Like what
would you do with it? Um would you back us? Like would you want to be a part of this?" And crazy enough, most almost every scientist at Stanford I talked to thought it was a stupid idea. It was I
was like, "This would be so cool." like generating DNA. Um, how how much would we accelerate the field? And then people would say like what would you do with it? I'm like, I don't know. And and then
I would get that comment or comments like that's not possible. Like we as humans don't understand the rules. How could you expect an AI to learn it? You can't even tell if it's right or wrong.
Like you can't tell the AI, yes, that's right or wrong. How can you expect it to learn it? Or there's too many repeat characters or DNA is too noisy of a distribution. like there there's no real
rules and there's just a bunch of junk in there. I heard all the reasons and I was just so stubborn about it. There's got to be a use case from from being able to generate DNA. I just it just
feels right and I didn't know what it was. So it was it really was an experiment of like what happens. And so when we first trained EVO, I remember um we had no idea if it was going to work.
We we we had no idea. We didn't know what thing was going to emerge. We just thought let's just train a big one. which is kind of ridiculous. But somehow, you know, folks at Arc, they
were like, "Sure. >> Yeah, why not? >> Why not? Let's see what happens. Let's pay let's pay some money uh for the GPUs and and let these crazy kids uh train a
model." And um I remember when the first result came back and and it it kind of like gave us chills. We're like, "Oh, maybe there's something going on." uh which was somebody took the model
checkpoint the first one and threw it at uh protein gym one of the protein um benchmarks and turned out to be competitive with uh protein specific models and we were like okay that is
pretty surprising because we never told the model what's proteins versus not and there's actually it was you know there's also proteins and DNA right so that was one piece I think it was just surprising
that it was actually competitive with protein specific models um and this was like the first experience And then they just kind of kept on coming one after another like oh this
competitive here oh it's state of the art on RNA and DNA and we started seeing like oh it can learn across different modalities like not just DNA and then then we felt like okay this there's
something there and so it kind of went from there and then you know folks at NVIDIA were like let's let's back EVO 2 let's make this even bigger and then Greg Brockman from OpenAI is like I take
a break from OpenAI and take a four month sobatical and like help these crazy kids out and you know then we're Slack messaging Greg brought me at 3:00 a.m. trying to debug our code, which is
wild. Yeah. So, it it just it it went the trajectory was very surprising in many ways, but at the same time, we still got a lot of feedback like, you know, what are these models good for?
What are they, you know, what are they useful for in the in the real world? And so, that's what really motivated us to start a company. We thought what we showed was just really just a taste from
like an academic flavor. In the similar way the language models when they first two you know if natural language came out people asked similar questions like what are these things good for? Oh,
cool. It can write some jokes for me. Is this going to lead to like an all, you know, all this thing a AGI that can automate everything? They did not think that, right? They thought it's like a
toy. It's got emerging capabilities and they would extrapolate the potential. And in many ways, I think what we're seeing here uh is even more exciting or reminiscent of that trajectory for for
DNA. But is Evo 2 even was showing the bacteria phage and that that wasn't a persuasive I mean it's almost a scary example right as you mentioned so that people didn't see that like that isn't a
light bulb moment for people >> yeah so I mean and some people right I think um either fair or or you know rough critiques you can even you know play devil devil's advocate and say what
it's generating is kind of you know pretty close to nature and you're kind of recapitulating just kind of a small variance and so there's I think there's a lot of ways to critique and and and
kind of you know minimize the the potential which can be fair argument like so I think at this point we think what is more important is not just pushing on the science and like cool
this can be done and like kind of leave it there as a sort of thought experiment but like how do we actually use this to improve human understanding of disease improve um treatments, make better rare
earth mineral extractors. How do we actually make this useful is what we care about now as a company in addition to some of these more scientific questions.
>> So maybe that's a good segue. Um there's that brings up two things for me. One is the mechan stuff that's in the end of the blog post and then also the biosafety
stuff. Those are both I think important applications. So can we let's do mechan first. Can you talk a little bit about um and this is actually kind of building on the work that EVO did as well. I
think um in I remember in EVO paper there was some mechan work in which they were discussing using the mod reversing the question and using the model to extract insights about biology, right?
And so so and this has actually become a common theme with I think maybe biology more than any other domain is that people because it's a scientific question and it's learning patterns
about the world. You can actually say okay well what patterns did you learn? Yeah. So I think mechan is an emerging uh field in in bio that we're extremely excited about and admittedly we are on
the early side I'd say. Um so we're building up that team but so far what we've showcased and and been excited about is looking really analyzing the embeddings and some of the activations
in the model. Um and so this idea of mechan for for for bio is borrowing a lot from the natural language community currently. Um, you could think of what the models are doing is compressing a
bunch of information it's seen, right? And so in this compression, it's really basically distilling it down into the key components, key patterns that uh helps it understand the data or learns
the distribution of the data. And so what we're trying to do is probe the models to see what did the model distill into its weights and its activations or sort of like the outputs. And so for us,
we started off with a lot of the outputs of the model. So the activations and we we wanted to see what kind of structure what kind of visualizations can we see that help us understand um some of the
complexities of of DNA which was a ton right and so some of these complexities can can range around GC content certain motifs of transcription factors there's a bunch of regulatory types of patterns
and motifs that um the model we believe has to pick up to be able to do its tasks right to understand whether a disease is is caused by a variant and generally
it's going to compress all these different motifs and distill them into the model weights. And so our job is to then find these and see if we can distill a pattern or structure that can
be generalized to other cases where we don't understand the patterns. Right. So that's sort of laral. >> So in this case GC means the two nucleotides are um the fraction of those
in a sequence. >> Exactly. Yeah. So the fra the fraction of the G G and C letters in a genome which are I guess amongst the more simpler things but um also you know
simple things as repeats number of retif I think transcription factor or TF motifs is another um one that is especially interesting for folks because I eventually we're going to get to a
case where we can design transcription factors meaning we'll have certain transcription factor patterns via like chipse modalities >> transcription factors are are well you
go ahead >> transcription factors are uh molecules that can bind to DNA and they would alter essentially the gene expression pattern. Um and so it has this
regulatory effect that doesn't modify the DNA itself but can modify sort of the effects of DNA and the products that DNA makes. Um and it can have a lot of implications on well pretty much
everything in your body. So it can modify disease states. It can modify I guess there a lot of aging related research is around transcription factor um design. And so we think being able to
understand some of the motifs via uh DNA but also additional modalities will eventually let us be able to design transcription vector patterns as well. And so I think this is very exciting and
in many ways the combinatoral space of learning these transcription factors like what binds and where they bind and what effects it causes is just far too vast to be able to do this um in a
manual way. And so we want to take a datadriven approach to to learn some of these motifs. The complexity here is partly because the transcription factors are themselves
um coding genes and so that you can reg those can regulate each other and so that you have this that's where that combinatorial effect. >> Yeah. So like just narrating for the
listeners only we're kind of marching through these increasingly complex and higher level factors all the way from GC content. we started now we're looking at disease which is maybe the most complex
thing you could or you were uh pointing towards in this analysis. >> Yeah. So I think um what we our first idea for previewing this was and and what we're working toward this broadly
is um this idea of mapping the manifold of disease. Right? So manifold there's like the sort of representation space of what disease looks like to a model in terms of the output scores or
embeddings. We believe that there's a lot more structure that can be gleaned from understanding some of these outputs. Um and so you know mapping this manifold or the landscape of what the
structure of disease and obviously there's many diseases and so um I think would be a big exciting area of research for us um to to actually drive motivation for me in bio. I think I
think we're just really scratching the surface because if you can get this much of clean this much of insight potentially from just DNA which is in my mind just one of the sensors that you
want to ultimately fuse into you know modeling biology then being able to do something similar across all modalities is something like oh by all modalities I mean protein RNA
epigenomics you know the attack chromatin accessibility and and methylation patterns all these other different types of sort of molecular phenotypes around DNA just
presents such a huge opportunity that this kind is kind of laid in front of us that is all green space green field. No, I I haven't seen anybody do um this level of sophisticated techniques from
you know machine learning deep learning into what I think is going to be the most important impactful area of understanding and applications for AI. That's what we're excited about. And I
think we're just showing a preview, very simple preview from the DNA only models. Um, but in our next generation of models, which will be increasingly multimodal, we're talking dozens. It's a
very exciting moment for us. >> Sidebar on that. Is language natural language one of the most? >> Not yet. >> Not yet.
>> Yeah. But it will be. So I'd say there's a lot of questions about how to fuse that with with bio, but I actually think it's pretty will be relatively straightforward. I
think the more tricky the tricky part for us is actually how to fuse the um biological signals more. So fusing language we there's a lot of examples with that with like with image and and
video space. Um so we we feel pretty good about that and we've done some early experiments with language and I think that will make it extra accessible for folks when you can connect it to
language but I think um this the part that recipe that folks still are trying to figure out is how to do this across biological modalities. >> It just seems like a natural way to be
able to do chain of thought. Right. >> Exactly. Yeah. >> Absolutely. And I think um you know especially when you start having um chain of thoughts from orchestrating
tools things like cloud science I think the idea of incorporating language um is already a lot of people's minds if you don't have any questions uh I let's talk about the the
>> for us we as a company thought it was very important to have a dual mandate what we call a dual mandate it's this idea of essentially being cognizant and and feeling responsible or wanting to
feel responsible for the capabilities that we're enabling on the design side. So if we're going to create models that can design function into sequences, um we believe and see a gap in companies
being able to safeguard that technology and make sure that it's used responsibly. um increasingly more I mean folks have I was just at a panel last night u a panel on AI scientists agents
that can do scientific discovery and and one of the last questions was what's what are some of the biggest risks or doomsday scenarios with AI can learn learn learning about science and every
one of them talked about um biological weapons and and at the same time I'm cur I was curious cuz I was like okay so then what are any of these folks doing about that and basically I didn't hear
anything about that and I you know these companies I won't say their names but and I look at the companies they don't have big efforts in those spaces. So anyways we felt it was important as a
lab in the lab that a team that was both building the design capabilities is actually best also best suited for building the defense capabilities cuz they're basically the same models. A
model that is good at generating turns out is also very good at discriminating or predicting with uh you know if a sequence is pathogenic or not. So we we felt it it was not just you know from a
principal standpoint necessary to to work on both biocurity and design but that it was you know strategically it just made sense um as well. And so we felt this resonated with our team but
also the broader community um folks in in in the US government and abroad even um that this there was a a clear gap in and need for a type of entity to exist in to do this. And so yeah we we felt it
was necessary to build it into our mission. And so for us what we have done and plan to do when you look at it or I should say bio defense biocurity sort of has three or four different pillars of
you know strategy for bio defense. The first one is around detection and surveillance right you know broadly can you detect from the environment if uh a sequence has a pathogen in it something
that can cause disease let's say from you know swabs at a airport a nasal swab or um a sewage system right collecting samples the next is attribution which is once you detect a danger can you figure
out where it came from is it natural is it from you know a random country abroad is it engineered by a human from a specific lab in a country and that helps you figure out you know what what to do
about it right so this next idea is around counter measures so once you've detected figure out where it's from what do you do about it can you make a counter agent can you make it antiviral
or antimicrobial and then fourth one's generally around um deterrence but that's more of like a a government kind of level thing but yeah so we focus primarily on the first
three so we create tools that can given a sequence detect if it's pathogenic but also So what we felt was missing from the community was not just detect you know broadly if it's pathogenic but
characterize the heck out of it meaning uh what parts of the sequence are dangerous what genes u for example attributing where it came from being able to not just look at its you know
sequence and match it to a database but be able to attribute its signatures and then also a big component to make this effective in the first place the community the the bio defense community
broadly focuses on sequence matching. So they'll take a sequence and they'll basically align it to a known database and see say have I seen this before? Does it match you know this list
of known pathogens? But I think what's emerging and a concern for a lot of labs is well one new stuff right if it's not on your list and two things that were intentionally obiscated to not be
detected in sequence space meaning the letters matching up exactly but also function space. >> Yeah. >> Right. Because basically models that
we're enabling now they will be able to be function aware or structure aware. And so that means for concretely you can have a sequence that has the same function like a pathogen but actually
look different in terms of the letters. And you can imagine based actually what what we have on the screen here still is the smack interp thing and there's this sort of manifold that that that the
model constructs internally that that is kind of coding for these among other things functions. And so that you can imagine how it would be able to say, "Oh, well that's, you know, maybe
genetically quite different or at least somewhat different, but still has a similar function." >> Exactly. Exactly. So what we talk about in our um defense blog is this idea of
things that can function similarly, they can start having separation in terms of um what the sequence looks like while maintaining the same functionality. Right? So this is can happen in nature
sort of um naturally but also what these AI models allow you to do is also intentionally do that as well. So have the same function functional capability but have diverse letters basically
diverse spelling but describe the same thing basically. And so in this case this work from Microsoft called paraphrases. So it's like you know kind of rewarding things where they showcase
that you can for example use protein language models to essentially keep the same structure which structure implies similar function but then change the spelling right and not just that they
wanted to test that if you have this capability and you send this through existing detection systems would it break the system like would it actually detect it or not and um I think one of
the interesting facts that maybe the general general public, but most biologists know is that there's these DNA synthesis companies, right, where you can basically send a design of
sequences and get back a DNA, you know, DNA molecule like it's an Amazon package. Like you just send it off and they'll send you the physical DNA of the sign. Um, they'll manufacture it for
you. And this is this runs the scientific community, right? It's the the pipeline that allows people to do research and understand biology and make drugs and everything. So it's it's
prevalent and and it's it's public. And so I think a cons one of the concerns for folks is well and one of the concerns for us when we first started working on this was in a world for
example where agents are prolific online presumably billions and trillions of agents building and taking all sorts of actions on the internet. It's wild that they don't have any tools to basically
tell it if it's making anything dangerous or not. And so that was that was literally our first motivation of like we should probably make something that can detect if something is
dangerous or not. Right. >> So you guys filter in the if for these manufacturers that they could say what am I making here? Is this dangerous or whatever? Yeah. And maybe you need a you
you could have an exception if you're like some licensed lab or something and I'm doing something dangerous. I know I'm doing it. Please let me do it anyway. Or something.
>> All sorts of cases. So that's that's one scenario. To be fair, the many of the DNA synthesis companies have detection tools. But I would I would uh strongly hypothesize that they're not AI based
and and fairly they're probably not robust. Yeah, they're all pattern matching mostly. >> Sorry, maybe maybe this I should ask this question later. I I'm just curious
about you know, we've all everyone who is working in bio has tried to use Fable and instantly got picked out on literally everything you type in >> my website for example.
>> Yeah. Yeah. [laughter] you you you know you can't do anything in Fable without um but in terms of people like scientists exploring synthetic biology and creating new sequences and designing
new sequences it seems like it it'd be very hard to get you know to to have a an ROC curve which you can you can live on that doesn't like impede novel scientific research which will
legitimate purposes how do you avoid you know even with an F1 of point you 9 which you're not even close to right now. I think you know that that still could easily if there are billions of
sequences you know sequence generated a day or at least like a year. I mean I I think you could really have a lot of it seems like a very hard balance to >> yeah it's a tough question right so I
think broadly the way we look at it is you know if we thought about like how do we 100% stop the dangerous design I think it's a it's it's a harder question to ask I think the question we asked is
on the capabilities design side there's plenty of folks pushing the frontier of that when we look at the defense side do we see frontier technology being applied there and the answer to that was no
right so we we we saw this huge gap and we wanted to sort of aid come to its defense I said come to its um its aid to give it a boost right so I think that perspective it's it's easy choice for us
to say let's push let's bring that to a better you know head-to-head match against the design side is it going to solve everything well I think that's what we're going to aspire to be but but
u realistically there's always going to be cases where it can get get around, right? And I think that's one that's a big motivation for why we think, you know, safety for language models and
chat bots is one one layer, right? But also, you're right, there may be things that kind of just get out there, get past that anyways. And so, what do you do about those cases is it's already
passed, you know, the chat bots, right? It's already it's already aided someone into making dangerous sequences, so it's out there. Um, I think the cool thing about our tools is that we what we're
building is tools for the folks that care about things that that's already out there out there in the environment that's made it somewhere and now there's this whole ecosystem that we want to
build into that does the surveillance that does the attribution that does the countermeasures we want to boost that community right and build stronger tools for that space. Does it have to be
perfect to be useful there? I don't think so. I think we can we can be helpful and and move the biod defense community forward in terms of you know bringing AI technology and AI frontier
technology to their aid without being perfect and so that's kind of [clears throat] >> maybe another say so threat model that it's not even clear what threat model
you're actually trying to defend against at this point but the capabilities don't currently exist at all. So you know you provide something and now this can be worked on with uh in terms of a larger
regulatory framework or um government or you know nonprofit whatever larger framework and now provides you a tool that the community can build upon. >> Um even if it's not perfect it's it
provides a starting point and if you don't have a tool then you can't do anything. >> Yeah the bar is the improvement right. Right. So the bar is like where you at
now? Can we move the needle? >> Can we move beyond sequence-based matching, alignment, match matching? Absolutely. I think there's tons and I think um the interest is is only
growing. I think we're hearing a lot of chatter at the at the regulatory side, you know, from different politicians and different folks in in think tanks. It does seem like folks are mobilizing. So
we're optimistic that this gets out and is more in top of mind for folks to actually take action as opposed to just talking about it. cuz like right now it does feel like there's a lot of talk and
one of the reasons why we felt like let's build tools and like put it out and give access to people cuz there's been a lot of talk about AI companies saying like we should do bio defense but
then like okay what does that mean like what are you going to do about it and so that was our our approach >> so I can just steal man this for a minute though can can you go to the
other diagram where you have the yeah and if you click on them then they show other similar similar compounds or similar um genomes. So arguably just steelmanning the the opposing
viewpoint if you are on the frontier which it seems like you are and you certainly strive to be you're moving the frontier of the attack and the defense at the same time. So like right now that
what you're doing clearly helps but that that maybe if you're on the frontier then that doesn't really matter for in the long term. So how do you think about that? I mean, I guess you just have to
nerf what people are doing or something. >> Yeah. The mindset of what was hoping to get folks to start thinking about it as less of a, oh, let's make a bio defense tool and like call it a day. It's
basically an arms race, right? It it similar to the cyber security community. You're going to make better technology, especially with something like Fable that can potentially attack. And that
means you're going to have this back and forth. the design side's going to get more capable. The defensive side needs to tries to get ahead, right? And then that then that just motivates other
folks to do past that too, right? On the design side. So I think inherently there is this arms race style dynamic that um the way it appears to us is that the defensive side has been far far lagging.
And so what we want to do is bring the defensive side closer to to par essentially. So that that's the mindset I I picture or the framing I think about it.
>> The cyber security uh analogy is interesting but I think it differs in some like key points. So first of all I think that with cyber security with a sufficiently strong model you might
actually be able to close all loopholes which are not sociological. There are certain ones which will always be hard always be ways of getting around things but you could in principle catch every
single exploit and I think that might be possible in the future. and then you can patch them and as long as people say updated you know you're secure we have fixed genomes right so you can't patch a
human I mean uh so once something so I think that in some sense the the you know attack surfaces and or the the way that you defend against that is much higher or much harder but then maybe the
the converse is that the uh it seems much less likely that someone would have the incentive to go on offense to the same degree and also the the like barrier to entry to success even if you
can print out arbitrary DNA and the the process of going from that to making a successful you know virus especially one which doesn't kill the person designing it is actually quite large. So I I guess
I'm maybe I I'm curious like what is the single from your opinion what is like the single biggest threat that we actually have? >> What would the thing which keeps you
asleep or keeps you awake at night? Is there something in particular or is this like you just think this is something we need to build and let's build it? It >> one it's hard to get in the mindset of a
person wanting to design a bioweapon. >> Yeah. Um, so we're we're not trying to um necessarily think of all the potential people or scenarios that you know, you know, the bad actors might um
work on. What I think broadly what I worry about is we're lowering the bar for how much expertise is needed and the speed at which folks can engineer these kind of things. And that means the
volume is going to just exponentially increase at some point. And so there's a mix of yes, there's intentional worries because there's certainly state actors that have had biological war uh
programs, very very very large ones. [laughter] And and one of our adviserss on a company has, you know, physically seen these facilities and and de decommission them. And so we've heard a
lot of stories about um states actually being motivated to to create such weapons. Um that is one concern. And I think during peace time it's less scary. during wartime it's particularly scary.
I think the unintentional ones are also things that um in my mind potentially more likely in the near term where folks do try to generate things and control function and inadvertently things that
um maybe maybe they tried to make a a certain thing to understand and to study but it got out right because these things can be hard to contaminate uh to to to contain for example and there's a
leakage. So, I think those scenarios potentially seem the most likely. Does it keep me up at night? Not necessarily. I'm I'm much more of an overall an optimist. I think building this type of
technology is ultimately a game that you weigh out the pros and cons and the cost and benefits. I think the benefits far outweigh the potential harm. And so, that's why we we work on it because
ultimately we do think it's going to be an engine for for discovery and human health improvement. But at the same time, we just felt like the the defensive side was sort of losing its
this this arms race. And so we work on it as well and try to push the frontier. But I I'd say overall, I think as a community of researchers, but also folks in the, you know, the policy side, I
think the the community is resilient enough and and has the ability to to mobilize to get ahead of it. So I'm very optimistic about it. We have two questions that we we like to ask um
every guest. So the first one is if you could by fiat remove a bottleneck that is important to you, what would that be? Interesting. Um could I get two answers on this one? Sure.
I think the first one is um kind of a copout because every AI lab says this like GPU GPUs. >> GPUs. [laughter] We've done that answer a few times. we we can use GPUs. Um the
other one which I think is a little more philosophical I think in this space and what we're trying to do aspire to do is is reinvent how scientists do their work in the
space and I think one hurdle we run into is folks who and you see this in many domains folks who are have the more expertise you have in something the more pessimistic you become about that space
and I think you especially see this in bio where you know a disease area or modality so well and then someone introduces something else new and you're
like oh but what about this and this and that and they're very pessimistic and probably rightfully so but also I think what I've noticed at the company what we strive to do and the folks that we try
to bring in are domain experts that do know that field but also are still dreamers meaning they still do have that imagination and desire to change how things are done. I think that barrier we
see that a lot in the field. I think if we embrace that more we can see a lot more progress and step change that I would love to to see. >> Brings us to the last question which is
uh >> yeah is there something that you want the audience to take away a single message? Yeah, I think um one message is I think folks have had especially into
the AI research community felt like there was a choice they had to make sometimes to either work on the frontier of AI technology and that was like consumer related apps or enterprise
related apps and like they just had to work on chatbots and that's that's that's the cutting edge technology but also doing you know I think when people think about the true potential
what AI can do and I think a lot of it is about improving human health understanding our biology but people have felt like they have had to choose and like if I work in that space I can't
work in the frontier of AI and what I would like people to take away is that you don't have to choose I think we can work on things you truly care about that you think will push humanity and work on
cutting edge technology and that's what we're trying to build at radical numeric >> yeah and I mean clearly and I encourage people to read the blog posts the architecture, the mechan, there's a lot
of innovation that is going into building these models and I firmly believe that biology is really on the forefront of AI. >> Awesome. Yeah, glad to feel that way.
Amazing. [sighs] >> Thanks for chatting. >> Yeah, thank you for taking making the long trip like [laughter] blocks away.
>> Anytime. >> Appreciate the invite. I had a blast. >> Great. Thank you. >> [music]