How We Got LLMs to Recommend Our Open Source Library — Christopher Burns, Inth

0:01 [music] >> The talk title, we'll see if it lines up by the end of it, but when we put this talk title in just to be honest with you, so much

0:21 changes in like 3 days at this point. We'll see how it goes. So, yeah, the whole point of it was that how I got LLMs to understand my open source library and what I did to do it

0:36 well. Is it some kind of scientific background? Am I from a lab? No. That's My slidey clicky thing's not working. So, I just like to say again, I'm just

0:48 like you. I'm just this side of the stage. I've just hacking hacking it together, figuring out what is useful, what is token efficient, these kind of things. And again, I am British. Please

1:02 don't think my accent makes me an expert. So, for quick contacts, I'm Christopher Burns. I'm the founder of Inth. I created a open source cookie banner

1:14 library called C15T. That really annoying thing on the internet. That is me. Uh I spoke at Next Conf uh after it started taking off and it had 1.2

1:26 thousand downloads at the time. Now it's closer to 2 million. In terms of like statistics, so we just checked that, you know, this is not theoretical. This is actual

1:37 uh something that is succeeding. We have 3 million NPM downloads. 4.5 uh 4.5 45% month-on-month growth. 2.8 thousand websites using it in production from Minify to Z to

1:52 Inphysical. And the whole concept of this talk was uh it goes back to we were doing all these things to make our library more

2:03 efficient. You know, we were batting upwards compared to every other tool. Every other tool was built for marketers and lawyers. We were built for the

2:12 developer. So, we had to make sure we had a very good developer experience. And we had an onboarding format said, "How did you hear about this?" And we started to get spikes that from April

2:25 13th, you know, now it is our number one source of inbound is Claude, ChatGPT, Codex, that is ChatGPT, Gemini recommending us. And I like to think of this as, you

2:42 know, the iceberg. You know, we start with the top of C15T and there's many many tools that go into it from, you know, LLMs.txt to site maps to

2:54 RSS feeds to robot.txts. So many micro optimizations that you can do from old methods of running the internet to new methods. And uh

3:05 how many of you have, you know, made these kind of tools? How many of you have, really put simply, said, "Hey, agents, we need to this to be done." And yeah, it said, "We should install this

3:23 library." And you've gone, "Okay." Raise your hands. How many people have done this? Pretty much most people. That's a lot of hands.

3:34 So, what's really funny is that we went from wizards installing our software to agents installing them. And I just went through Y Combinator. And what's really interesting is it if

3:48 you know who these two people are. These are the co-founders of Stripe, the Collison brothers, and they had a really classic saying of like a Collison brothers install.

3:59 And they would hand you their laptop, and they would install Stripe. These days, it's kind of like just a prompt. Being in Y Combinator, we just give people a prompt. And really what that

4:11 means is that our very good developer experience primitives are now hitting agent primitives. So, as we was pulling all these things

4:23 together, there is no one tool that fixes everything. I like to think about these problems like, you know, Batman's utility belt. Loads of really small things targeted in different areas to

4:36 get it done. And we built all of these things into C15T because we wanted C15T to be the best developer framework in this tool. Think of it like Stripe Docs. And as we

4:50 was building more and more tools, more and more documentation websites, we actually started abstracting these tools into a side quest that we call Lead Type. So, all of the

5:03 things that we're going to talk about now are things that we have already solved with this open source framework. We have our friends at other developer companies implementing it and seeing

5:14 similar results about how to like optimize for the agent experience. So, again, this isn't a magic SEO tool. It's actually a very non-sexy title, but a

5:27 framework-neutral docs pipeline. Complex. But really, all it basically does is take your .mdx files, you run Lead Type generate, and it will spit out

5:38 everything for um optimized agent experience for your websites. And the rest of this talk is going to look a bit like a BuzzFeed list, to put

5:50 simply, of these problems, because again, not everybody knows even how to put an LLM.txt on their website. So, you know, that comes to the first problem of if

6:02 your docs have hundreds of pages, and how can it navigate them to find the right questions? The first solution is obviously an LLM.txt.

6:14 What we found in our research is that it's much better not to just generate this. It is much better to write your LLM.txt from hand. Obviously, AutoRaptor, but write it as you are

6:28 trying to get the answers across to the LLMs. For about 40 good lines beats 1,000 lines of noise from our testing. And that comes to the second issue of

6:42 agents don't know how to browse. They know how to fetch. So, you then need the second part of the solution of the LLMs full. Again, think of this as a sitemap, where

6:54 it takes the actual page and the links and a short description of what each page is for the LLMs to reference. Again, most people have heard these two solutions.

7:07 But, where things are starting to get very complicated and we're seeing a lot of optimizations right now, is that HTML is expensive, and why can't we just ship markdown to the agents?

7:20 And we can. And you've seen that everybody has started creating twin MDs. So, that's taking the normal website, such as Next.js quick start, and then having a dot MD on the end of it.

7:37 And when you load that, it goes to the markdown But, what's really important here, and it's really worth noting, is this line at the bottom. If you look at all the

7:49 best documentation websites, Minify, Vercel, C15T, pat myself on the back. Um they all have this in the header. This is saying to the agents whenever they visit the website that there is an

8:04 alternative version of this in mark markdown. Again, who actually supports it? Don't ask me. Perplexity, some of the agents, it's all up in the air.

8:17 And then, the second thing as well is that taking the .mds, you need to make sure that they're available through multiple methods. So, one of them is like the .md, so as you

8:31 like copy it to an agent, you say .md. Another one is just taking the normal um link and then adding a uh redirect into your like your Next.js config, so that if it detects an agent has the

8:47 header of accepting markdown, instead of returning the HTML, it will return the markdown. And then, the third one is that not all agents can append header tags. So, there's also a URL query of

9:03 mode equals agent. So, they're the ones that pretty much everybody knows. Um and it's pretty basic internet knowledge at this point. Um

9:15 but one of the really interesting ones is where we're going next. And our tooling is also helping this is that an agent can't ask your website anything. So, we need to think about the web MCP.

9:31 And this is still very early, but our tool is already uh um exposing three different tools to WebMCP. Search docs, get pages, and ask docs. Again, um our library lead type is

9:48 pulling all of that context together so an agent can easily ask it the right questions. I think we'll even see a future where communication happens over email and there's companies in San

10:00 Francisco building that today. But this is actually the most interesting one and I think the most important one that anybody who has any type of developer module surface, NPM

10:15 modules, cargo, Python, whatever. Is that the uncomfortable truth is that coding agents are actually never visiting the website if you have a library.

10:27 They're actually visiting the node modules. They read the repo and they read the node modules. They They have previous stale training data and they're trying to work it out on

10:41 what it can do from the the compiled source. So, again, following what people like Vercel are doing and people who are thought leaders in this industry is that

10:54 we take the bundled markdown documents and then we also put them in the node modules with an agents.md file.

11:07 And the agents.md file basically says, "If you've got a problem, if you've got a question, all the documents are here. Grab them." And we actually see that this has

11:19 surprisingly real effects. We can see that between many different models almost 50% token saving on instead of trying to search the web, find the right tools, pulling the markdown files from

11:34 your code base. So, if you have a library that's forever changing, then having the node modules built in is a very effective solution. This is also working without any skills, but if you

11:48 want as well, you can add skills to it to say, "Look at the node modules and go from that." And again, just uh doubling down into this point, looking at like the

11:59 agents.ai and DFile, you can say like when working with Z15 T Next.js library, read the bundles and verify that they match and go from there.

12:11 So, that's really like how we've done it. I don't want to say this is like prescriptive, that I know the answers. If you have documentation websites, or

12:23 if you have any type of markdown, if you're running your own blog, you know, I've been using our package as well on our marketing website. Every part of our marketing website also has a markdown

12:34 file. It can be something that's used for many things. We're currently just um most people are just using it for documentation. But, you can literally run it and it

12:44 will pull out all of these extra files. And one of the big things was when I put this talk together, you know, we were seeing the results

12:54 that Claude was recommending, but there was not really any like test suites yet, or test harnesses, on like is your site agent ready? And Cloudflare brought one of them out.

13:07 But, my favorite is actually one called Aura AI. Um this is brand new and it tests a lot. I'm happy to show off score of 59 because it's constantly changing. 3

13:22 weeks ago, it was a lot higher. And again, this is a forever changing area. So, aura.ai, put in your website, and it will start giving you recommendations. It's forever changing. Again, we can

13:35 just stay on top of it. And yeah, this is like one of my final slides is that the slide the the market agents, LLMs, everything is forever changing. There is no such thing

13:48 as perfection. When I started making these slides, I got so caught up of like, everyone expects me to be the expert here, but I've just been hacking on this problem a little more than you

13:58 guys have so far. So, never get caught with being perfect. Every small little increase really does matter. Every small little thing you add really does matter. Thank you so much. You can find me on X,

14:14 Burn Chris, and LinkedIn, and everywhere. >> [applause] >> WOO! I THINK

14:21 I THINK WE HAVE TIME for one or two questions. Yeah, of course. >> So, if you were building um uh we're a website agency. We work with a lot of startups building like their

14:38 own websites. >> Mhm. >> If you were just building a website, not necessarily like developer tool, but just a website to be found, which of

14:45 these methods like would you concentrate on if you're starting from scratch? >> Yeah, I think the most important ones, and we're starting to see this more and more, is

14:57 trying to provide a dot md file for every single page. A lot of CMSs are not built in this way. Um and we see this optimization happening more and more where I didn't put in the slide, but

15:10 we're seeing more and more websites being visited by agents instead of real humans. So, in terms of even like trying to be proactive and token efficient, you should provide a markdown file if you

15:24 can. Again, a lot of CMS's are not built this way. I actually built my own CMS. My name is Chris and I built ChrisCMS, short for Christmas. It's a whole It's a whole thing my team wishes I never

15:37 built. But, it does work and it does bring this like token efficiency up. So, yeah, I would say llms.txt is your first shout.

15:49 llms.txt full form .txt. Second, if you if you can, just do them manually. Say you're not even working on systems that have markdown, I still recommend them.

16:03 But, you can always get creative with creating these files on the on on the go.