Enjoying this issue?
Get tomorrow's AI & engineering digest in your inbox — hand-picked, summarized, and always spam-free.
TLDR
A three-stage filtering pipeline—deterministic dedup and code filters, then Jev for cheap obvious decisions, then an LLM (Claude Haiku) for nuanced filtering—reduces the cost of processing 864 AI news posts to about 45 cents per day. The speaker provides a skill that guides a coding agent through a six-stage process to build this system, emphasizing customization and best practices learned from six months of use. The key insight is that Jev alone without the LLM reduces accuracy significantly, making the two-stage Jev+LLM combination the sweet spot.
Key points
The system uses Jev as a first line of filtering before sending to an LLM for obvious decisions.
The system costs about 45 cents per day for processing 864 posts.
The speaker's skill guides a coding agent through a six-stage process to build the content engine.
The system uses deterministic deduplication and code filters before any model calls.
The speaker recommends using Appify for optimized markdown from sources.
Tools mentioned
Techniques
- Deterministic deduplication and code filters
- Three-stage filtering pipeline: deterministic, Jev, then LLM
- Using Jev for cheap obvious decisions
- Using LLM for nuanced decisions
- Using a configuration JSON file for source management
- Using a skill to guide a coding agent to build the system
Stop scrolling. Start reading smarter.
Receive the day's most important AI & engineering updates in one concise email. No spam.
Transcript (captions)
If you've never felt overwhelmed with all the news in the AI space, then you should just go start your own YouTube channel and teach us your ways. Because I can guarantee that's not the case. At
least some of the time, maybe even right now, you're feeling overwhelmed with the onslaught of new models and product releases, viral tweets, and GitHub repositories that you have to check out.
I mean, every single week, it feels like there are multiple things that come out that change everything. And so, it becomes overwhelming. And the big question we have to ask ourselves is,
how do we keep up with it all? and how do we figure out what's fluff versus what is actually worth our time looking into and maybe incorporating for ourself? And that's the big question
I've been answering for myself this entire year, building what I call my AI content engine within my second brain. And so in this video, I'll show you how this whole system works and even better,
how you can build it yourself pretty easily using a skill that I have for you. And I'm excited for this because this system is a lifesaver for me. It tracks hundreds of sources on my behalf.
And then for all of the announcements and blog posts and articles, everything that goes through the system, it filters based on what it knows I care about and my content. So all I get every single
day is a concise and easy to read daily digest that covers everything I should care about. So it's not helping me track all the AI news because even for a digest, that would be a lot to read. But
it's just the things that I care about. And I've made this system very accurate over time. So this filter truly is taking out the fluff for me and helping me hone in on what's going to make my
content better, my AI coding workflows better, just the things that are personal. So within my main skills GitHub repository, I have our skill for today. Build signal engine signal
content. I call my system either. This workflow guides your coding agent through building the exact same system that I have that's saving me hours and hours a week of going through all the AI
news. And the reason why I have a skill instead of just giving you the system as an open source repo you set up right away is I want you to be able to customize things as you build it from
the ground up. And so as a part of this process at the very beginning, it grills you. It figures out what sources you want to track, how you want to fetch them, the filtering you want to do, like
what you actually care about, and then it builds the system. But the value in this skill is that all the best practices I've learned, you know, lessons I've learned the hard way,
refinements that I've made as I've been using my content engine for over six months now, that is all built into this skill. But before we go through this skill together, I want to show you how
the system works because I'm a firm believer for anything we set up or run ourselves, we should truly understand how it's working. So, the first thing we have to figure out before we set up any
of the filtering or teaching it what we care about is we need to build the data layer, the connections to the different platforms, and figuring out what we're going to pull into the content engine in
the first place. There are so many sources that I'm pulling in every single day. Reddit, YouTube, RSS feeds, hacker news, GitHub repositories, change logs from my favorite platforms like Claude
Code. There are hundreds of posts that I'm feeding into my content engine every day. But it's also able to work through everything very, very quickly. So, a ton of sources, but I made it very
efficient. And of course, that's part of what the skill will help you build as well. So, the skill is going to ask you the different sources you want to connect to and then also how you want to
do so because for every single platform, there are many different ways we can connect directly with an API or a CLI tool or MCP server. And so, I've done a lot of that for, you know, like GitHub
and archive and hackernews. But something that I've started to use recently is Appify. It is a single platform that allows me to very easily connect my agents to thousands of
different sources like YouTube and Reddit. Also, the kinds of things where usually the APIs are more restricted. And so again, it's not like you have to use the same solutions that I'm using,
but this is what's working really well for my content engine. And the skill will help you customize. So you figure out your data layer and what you want to connect to. And then we move on to the
rest of the process where we're actually setting up the filtering and getting to that daily digest. So first thing we do when we run the content engine is we pull all of our sources and I run my
content engine every single day. But the very first thing we have to do before we do any kind of processing or filtering is dduplicate. So storing everything in a central database and keeping track of
the announcements and blog posts and things that we have already seen because we don't want to have Jev and our large language model process the same blog post or announcement or whatever twice
because that's just a waste of our tokens. And so before we get into any of the model usage, we want to do deterministic filtering and dduplicating as much as we possibly can. That's part
of what makes this whole system very fast and token efficient. And so we dduplicate. And then I also have some code filters, some deterministic filters that I've built in. And so if there are
really old posts that are likely stale, I'm just going to scrap that right away. or if there's a lack of engagement on a post, like if there's, you know, for example, a hacker news article or a
YouTube video I pulled in that only has a couple of hundred views. I mean, maybe that's good information, but a good general rule of thumb that I've seen is usually I can skip past that thing. And
so, being able to track that before I'm actually using my tokens to determine if it's important to me. And then for whatever gets past the deterministic filters and dduplication, I send that
into the first model call using Jev. So, this is something I covered in my last SecondBrain Jev use cases video. I'll link to that right here. But, I love using Jev as the first line of defense
for models doing the filtering. And so, Jev hasn't been around for super long. Before Jeb, I was using LLMs to filter everything. But that meant making an LLM call for every single source that made
it past the first couple of stages. And you can use a cheaper model for this like Haiku. Especially with the new Haiku 5.5, you can have a lot of reasoning capability for super cheap,
but I still don't want to use an LLM as much as I possibly can avoid it. And so that's why I'm using Jev as the first line of defense. Now, I did try a version of this system where I didn't
have an LLM at all, and Jev was the last line of defense. My accuracy actually went down a lot. I had a lot of things slip through that I didn't care about, a lot of things filtered out that I
actually did. But I found this to be the sweet spot here. So I use Jev to make more of the obvious decisions. Anything that is very obviously fluff or very obviously something I care about, it's
going to make that decision first, right? So if it's like, yes, we definitely should drop this, then it exits and I never have to make an LLM call. But for anything where it's unsure
or it knows that I need to definitely keep, that's when we send it to an LLM to just get everything ready for the digest and do any further filtering that I need. And so to give you some concrete
numbers here, for this run that I did with 864 posts, it only costed three cents for all the calls I made to Jev. And then for what survived to Claude Haiku, it only costed 42. And so it's
less than half a dollar every single day. Not that bad. For how much time it saves me, it is definitely worth it. Now, obviously, the price for you is going to change depending on how many
sources you're pulling from and how much text you get per source. And so also something you want to be very aware of here is for every source you pull through, generally you don't want to
just dump the raw output of the MCP server call or API call into Jev or your LLM. There's almost always going to be some things you can deterministically strip like headers and footers, just
extra spacing and things that the agent doesn't really need. That's one of the reasons I like using Ampify is because for all the runs that I have here as it's pulling from my different sources,
it automatically returns markdown that is optimized for the agent, like just the information that it needs. And so using something like Ampify, you have to worry about it less. But if you do have
a custom setup, just keep in mind that's very important for you to establish upfront. And of course, that's one of the things that my skill helps you with. And I also put together this little demo
here just to make things really concrete. So this is not my full content engine. This is just from pulling through a couple of subreddits, but it's a nice visualization to see what it
looks like to filter some things with Jev and then send other things to the LLM. And so for every single source that makes its way to Jev, it's going to evaluate based on three pieces of
criteria, and this is built into the skill as well. It's going to figure out, is this on topic? Is this something Cole would care about? Is this actually new and concrete, or is it just a toy demo?
A lot of things that I find with blog posts and especially subreddits is people are just sharing little things that they built to try to benchmark models that I don't really care about.
And so this is one of those lessons I learned where I see these like toy demos or they like to call them a lot and I just want to filter that out right away. So anything that scores even reasonably
high with this like a 7, I'm just going to filter it out. I'm just going to drop it. I'm not going to send that to the LLM. And so there's some things that are obvious enough where like this is so
good. Like the announcement for Sonnet 5.5 or the release of the official 1.0 of Pi. Like yes, I'm definitely want to keep that. I don't even need the LLM to decide. And then other things where it's
more nuanced and maybe I want the LLM to dive a little bit more into my memory system because Jev can't really do that. That's when I send it off to the LLM to reason more. Do I actually keep it? The
system is still light. It's still going to be cheap when it goes to the LLM, but just not quite as much. And then for every single one of the sources that I want to bring through the content
engine, I just manage them all with a single configuration JSON file. It could not be easier for me to manage and extend this. Like here are all of the RSS feeds I want to follow, all the new
blog posts from this YouTube channels, GitHub repositories, the categories for archive research papers. For Ampify, I have the searches I want to perform every time with Reddit and web search
and YouTube. So catching more general things outside of the individual channels as well. All this together makes it so I feel like I have a really good comprehensive grasp on the whole
internet as new things are coming out. Awesome. So, now that you understand the system at a high level, we can get into using the skill to build your own personal content engine. I'll just walk
you through getting started because the skill really does go through the entire thing with you. And so, the easiest way to install this skill is to follow the instructions in the readme in my skills
repository. I'll link to this in the description. If you're using Cloud Code, I have a Claude Code plugin to make it so incredibly easy to bring this into your system. And if you're using a
different coding agent, you can just point it at the URL for this repository and ask it to set up the skills or just download the single skill that we're using right now. And so once you have
this set up, I would recommend running this skill in a new repo, a new fresh folder where you want to build the content engine. You can optionally specify the path to a separate
repository, like if you're building it in your second brain, but you want it to actually be created somewhere else. I just wanted to leave that as an option. But generally when you build this,
you're simply going to do /build signal engine. And that is it. You don't even have to specify any arguments. So I'm going to send this in. And the first thing it's going to do, like I mentioned
earlier, is grill you with a bunch of questions. So it can really be custom to you. So it starts by asking where should we actually build this? It's going to default to recommending where we
currently are. So I'm just going to go with that because I created a fresh folder for this. And then it'll start getting into asking, you know, what do you actually need to keep up with?
Right? So the higher level questions. So it can even maybe recommend different sources. So I want to keep up with, you know, AI tools that I use or maybe you want to keep up with specific platforms
you want to call out. It's entirely up to you how you answer these questions and communicate with it. Tell me about something in your AI tools that you found out about late or missed entirely,
right? Like what is the actual problem? If we get more granular here that we're trying to solve, right? So maybe it's like later we found out about something in the change log that we wish we knew
about a week earlier or we found out from a YouTube video like oh man this thing came out a couple days ago and I didn't find out until this YouTube video like I really wish I knew about it right
away and then getting more into the sources and everything. I'm not going to go through and answer every single question here cuz I want you to do that very accustomed to what you care about
but I'll show you the final output. So I went through this process already where I answered every single one of the questions. It really does grill you. You have to be a little bit patient here,
but I I recommend really investing the time into this upfront because it's going to make it so that it can just rip through the entire implementation. So, you spend, you know, 10 15 minutes
answering its questions and then it builds out the entire thing and it actually does it in phases so it's very comprehensive and even testing things for your personal engine as it's
building it. So, going back to the diagram here, after the interview, the skill is going to walk itself autonomously through a six-stage process to build out everything. So, we're not
just trying to oneshot it. We're having it split up into bite-sized pieces of work, and it's going to test after every single one of them. It would definitely fall flat on its face if it just tried
to oneshot everything. So, it starts very simple, just one source that you agreed upon in the interview, building that end to end and testing it. Then moving on to building every other source
and fully establishing this data layer here and doing it according to the platforms and methods that you specified. Like for myself, for this demo here, I said that I want to use
Appify for everything. And so that's what it's going to be integrating with as it builds out the data layer. But whatever you decided, whatever customizations you have, it's going to
work with that. If you want to use APIs directly, whatever, it's going to build that out. And then it's going to build the code filters, the deterministic ones, and test that, the decision steps
with Jev and the LLM, building the format for the digest, and then also helping you set the schedule for the frequency you want to run this on. And so all that together is just going to
rip through this. And so if we go back to my conversation here to the very end, boom, we have the full engine built. Now, this might not be perfect right away. So you can definitely iterate with
the agent here, but it is going to be a very good starting point. The other thing that I built into this skill is it produces a profile at the end. So you have an at a glance view for the sources
you're pulling from, the filters and decision questions that you have. So if there's something that doesn't look perfect here or exactly what you want, you can just adjust it and then point
the agent at it and say that like, you know, here's a new profile. I want you to implement the differences. And so this is kind of your way to just constantly adjust things. So keep in
mind the content engine is not reading this every time it runs. But whenever you want to change something, you can just update it here in natural language and then tell the agent to implement
those changes. So there you go. That is how my content engine works and how you can build it yourself with the skill. And even if you don't build it from scratch here and you just want to take a
couple of golden nuggets from this video and incorporate into your own process, I hope that you find that really useful. And so if you appreciated this video, you're looking forward to more things on
agentic coding and building up your second brain, I would really appreciate a like and a subscribe. And with that, I will see you in the next