What If You Run a Database Query on a GPU?

0:00 two numbers from a paper published two days ago. 2.04 1.76. The first one is a CPU running a database query. The second one is the same query on a GPU. Higher is better

0:12 here. So, the graphics card lost. That result comes out of Bamberg published on the 12th of September. I want to walk you through why it happened because the reason is not the one you would guess.

0:22 First, the losing run, then where the time actually went, then the one change that flipped it. The paper is by Mha Alahhabi and Maxmleian Schula. It's open access so you can go and read the tables

0:33 yourself. The machine they used is ordinary. A desktop graphics card, an 18 core Xeon, plenty of memory and a single NVME drive. Nothing exotic here and that matters later. And what they built on it

0:46 is a buffer manager, fixed size pages, a 64-bit page ID. Pin a page before you touch it. Evict it when you need the slot back. That machinery has been in databases for decades. The new part is

0:58 that GPU memory is just another tier inside it. The SSD is the storage. RAM is the host side cache. The card is the device side cache. And a page lives on exactly one of them at a time. The query

1:09 is about as plain as analytics gets. Sum a column join to a small lookup table filter on a value. 128 million rows. One gigabyte of data sitting on the disk. Now the part that should have settled

1:22 it. They also measured the operator on its own with the data already in memory. The CPU managed 23.98 GB a second. The GPU managed 71.37. Same query, same units, same machine.

1:35 The kernel on the graphics card is three times faster. That measurement is real and it isn't in dispute. It's also completely useless as a prediction. Three times faster on the inside, 14%

1:46 slower on the outside. That gap is the whole video. So where did it go? Their tables are in bandwidth, so I ran the numbers into milliseconds for the 1 GBTE table they used. Time is easier to hold

1:58 in your head. Let's take the three runs in order. The CPU pipeline cold from the disk about 490 milliseconds start to finish. The GPU pipeline same job about 568 and the GPU kernel inside that 568

2:12 milliseconds about 14 14 out of 568. The card is computing for 2.5% of the run. The other 97% is data getting to it. So let's follow the bytes. They start on the SSD. They come off the drive into

2:27 system memory. Then they cross PCIe onto the card. Then finally the kernel runs. And a pipeline only moves as fast as its slowest stage. On their machine, the drive delivers about 1.8. Every stage

2:40 after it is 10 to 40 times quicker than that. So a kernel that is three times faster is saving you time you never had. The clock was already spent upstream. Think about a kitchen. You can hire a

2:50 chef who chops four times faster. If the vegetables arrive one crate an hour, dinner is not four times sooner. That's the shape of it. But it explains a tie. It doesn't explain why the GPU came out

3:01 worse than the CPU. The paper says that part in one line. Pipeline B has to transfer every bite over both SSD and PCIe. The CPU path stops at RAM. The GPU path keeps going. And the bus itself is

3:15 not the problem here. It's wide. The problem is that the GPU pays the disc and then pays again to cross the board. It's worth being precise about which stage is actually narrow because this is

3:25 easy to get backwards. The drive gives them about 1.8 a second. The paper puts PCIe at roughly 16 to 32 GB a second. The bus is around 10 times wider than the drive. So PCIe isn't the villain.

3:39 It's just one more full pass over every bite on a budget that was already spent. Same bites, one more hop. The kernel wins its fight and the application still loses. Now they did one more thing that

3:51 I think is the most convincing part of the paper. They swept the filter from 1% selectivity up to 90. The number of rows that survive the predicate moves by a factor of 90 from about 1.3 million to

4:02 about 121 million. 90 times more work after the filter. And all three pipelines stayed flat inside their own run-to-run noise. Which tells you something exact. Change the compute by

4:14 90 times and the answer doesn't move. So the compute was never what you were waiting for. There's a name for this. The query is memory bound which means it's waiting on bytes arriving not on

4:24 arithmetic finishing. The ratio that decides which one you are is called arithmetic intensity. That means how much maths you do for each bite you read. This query does one compare, one

4:34 lookup, and one add per eight byt row. That's almost no work per bite. So the machine sits under the memory roof, not the compute roof, and you cannot buy your way out of that with more

4:45 arithmetic. A faster card would have changed nothing, which points at the only lever left. If the bottleneck is bytes on the wire, send fewer bytes. Their column is 32bit integers, but the

4:56 values in it all fit inside 16 bits. So they store the low 16 and drop the top half. Nothing is lost. Same data, half the space. 1 GB becomes 512 megabytes. Now watch what that does to the drive

5:10 because this is the part I find elegant. The SSD still runs at about 1.8 physically the same bytes per second. Nothing about the hardware changed, but every bite now carries twice as much

5:21 table. So the table arrives at 3.6. The paper keeps those two ideas apart and the distinction is worth stealing. Real bandwidth is the bytes that actually moved. Effective bandwidth is the data

5:32 those bytes stood for. Think of it as a delivery van. Real bandwidth is how many boxes it carries. Effective is how much is packed inside them. Compression doesn't make your drive faster. It makes

5:43 each bite worth more. Which leaves the obvious objection. Somebody has to unpack it and that cost something. So let's see what it cost. On cache data, the GPU operator ran at 71.37

5:55 uncompressed and 67.78 on the packed column. 5%. That's the entire bill for decoding. And it's that cheap because of where they put it. The unpacking is not a separate pass that writes a

6:07 decompressed copy into memory first. It happens inside the operator. At the moment, each value is loaded in the same instruction stream that filters and joins and sums. The widen value lives in

6:18 a register for as long as the arithmetic needs it and then it's gone. It is written nowhere. That matters more than it sounds. A decompressed step that materializes the data would hand

6:28 straight back the bandwidth the packing just bought. So, the card is doing more work per bite on a processor that had compute to spare. It was idle anyway, waiting on the drive. There's the trade

6:38 in one sentence. You spend 5% of a processor that was doing nothing and you buy back half the wire end to end. That took the GPU pipeline from 1.76 to 3.28. The paper puts it at 1.86 times and it

6:53 is now about 60% ahead of the CPU. Same silicon in the losing run and the winning run. The only thing that changed is what shape the data was in when it left the disc. Now, I want to flag

7:04 something because there's a wrong lesson sitting right next to the right one. Compression is not a GPU trick. Look at the CPU line on the packed column. It went from 23.98 to 32.60.

7:16 Same reason, it reads half as many bytes to answer the same question. The GPU just gains more from it. And why it gains more is the useful bit. It had more idle compute available to spend on

7:26 decoding. So the rule generalizes past this paper. When a processor is starved for data, its spare cycles are worth something. You can trade them for bandwidth. Though there's a ceiling on

7:37 that, and the paper is good about showing it. They also tried a 12 bit packing, which squeezes harder than 16. On cache data, it came out at 212 against 217 for no compression at all.

7:49 Very slightly slower than doing nothing. Their explanation is that the bit position arithmetic in the inner loop costs about as much as the bytes it saves. You pay for the unpacking in

7:58 exactly the arithmetic you are trying to feed. So the rule has a second half and I'll give it to you in their words. Compression ratio alone is not a sufficient predictor of effective

8:08 bandwidth. The codec has to be cheap to decode, not just small. And there's one more result which is the practical one. They stopped picking. Instead of choosing a processor, they split the

8:18 scan across both of them. 40% of the rows to the CPU reading uncompressed data out of RAM. 60% to the GPU reading the packed copy on the card. Then add the two partial sums together, 3.76, a

8:32 further 15% on top of the GPU on its own. And they can predict where the split belongs. Their model said 38%. The measured best was 40. That's worth sitting with for a second. The two

8:43 processors aren't competing for the query. They're two lanes of different widths, and you want both of them full. And that balance point moves with the codec. Squeeze harder and the best split

8:52 slides back toward the GPU because once the payload is small enough, the disk is dominant again. Now the scope because this is one paper and one workload, one query, one table, one workstation with a

9:05 mid-range card in a single NVME drive. Change any of those and the ranking can move. Put the data in GPU memory and leave it there and the loading problem you just watched simply isn't there. And

9:16 there is a paper arguing exactly that. Sirius, presented at CI this year, is a GPU native SQL engine. It reports 8.3 times better cost efficiency on TPCH when you bolt it onto duct DB. Both of

9:30 those results can be true because they're answering different questions. Sirius asks how fast a GPU is once you build the system around it. Bamberg asks, "What happens when the data starts

9:40 cold on a disc on a machine you already own?" And the mechanism isn't a one-off. A 2025 paper on running SQL analytics over compressed data lands on the same requirement. To avoid paying for costly

9:52 decoding, you need operators that work on compressed data directly. So here's what I'd take from all of this. If you're moving work onto an accelerator, measure the whole path cold, not the

10:03 kernel. A profiler telling you the kernel got three times faster is telling you the truth about 2% of your runtime. The number that predicts your wall clock is bytes moved per answer. And if your

10:13 accelerator is starved, its spare compute isn't waste, it's currency. Spend it on a denser format and fuse the decode into the operator so the expanded copy never exists at all. The verdict

10:24 for the workload in this paper, an uncompressed GPU pipeline reading from cold storage is a downgrade, not a disappointing win, an actual downgrade. Compress first, then accelerate. And if

10:35 you're still picking one processor for the scan, you're leaving about 15% on the floor. The paper is open access and the link is below. If you want the companion to this one, the video on

10:45 Bun's four times faster benchmark is the same disease in a different system, a component that got quicker inside an application that didn't.