A 3-billion-parameter model and a 30-megabyte one can make the same security decision. For real-time protection, only one of them actually works.
Behavior detection is a decision problem, not a chat problem. On every action a user takes, you want one answer: is this risky, and what kind of action is it? That's classification, and it lives or dies on context: what screen you're on, what just changed, where you are in the workflow.
The new class of "System One" decision models is built for exactly these decisions. You give them the state and a closed question, and they hand back a calibrated answer in a single pass. TypeSafe's Jev and the open Kev family are early examples.
I'm excited to see Chrome now building a decision model right into the browser with their upcoming Decisions API.
But for real-time protection, I predict the generic models won't be enough. Running detection on every action has to be fast enough that the user never feels it, and small enough to run on any device.
A model trained for behavior analytics is an order of magnitude smaller and faster: tens of megabytes, a few milliseconds, no GPU required. That's the kind of specialized model we're building at PixieBrix, and for always-on, on-device protection.
Decision models are the right direction. The winning move today is a small model, trained for the problem domain.