To run at decent speed, all models try hard to use only most likely relevant part of the context and most likely relevant weights (MoE) to predict the next token. Doing the math in full is unfeasible.
As a hobbyist, I find it difficult to figure out how to make Claude stick with some repeating things I want it to do after every major action, like re-evaluate the completeness of tests, update the documentation, etc. And CLAUDE.md/AGENTS.md definitely did NOT help there, sadly.
What I'm currently doing in a large refactor, is I created a super-run script; the super run script is devided into super-dev (Setup dev), super-test (run all tests), super-build (build artificats), super-e2e (test all artifacts), super-deploy (deploy finished).
Each super's sub functions should _fail hard_, and each script should be highly detailed; of course I'm not doing it myself, but in small increments of directed work, it can build up the necessary harness.
What I get is a CI that just starts with "run super-run.sh" and that gives it context, then each sub script provides context depending on if it succeeds or fails. If it fails, the agent is provided what it needs.
It's basically, you have to design the products of the AI to give itself the context. Another technique I'm testing out is a parallel set of files like <subject-module>.js, <subject-module>.test.js, <subject-module>.md which get pulled up if the Agent is looking for a file.
If the harness runs on your localhost, but the inference doesn't, it usually means it's calling some API. Whether you want to break TOS of your provider like that or simply buy tokens from our friends from UTC+8 timezone is an open question.
This is the way. Making your own agent to have a sticky memory context that is prepended to every execution is necessary to ensure each task is bounded by those precepts.
The trick I'm doing -- the model is given a tool that runs a prompt in the current thread to consolidate it's working memory and identity (it has a memory tool bound to the agent persona). When the prompt ends, the parts of memory that are marked as identity are merged together into a new system prompt, then the context restarts with only system prompt and this tool call surviving. Then it just keeps going.
<project_prompts> //Duplicated with new user messsage
<llm_response>
It wouldn't bust cache, it would just make your input prompts slightly larger. Technically inefficient as you're duplicating the same rules over and over but I imagine for a smallish checklist/principles that it is tremendously more efficient than a cache break every message
so first, we treat context like a stack. Things are popped onto the stack; if we try to remove anything but the top of the stack, we'll break the cache.
So a good harness should be popping tool prompts, user hints after every return from the top so the context of tools and user hints never repeats and are allways prefixed to the current user prompt.
I've never inspected any harness, but opencode with the dynamic context compaction plugin appears to do this well.
By the time it does 10 turns of tool calls it forgets about the nagging again. It can and does ignore direct instruction from the user prompt too, sometimes repeatedly
I'm curious about how this will work on the receiving end - is that part of the infrastructure also provided by Apple or is there a common capability by 911 dispatch software to support incoming video?
I've worked with two 911 computer-aided dispatch (CAD) systems (both from Motorola, but varying wildly in quality and provenance) and neither has any such thing. Both don't directly handle phone calls either, however. They have APIs that interface to the phone and radio systems to move traffic between. In the case of the PSAP (a Public Safety Answering Point, aka a 911 call center) I support the caller is being handled by one system that passes the calling number (ANI), static telco-provided geolocation/metadata (ALI), and live GPS geolocation to the CAD system. Aside from these defined interfaces the systems are separate islands and function (and, more importantly, fail) separately.
Edit: Text messages are handled by a third-party with a SIP-based interface (running over a VPN across the Internet) to the phone system. My guess would be this is how video will be brought in. These third parties are already positioned in the telco system to handle this effectively (from a regulatory perspective).
When I was a financial auditor, one of the tests on revenue we did was a concentration test, since having a large portion of your revenue coming from a few customers is a risk to business continuity. Wonder how Nvidia folks are thinking of mitigations for that.
“ Does aspartame cause cancer? There’s a small chance that the answer is yes – but probably only if you consume 50 Diet Cokes every day from birth until death – and even then, we don’t have any evidence that this cancer would lead to premature death. At consumption levels equivalent to the ADIs set by the FDA and other organizations – let alone at lower, more typical intake levels – we have no more reason to suspect aspartame as a carcinogenic threat than we have for suspecting carcinogenicity from chewing with your mouth open or wearing flip-flops in wintertime.”
If you like this, I can only recommend Will Larson’s (the author of the blog post) book called “An Elegant Puzzle: Systems of Engineering Management” published by Stripe Press. It a collection of very tactical things of how to think about engineering careers and managing others.
not sure I got it?
Separately, the frontier labs are kinda pushing us into that behaviour by releasing models with ever-larger context windows.