Slumdog Millionaire is an excellent movie about context engineering. It explores how specific questions (inputs) retrieve specific memories (data) to generate correct answers (outputs) based on lived experience. This movie dramatizes the idea that applied intelligence is about the relevance of the context you can retrieve at the moment of inference.
Silos vs Physics
Many people struggle with context engineering because they are siloed specialists with relatively low fidelity models of adjacent or one step removed domains or even their own field at higher logical levels.
This means they can be effective at what they do for as long as the context they have optimised their skills and tooling within remains stable and was sufficiently accurate to begin with. Yet, when the operating environment or the business model dynamics shift, they risk going out of sync with what works now and they struggle to parse why. Automating this cannot work when based on a flawed premise.
World Models & Context Dynamics
Context engineering often attempts to solve for related issues at the wrong level of abstraction and with insufficient lenses of abstraction. There is a prevailing misguided notion that slightly better contextual information is sufficient. True context engineering requires a calibrated integrated world model to understand the context within context, recursively, with calibrated dynamics.
In this regard, first principled polymath autodidacts have an advantage because they can cleanly slide awareness across multiple contexts and abstractions and logical levels while staying (more) oriented and integrated and in sync with system dynamics. AI tools can be very helpful for this but only if you think this way to begin.
Rock the boat, baby
An old friend who is a PhD theoretical physicist helped unblock then fix a project to build a naval radar system by zooming out. On thinking the project through and running different simulations in their mind (aided by high IQ) they quickly realised something a whole team of specialists failed to spot. While the vessel had sufficient buoyancy to bear the weight of the radar system, by placing it too high above water level, in stormy waters, the whole ship risked being destabilized and wobbling so much it would inhibit the function of the radar, or in a worse case scenario, capsize. They were able to foresee this risk by mapping context and using a world model that included physics.
Physics & Qualitative Data
Better context engineering should include a distillation of dynamics related to the task at hand as well as the contexts of their context.
This distillation is almost always easier and more reliable when it flows top-down from a calibrated, multi-scale world model than when it is informed from the immediate task or sub-goal.
Thus, better context engineering should factor the physics and dynamics surrounding the task at hand. These can augment and steer interpretation of qualitative data, allowing you to calculate context at the moment of inference and just maybe, change your destiny.