Building semantic tools

Ask a coding model to build a searchable catalog, or a media model to assemble a film from several clips. A remarkable part of the result is what the model does not have to invent.

It inherits an accumulated vocabulary:

• Shell composition: one program’s output can become another’s input through a defined connection. The model can assemble existing programs instead of implementing each transformation. The shell pipeline specification gives that connection precise behavior.
• Media operations: FFmpeg exposes streams, filters, timestamps and ways to connect them. A cut and a change of time origin are distinct operations: trim preserves timestamps; setpts can rebase them. That distinction matters when segments are joined. The model is working with a temporal representation, not merely an array of colored pixels. FFmpeg filter documentation
• Interface structure: React provides components, data flow and state. A search field and its result list can depend on the same state instead of becoming unrelated patches to the screen. Thinking in React

These tools carry decisions made over many years: what an object is, how operations combine, what changes when they execute, and which details a caller can leave to the implementation. Model capability arrives inside that accumulated work.

This accumulated work suggests a productive question: what capabilities could appear if we deliberately built equally useful vocabularies for work that is still awkward to express—and trained models to use the tools we created?

Give an intention something to operate on

An editor says: “Keep the quotation and its attribution together. Give the opening more air, without adding a page.”

There are many possible low-level responses: move text, shrink a font, alter line breaks, increase a margin. Some satisfy one part of the instruction by quietly violating another. A more useful interface would represent the quotation as a protected object, the attribution as its dependent, and the page count as a constraint. It could then propose changes within the editor’s permitted range.

The operation needs more than a pleasant name. It needs a meaning that survives execution:

• Objects: the quotation, attribution, opening paragraph and page—not just coordinates that happen to contain them.
• Allowed changes: spacing, measure or placement, with explicit ranges.
• Preserved relations: the words remain unchanged; the attribution stays attached; reading order survives.
• A useful result: feasible alternatives, or a precise conflict that lets the editor decide what to relax.

This is what I mean by a semantic tool: an operation whose objects and consequences correspond to distinctions that matter in the work. It can contain algorithms, ordinary code, learned judgments, or a combination of them. Calling a function make_elegant does not supply the missing meaning.

Typography supplies a mature example of this move. Knuth and Plass represent a paragraph using boxes, adjustable glue and penalties for possible breaks. Their line-breaking method considers the paragraph as a whole, rather than greedily finishing each line. That representation makes a family of typographic choices available to an algorithm. Breaking Paragraphs into Lines, 1981

The optimization is exact relative to its objective. The objective remains a choice. A publisher may value compactness differently from a designer; an editor may reject a mathematically preferable layout because it gives the wrong emphasis. The proposed tool should expose that choice rather than conceal it in a supposedly universal “quality” score.

Work through a conflict the tool can explain

Here is a small operation we can actually calculate. Take a fictional feature with two columns, each 500 layout units high. Keep its words, font sizes, column width and existing line breaks fixed. The opening occupies fourteen lines at a line height of 20: 280 units. A six-line quotation at 30, followed by a 20-unit attribution, occupies 200. Sixteen closing lines at 20 occupy 320. Putting the opening and quotation in the first column uses 480 units; the closing uses 320 in the second. Inspect the computed original page.

For this example, the editor resolves “more air” into a specific change: raise the opening’s line height to 24. The opening now occupies 336 units. The whole piece needs 856, less than the page’s total capacity of 1,000. Yet it cannot fit while the quotation and attribution remain together:

• Break before the quotation: the columns need 336 and 520. The second overflows.
• Break after its attribution: they need 536 and 320. The first overflows.

The conflict is more precise than “the page is too full.” Let b be the amount of content before the column break. The first column requires b ≤ 500; the second requires 856 − b ≤ 500. A fitting break must therefore lie between 356 and 500. Every such position lies inside the protected quotation-and-attribution block, which runs from 336 to 536. Breaking there would satisfy the geometry by violating the editor’s instruction.

Now let the editor permit quotation line heights of 26, 28 or 30, and closing line heights of 18, 19 or 20. The opening stays at 24. Exhaustive search finds six feasible settings among the nine combinations. Three retain a meaningful tradeoff: none changes both kinds of spacing less than another feasible setting does.

Quotation line height	Closing line height	Column heights	Inspect the computed page
26	20	336 / 496	Keep the closing spacing
28	19	336 / 492	Change both more moderately
30	18	336 / 488	Keep the quotation spacing

These are alternatives, not first, second and third place in a taste ranking. If preserving the closing spacing takes priority, the first setting follows. If preserving the quotation spacing takes priority, the third follows. Requiring both original spacings leaves no solution on this page; with everything else fixed, the smallest feasible equal column height is 520. The tool can return that exact change for consideration instead of silently reducing the type or rewriting the text.

Constraint hierarchies give this distinction a formal precedent. The Cassowary paper separates required relations from preferences and compares errors by strength: an improvement at a weaker strength cannot compensate for a worse result at a stronger one. It also represents a preference for retaining an existing position. Those mechanisms make priorities and continuity part of the operation’s specification. Our finite enumeration is not an implementation of Cassowary, whose linear solver does not by itself choose discrete text breaks. The Cassowary Linear Arithmetic Constraint Solving Algorithm, Section 1.1

The example’s boundary matters. It holds wrapping fixed, keeps at least two lines on each side of a split paragraph, and searches only the stated spacing choices. A wider column, a new image or a rewritten sentence would require a different operation and fresh measurements. The source text, exact font measurements and exhaustive arithmetic are retained with the construction; the pages show computed line-box arrangements, not a completed publication-design product.

This also gives custom training a sharper target. “Keep the closing as it is” and “keep the quotation as it is” should produce different priorities over the same available operations. “Keep both” should produce the conflict. Training and evaluation can vary the document, requested preservation and feasible alternatives independently. Matching a tool name is insufficient: the selected authority, operands and resulting preserved relations must agree with the request.

Now the first operation can become a component of another. “Balance this spread” might coordinate paragraph fitting, quotation placement, captions and image crops. A publication-level operation could coordinate several spreads while preserving section openings. Each level makes some relations explicit and delegates others. The useful vocabulary grows recursively.

Give the work a different rhythm

Now give the editor a six-page feature about a repair workshop. The source material includes a photograph of a repaired chair, a close-up of its joint, a drawing of the repair, the maker’s explanation and the owner’s account of why they kept it.

The editor asks: “Let readers notice the repair before explaining it. Open quietly, build density around how it works, and end with the chair back in use.”

That request describes an experience. A useful operation could make several arrangements of the material available for inspection. One opens with the repaired joint and very little explanation, uses the middle spread to reveal the method, then returns to the whole chair. Another begins with the owner’s reason for keeping it; the detail and method become answers to a human question. The editor can compare how each version changes the significance of the same material.

The operation has several nested levels:

• Argument and experience. Represent the parts by what they do: establish a question, explain a mechanism, supply context, offer testimony or return to an earlier image with a new meaning. A learned interpretation proposes those roles from the material and the editor’s intention. The editor can correct a reading: the owner’s quotation expresses attachment; it is not evidence that the repair is structurally sound.
• Sequence and rhythm. Represent when each role appears and how attention moves between them. “Open quietly” could mean a full image with a short line, followed by a deliberately denser explanatory spread. “Keep the doubt alive longer” could move the explanation while preserving the material that makes the question intelligible. These are alternative editorial strategies, not a universal recipe for good pacing.
• Spread composition. Turn a strategy into choices about image scale, contrast, text density, repeated motifs and relationships across the gutter. A learned model could propose a composition; a solver could find layouts that meet its selected measures and spacing rules. The page-fitting and quotation operations become components here.
• Rendered particulars. Resolve line breaks, crop bounds, alignments and linked captions. Exact algorithms can check many of these consequences. The rendered alternatives return to the editor, who judges whether the intended rhythm actually works.

Rhetorical Structure Theory offers a useful precedent at the first level. Mann and Thompson describe relations among text spans through their function and the effect an author plausibly intends for a reader. They explicitly allow ambiguous and simultaneous analyses. A functional account can therefore be precise enough to discuss without claiming that a passage has one objectively correct role. Our extension into visual sequence and interactive editing is a design proposal. Rhetorical Structure Theory: A Theory of Text Organization, 1987, Sections 2, 5–6 and 9

The new power is the ability to explore an editorial intention across these levels. “Keep the quiet opening, but let the owner’s reason arrive earlier” can regenerate a sequence while retaining a chosen compositional quality. “Show the method without turning the feature into a manual” can produce a different distribution of detail. The person is shaping how the material becomes an argument, not merely choosing a page template.

A theory supplies distinctions; the learned model interprets this material and request; the algorithms turn chosen relations into candidates. None of those alone determines the result. Together they could create a working vocabulary for a kind of editorial judgment that is otherwise scattered across many small edits.

Representations change the work itself

An external representation can do more than save keystrokes. It can change which mental task a person performs.

In Edwin Hutchins’s study of cockpit coordination, prepared marks on an instrument let pilots judge spatial relations instead of repeatedly recalling and comparing numerical speeds. The information was distributed across people, procedures and physical representations. The explanation concerned the whole system, not an isolated improvement to individual memory. How a Cockpit Remembers Its Speeds, 1995

There is a corresponding opportunity in an editor’s workspace. If the tool retains the protected text, allowed changes and reasons a layout was rejected, the person can compare alternatives directly. They no longer have to remember a sequence of coordinate edits and reconstruct what each one disturbed. A correction such as “keep this emphasis, but restore the original measure” can address the structure of the work.

The representation also determines what can be asked. A document that knows only glyph positions cannot directly answer “which captions will lose their illustrations if this section moves?” One that represents those associations can. A model may help recover the associations from an old document; once accepted, they can become explicit objects used by subsequent operations.

This is a practical reading of Engelbart’s augmentation framework: improvements to symbol manipulation can enable new working processes, which in turn make more powerful representations useful. The opportunity is not limited to accelerating an unchanged procedure. It includes work that becomes thinkable because the person can now organize and inspect it. Augmenting Human Intellect, Section II, 1962

Edit a useful view without losing the rest

Return to the repair feature. The editor wants to move the owner’s account before the explanation of the joint. An outline is a convenient place to make that change. It is also an incomplete view: it may show each block’s role and position while hiding the exact words, image crops and credits.

The useful operation is to change the sequence while retaining that other work. Regenerating the whole document from its outline would give the system far more discretion than the editor requested.

Bidirectional transformations provide a precise starting point. Foster and colleagues define lenses with a function that extracts a view and another that puts an edited view back into the original structure. Two laws constrain their relationship, wherever the operations are defined:

• GetPut: putting back an unchanged view leaves the original structure unchanged.
• PutGet: extracting a view after putting it back returns the requested view.

The laws do not choose a unique update policy. The paper develops combinators for expressing that policy and composing transformations. Combinators for Bi-Directional Tree Transformations, POPL 2005, Sections 2–4

For our proposed editor, the policy could be deliberately narrow:

• The outline refers to existing blocks by stable identity; it does not reconstruct them from their titles.
• A sequence edit changes order. Words, crops and credits remain attached to their blocks under separate preservation checks.
• A request that requires new material or breaks a protected association returns a choice about that additional change.

The learned model has a specific job here: interpret what should move and why. “Let the owner speak earlier” proposes a sequence change; “make the owner’s argument clearer” may require rewriting. A model trained on this vocabulary should learn that difference. The document operation carries out the accepted change and checks its effects.

This could make several levels of a work directly editable: its outline, its composition and its particulars. A person could move between them without asking a model to recreate the whole work each time. The new freedom comes partly from the model’s interpretation and partly from a representation designed to remember what the person has already decided.

A high-level tool can be probabilistic

Consider a survey team asking: “Which wetlands need another visit before we draw conclusions about this species?”

The raw material is a series of visits. A species might be detected, not detected, or not surveyed. Those are different observations. A zero on a completed visit is not the same thing as a blank cell, and neither directly establishes absence.

MacKenzie and colleagues developed an occupancy model that separates the probability a site is occupied from the probability of detecting the species when it is present. Repeated visits provide information about those distinct quantities. Missing visits contribute no detection observation. Their basic model assumes occupancy remains unchanged during the survey period and that detections are not false positives. It also explains why weak detection and limited sampling can leave occupancy poorly determined. Estimating Site Occupancy Rates When Detection Probabilities Are Less Than One, 2002

Here is a small illustration, using assumed probabilities rather than fitted field results. Suppose occupancy probability is 0.6 and detection probability on each visit, given occupancy, is 0.4. With conditional independence and the same occupancy state throughout:

• Three completed visits with no detection leave a probability of about 24.5% that the site is occupied.
• Two nondetections and one missing visit leave about 35.1%.
• No visits leave the original 60% assumption unchanged.

For three nondetections, the calculation is 0.6 × 0.6³ / (0.4 + 0.6 × 0.6³). The numerator is the occupied-but-missed possibility; the denominator also includes the unoccupied possibility. The missing visit supplies no extra factor. These values illustrate the model’s logic; they are not estimates for real wetlands.

A useful review_survey_coverage operation could combine the histories, sampling conditions, fitted model and its diagnostics. Its output would help a specialist distinguish “little evidence of presence” from “little opportunity to observe.” A planning operation could then propose another visit subject to travel time and the team’s objectives.

The model interpreting the request should not silently substitute a different scientific question. “Where is the species disappearing?” requires evidence about change across periods; a single-period occupancy analysis does not answer it. The tool vocabulary makes that mismatch visible and gives the conversation somewhere useful to go.

This example also breaks a misleading hierarchy. High-level tools are not necessarily vague, and probabilistic tools are not necessarily low-level. A paragraph optimizer may be deterministic. An occupancy analysis has explicit uncertainty. A learned interpretation may select either one. Each deserves the representation and evaluation appropriate to what it does.

Work through the decision the tool would make

“Which site should we visit next?” sounds like a single operation. Even in this small example, its meaning depends on what the team wants to learn.

Continue with three fictional sites: A has three nondetections, B has two, and C has not been visited. Keep the same assumed occupancy and detection probabilities. Suppose one further visit costs the same at each site. If the purpose is to obtain a detection, C is the best of these three choices: it has the largest probability of being occupied. If the purpose is to reduce uncertainty about occupancy, the answer changes.

We can calculate that distinction without asking a language model to invent a ranking. Let Z indicate occupancy, Y indicate detection on the next visit, q = P(Z = 1 | history), and p = P(Y = 1 | Z = 1) = 0.4. Under the stated no-false-positive assumption, the complete prediction is:

Possible state and observation	Probability
Occupied, detected	q × p
Occupied, missed	q × (1 − p)
Unoccupied, no detection	1 − q
Unoccupied, detected	0 under this model

There are only two possible observations to average over. A detection makes occupancy certain within this model. A nondetection changes it to q × (1 − p) / (1 − q × p). Neither result is known when choosing the visit.

Write binary uncertainty in bits as h(q) = −q log₂(q) − (1 − q) log₂(1 − q). The expected reduction is the uncertainty now minus the probability-weighted uncertainty after observing the result:

G(q,p) = h(q) − (1 − qp) × h(q(1 − p)/(1 − qp)).

The detection branch contributes zero remaining entropy; the nondetection branch carries the other term. The same quantity can be derived by exchanging the roles of state and observation: G(q,p) = h(qp) − qh(p). That second expression subtracts the uncertainty caused by imperfect detection from the uncertainty in the observation. Our accompanying calculation enumerates the complete joint table and checks both expressions against mutual information, including perfect detection, impossible detection and already-known occupancy.

Site	Current occupancy probability	Chance of a detection next visit	Expected information about occupancy
A: three nondetections	24.5%	9.8%	0.225 bits
B: two nondetections	35.1%	14.0%	0.244 bits
C: no visits	60.0%	24.0%	0.212 bits

These are calculated values from an authored example, not measured survey results. C offers the best chance of a detection; B offers the most expected information. A tool that treats “find the species” and “resolve uncertainty” as interchangeable will give the wrong answer to one of those requests even if its arithmetic is flawless.

Expected information gain and its sequential extension are established experimental-design objectives. Rainforth and colleagues derive them from a model of possible observations, explain updating the design after each observation, and examine the computational difficulty of doing this for richer models. They also distinguish information gain from other utilities. Modern Bayesian Experimental Design, Sections 2–3, 2023

The professional operation now has a more substantial architecture:

• Interpret the purpose. Map the request to an explicit target: maximize the chance of a detection, learn occupancy, estimate detection reliability, or support a specified management decision. These are different quantities. The domain model should learn to recognize them and ask when the request does not choose one.
• Construct the belief state. Combine the admitted visit histories with the observation model. Preserve unvisited sites, missing observations and the period over which occupancy is assumed stable. This stage produces probabilities, rather than a categorical “absent” label that destroys uncertainty.
• Evaluate candidate observations. Compute the possible outcomes and their consequences for the selected target. Here exact enumeration suffices. A larger problem may need numerical integration, simulation or a learned estimator, with its approximation error exposed.
• Plan an achievable visit. Combine the selected utility with access, travel, weather and remaining time. Ranking single visits by bits per hour does not generally solve a route or a sequence of dependent experiments. The planner must represent those constraints and interactions.
• Update from what happened. A new observation changes the belief state, which changes the next decision. Replaying an old ranking after the visit would break the meaning of “next.”

The small calculation also exposes what must change when the operation grows. If detection probability is uncertain, an observation can teach us both about presence and about the chance of missing the species. Marginalizing over that uncertainty generally differs from inserting its mean into the equations above. If detection conditions are shared across sites, learning at B may change what we believe about A. A per-site ranking that assumes independence would then omit a useful connection. If false positives are possible, a detection no longer makes occupancy certain.

These changes require a different model or state representation, not a more confident explanation of the same number. They are opportunities for new operations: estimate the observation process, propose a visit that distinguishes two explanations, or compare a route under competing assumptions. An expert can expose those operations as tools and train a model on their distinctions, operands and limits.

This is the coupling worth developing. The model recognizes what inquiry the person intends. The tool gives that inquiry an executable definition. Their training examples can include the same survey history paired with different purposes, histories with missing visits, and requests that the current assumptions cannot answer. Evaluation can then check the selected objective, the calculated decision and the behavior after a new observation separately. A correct-looking site name alone would miss most of the work.

Meaning has to survive composition

A well-formed chart can still say something false.

Suppose our survey interface maps “detected,” “not detected” and “not surveyed” to bars of increasing length. The encoding implies an order that those categories do not possess. A more attractive chart does not fix the relation it invents.

Mackinlay’s APT work separates expressiveness—whether a graphical language can represent the intended information—from effectiveness—how well a person can perceive it. Its composition approach joins compatible encodings of shared information. This is a deeper requirement than arranging independent charts neatly. Automating the Design of Graphical Presentations of Relational Information, 1986

For a new tool, that suggests questions at every join:

• Does this map refer to the same sites and survey period as the table beside it?
• Does an estimated quantity remain visibly different from a direct observation?
• Does combining two outputs preserve their units, denominators and uncertainty?
• Does the visual operation imply a rank, precision or causal relation that neither input supports?

Semiotics helps articulate why these are substantive questions. In Peirce’s account, resemblance, connection to a particular thing, and learned convention do different work. A picture may resemble a place without establishing that it records an observation there. A symbol’s familiar appearance does not supply its referent. What Is a Sign?, 1894

For an interface, the consequence is concrete: “observed,” “estimated” and “proposed” should remain distinct kinds of things as they move through tools. Their identity cannot depend on a paragraph of explanation that disappears at the next step. This is our design inference from the sources, not a claim that a philosophical taxonomy is itself an executable type system.

Make an operation reusable enough to become a language

Imagine a workshop repeatedly making enclosures for different installations. A useful operation could accept internal dimensions, a mounting pattern and a clearance rule, then construct a part. Another could generate a compatible family of parts and a material list. A third could compare proposals against available installation space.

The vocabulary progresses from geometry to a part, from a part to a system, and from a system to a decision someone can inspect. The higher operation remains connected to its lower-level meaning. Changing the material list cannot silently change the geometry; satisfying a clearance rule does not prove an unmodeled load requirement.

CAD shows how rich this vocabulary can become. FeatureScript makes custom features ordinary participants in a parametric model, with dimensional types and deterministic execution. Its language introduction describes that foundation. The current official AI-assisted feature workflow goes further: describe a custom feature, generate and refine its code, then reuse it. That is a constructive precedent for turning a local method into a continuing capability.

The opportunity extends to operations that a particular practice has not yet made explicit. An enclosure specialist might care about a recurring installation exception. An editor might have a recognizable way to preserve emphasis across formats. A research group might need a composition that keeps observation and inference separate through several transformations. The valuable unit is neither an isolated API call nor an entire profession. It is a useful piece of work with enough structure to be reused and enough openness to be revised.

McIlroy’s early argument for software components already treated families, interfaces and discoverability as central problems. The accompanying discussion also exposed how difficult it can be to choose the right units. That history is useful here: a reusable vocabulary requires invention, not just packaging. Mass Produced Software Components, 1968

Teach a model the tools you have created

There are several ways a model and a tool can meet. They should not be collapsed into one claim.

• Learn to use supplied tools. Toolformer constructs tool-use training examples by proposing calls, executing them and filtering for their contribution to a language-model objective. The APIs are supplied; the system does not invent a missing domain operation. Toolformer
• Learn API selection with documentation. Gorilla studies fine-tuning and retrieval for invoking existing APIs. Its single-call evaluation is not an evaluation of an entire professional workflow, but it makes the learned interface a concrete research object. Gorilla
• Create a reusable tool, then use it. LATM separates a tool-making model from a tool user. Its user learns from examples in context; that is different from training new model weights. Large Language Models as Tool Makers

Let the operation change the learning problem

Return to the exact two-column construction. The opening’s line height must become 24. The words, wrapping, reading order and protected quotation stay fixed. Compare the original 500-unit column height with a second document state whose columns are 520 units high. The only other permitted changes remain the nine quotation/closing spacing pairs calculated earlier.

Make the editor’s priority explicit. Closing first minimizes the change to closing spacing before considering quotation spacing. Quotation first reverses that order. Both required permits neither spacing to change. These are chosen preservation rules, not a universal measure of good design.

Preservation rule	Exact preferred result at height 500	Exact preferred result at height 520
Closing first	Quotation 26, closing 20; columns 336 / 496	Quotation 30, closing 20; columns 336 / 520
Quotation first	Quotation 30, closing 18; columns 336 / 488	Quotation 30, closing 20; columns 336 / 520
Both required	Conflict: no legal fit	Quotation 30, closing 20; columns 336 / 520

At 520, the original quotation and closing spacing fit. Changing them would introduce an unnecessary deviation under either preference order. “No other spacing change” still includes the requested opening change to 24; it does not mean leaving the whole document untouched.

Now consider two interfaces we could teach a model to use.

A numeric interface asks the model to select the actual spacing. Its target changes with the column height. A single-step selector given only the request cannot distinguish the two states. For each of the three requests, it must emit the same answer in both states, although their exact preferred answers differ. Over these six equally weighted constructed cases, any such selector can match at most three of six. Enumerating all 1,000 deterministic policies over the nine spacing pairs plus conflict gives that same bound. Randomizing among them cannot improve the maximum expected match rate.

That is a limitation of this deliberately incomplete input, not a measured failure rate for a model. Supplying the current height makes all six target choices representable. Allowing the model to inspect the document through another call also changes the problem. More training on the same indistinguishable inputs, by itself, does not supply the missing distinction.

A semantic interface asks the model to select the preservation rule. A fit_opening operation receives that rule, the identified text blocks and the current document revision. It reads the current geometry and searches the permitted spacing/break combinations itself. The interpretation “closing first” is now the same in both document states; executing it produces different, appropriate numeric results. The tool can also report the exact conflict when both original spacings are required on the shorter page.

The document information has not disappeared. It has moved behind an operation that knows how to use it. The model still has to identify the intended blocks, distinguish a preference from a requirement, and recognize requests outside the operation’s authority. “Shorten the quotation” requests different authority from “keep the quotation with its credit.” A correct solver cannot repair the wrong interpretation of that distinction.

State-abstraction research gives a useful way to ask what a representation must preserve. Li, Walsh and Littman distinguish aggregations that preserve an environment’s transition/reward structure, optimal action values, or only a common best action. Those are different guarantees: representing a correct policy does not by itself ensure that planning or learning over the aggregated problem recovers it. Their setting is finite Markov decision processes. Our six-case construction is a narrower, one-step interface comparison, not an application of their reinforcement-learning convergence results. Towards a Unified Theory of State Abstraction for MDPs, Sections 3–4, 2006

The practical question is therefore sufficient for which decision, through which tool? A numeric selector needs information that the higher-level fitting operation can handle internally. A model deciding whether a quotation belongs to a photograph needs different context again. There is no single amount of “semantic detail” that is sufficient for every task.

Train and test the boundary you actually built

For the semantic interface, training examples should pair requests with the relevant objects, preservation rules and authority. The same request can recur across different geometries while keeping its interpretation. A changed preservation request can reuse the same document while changing the result. Those two variations test different parts of the system.

Evaluation should inspect the chain:

• Interpretation: were the intended blocks and preservation rule selected? Did an ambiguous reference remain a question?
• Execution: did the exact operation preserve words, attribution and reading order, and return the specified optimum or conflict?
• Composition: did a later move or crop retain those relationships, or invalidate a dependency that needs fresh interpretation?
• Continuation: can the editor accept a tradeoff, change a priority or restore part of the work without reconstructing the rest?

The six-case calculation supplies the intended interpretation directly. It establishes the resulting operation and the input ambiguity of the restricted numeric selector; it does not establish that a trained model understands new editorial requests. That needs held-out documents, unfamiliar combinations and deliberately changed authority—not merely new paraphrases of the same examples. Both interface designs remain legitimate candidates for an actual comparison.

A compound “balance this spread” operation could use these components: interpret the emphasis, collect accepted caption/quotation relations, compute feasible layouts, then compare their effect on the argument. An uncertain association must remain uncertain at that join. A model trained to select a preservation rule has not thereby learned to judge a publication’s rhythm. Different learned decisions need different examples, while exact tool operations provide consequences that can be checked independently.

Let the vocabulary and the model develop together

DreamCoder offers a particularly relevant research precedent. It learns reusable program components and a neural policy that guides program search. Solutions yield new abstractions; the growing library supplies replayed and generated problems for training the policy. Earlier learned functions can become parts of later ones. Its evaluations examine the contribution of both library learning and neural guidance. DreamCoder, PLDI 2021

The starting conditions matter: an expressive base language and a corpus of related tasks are supplied. The authors identify learning richer types, idioms and domain expertise as continuing challenges. This is evidence for a joint-learning mechanism, not proof that a model can infer an entire profession from a few examples.

A professional system could make that loop deliberately inspectable. Repeated accepted edits might suggest a new compound operation. The expert could examine the common structure before it becomes part of the library. A new version would need new examples and tests: the same command name can acquire different consequences when its implementation or assumptions change.

It would be a mistake to promote every repeated sequence. Three edits might look similar while one preserves wording, another rewrites it and a third changes the evidence being shown. Compression is useful only if the new abstraction preserves the distinctions that matter. Sometimes the right discovery is that a supposed single operation should be split.

To test the proposal, we should compare systems rather than merely display a convincing result: generic operations versus a domain vocabulary, and a prompted model versus a model trained on that vocabulary. Hold out whole projects and unfamiliar combinations, not just paraphrases of training requests. Measure task completion, unintended effects, corrections and the time until a person can continue useful work. The result may show that a better tool matters more than more training—or that both are needed.

What could become possible to ask for?

The most interesting tools may arise where a person already recognizes a meaningful change but cannot yet ask software to perform it coherently:

• An editor could revise the emphasis of a publication across formats while retaining quotations, caption relationships and deliberate exceptions.
• A survey team could move from “show the observations” to “help us decide what to observe next,” with sampling assumptions and uncertainty carried through the decision.
• A workshop could turn a recurring installation method into a configurable part family, then teach a model to recognize which customer constraints call for which variation.

Each possibility changes who can act. A specialist can make a method available to others. A colleague can invoke it without reconstructing every low-level step. Both can inspect the result and discover where the method needs to grow.

The aim of SYSTEM1 is to streamline that path: from an idea about useful work, through its architecture, to the tools, data and custom models that make it a continuing capability. The broader research question comes first. What operational language would let people do valuable work they can already imagine—and what new work would become imaginable once they had it?