Interactive canvas editing
Resolve an edit and its target, then apply it to the selected canvas context.
The heading needs more presence. The button needs room. You can see the change before you know its dimensions.
A useful design interface can start with that judgment. Point to the object, give a direction, watch it change. Then move it by hand, ask for another change or undo the last one. The canvas remains the place where the work happens.
The experience
Our prototype pairs a small design canvas with short instructions. The selected layer and current document accompany each request. A System1 decision model selects an allowed operation; the tool applies the edit and shows the changed state. The same document can be inspected throughout the sequence.
The public showcase replays tested examples; it makes no new model request when you select one. Its timings describe the recorded run.
Text stands in for speech here. The interesting part is what happens after the instruction: a particular object changes in a particular way, and the next request can start from that result. A future voice input would enter this same loop.
Try successive edits rather than treating each request as a fresh composition. Enlarge a heading, align it, change the button and undo a change. A good interaction has continuity: the person can recognize what was preserved as readily as what moved.
The model and the tool
“Make it bigger” is incomplete without an object. “Move it left” is incomplete without its current position. The model receives that context along with a finite action vocabulary. The tool supplies the executable meaning of each action and checks its bounds.
instruction + selected layer + document
↓
decision model
↓
permitted edit operation
↓
changed document + undo state
This division gives failures a location. Selecting the wrong layer is an interpretation failure. Moving the right layer outside the canvas is an execution failure. Applying an old instruction after the document has changed is a state-management failure. A useful test checks the resulting document, not only the words returned by the model.
It also makes precision available at the right moment. A person can begin with “wider,” inspect the result and then enter an exact width. Natural language and direct controls can share one object instead of competing to become the entire interface.
A useful research connection
Hutchins, Hollan and Norman’s Direct Manipulation Interfaces distinguishes the distance between a person’s intention and the operations a system offers from the feeling of acting on the represented objects themselves. That is a useful way to examine a language-controlled canvas: easier expression helps, but the result still needs to remain available for direct inspection and further action.
Horvitz’s work on mixed-initiative interfaces supplies another concrete idea. His LookOut system could narrow a calendar to a relevant week when it could not identify a precise appointment. Partial assistance could leave the person in a better place to finish the task.
For a design tool, that suggests useful intermediate actions. “Make this easier to scan” might first identify the relevant group and offer two spacing changes. The designer could compare them on the document before adopting either. That is an exploration beyond this prototype: the model helps form an editable proposal, while the canvas gives the proposal meaning.
Building it with System1
The project is the behavior and the tool together: which instructions matter, what document context they require, which edits are available, and how the result is used. System1 develops those definitions with examples and a callable API contract.
The current demonstration is a prototype of that interaction. Task-specific training and broader evaluation are further development work. Useful evaluation sequences would change the selected layer, combine relative and exact edits, reverse an operation and repeat a request after the document changes. The object being tested is a continuing piece of work.
A company could commission this capability around its own templates and vocabulary. A sign maker’s “fit the shop name” is a different task from an animator’s “hold the pose longer.” The same production process can develop a model and tool for either without making their professional judgments identical.
Where this could lead
Consider a packaging designer with six regional versions of one carton. “Give the ingredients more room” could propose coordinated edits across all six, show which layouts would overflow, and leave the designer to resolve the exceptions. Independent model decisions examine each layout; a shared constraint preserves the relationship between them. The interface becomes a place to direct a family of objects while still inspecting individual consequences.
A workshop planning its floor could use the same pattern differently. “Make room for two people to load this machine” could highlight the affected clearance area, propose a move and show the resulting paths. Geometry would enforce the physical constraints. The model would connect the operational request to the relevant objects. That possibility begins with knowledge of the work, not fluency in a drawing package.
These proposed interfaces share a specific architectural move: interpretation sits beside editable state, and each decision produces another state the person can use. A model becomes one of the ways a business works on its own material.