Note
Human control in autonomous systems
Autonomy in a physical system should remain legible. People responsible for the system should be able to understand its current activity, applicable limits, and the controls available to them.
Permission as architecture
Approved spaces, routines, schedules, and connected features should be defined explicitly rather than inferred by the system after the fact.
Permission is not a settings screen. It is a design commitment that shapes what the system is willing to attempt.
Confirmation where consequence matters
Some actions warrant explicit approval before execution because the environment, activity, or downstream effect justifies human judgment.
Overusing confirmation degrades trust. Underusing it removes accountability. Calibration is part of the engineering task.
Interruption that people can rely on
Understandable ways to pause, stop, restrict, or redirect activity should exist in both physical and software forms.
Interruption should behave consistently even under adverse conditions. It should not depend on ideal connectivity or perfect environmental awareness.
Review that supports accountability
People should be able to understand what the system has been doing, why it made supported choices, and how to change those parameters going forward.
This is not surveillance. It is the documentation that makes intelligent systems maintainable and appropriate for use in shared environments.
Related notes
The difference between capability and reliability
Why a successful demonstration does not establish repeatable real-world performance.
Privacy in sensor-rich environments
The questions created when intelligent systems perceive and interpret physical spaces.
Evaluating embodied intelligence
Why testing must include uncertainty, recovery, repeated operation, and environmental variation.