Vimpulse and vim-mode

[Vegard Øye]

On 2011-02-28 10:02 +0100, Frank Fischer wrote:

> Am Samstag, 26. Februar 2011 schrieb Vegard Øye:
>
>> One solution may be to define the repeat history as a
>> collection of keystrokes and buffer insertions, e.g.,
>>
>>     ([a b c] "inserted text" [d e])
>>
>> Most commands are remembered as keystrokes (vectors), but certain
>> ones, like `dabbrev-expand', are remembered as insertions (strings)
>> instead.
>
> I wonder if it is that easy. What kind of repeation is intended?
> Sometimes you want to repeat the inserted text, but another person
> wants to repeat the expansion context sensitive.

No, it is definitely not easy. :) But it might help to extend the
"inserted text"/"actual keystrokes" split to the changes themselves.
On one hand, there are "linear changes", which are changes for which
one doesn't need a keystrokes-based system at all (the insertion-based
approach of Vim would work equally well). Examples:

    * Insert two letters at (i.e., before) point
    * Insert three letters and delete the last two
    * Insert four letters, delete the last three, and insert five
    * Delete the last two letters

Since we may delete more than we insert, a mere string won't capture
the concept entirely; we'll need an offset as well. Thus, the changes
above may be represented as:

    * ("aa" . 0)
    * ("aaa" . 2), or just ("a" . 0)
    * ("aeeeee" . 0), which may be seen as the "sum" of
      ("aaaa" . 0), ("" . 3), and ("eeeee" . 0)
    * ("" . 2)

Now for some non-linear changes:

    * Insert "(" at point and ")" after point
    * Delete "(" before point and ")" after point

These cannot be represented like above (and, indeed, Vim's repeat
fails miserably when using the extension AutoClose.vim). Hence, for
such changes we must default to keystrokes. (Vim's repeat actually
fails even in trivial cases: try to repeat "i<Del><Del><Del>".
It is not pretty.)

> First using markers does not work in all cases, only if point is
> placed right after the inserted text. Other insertion packages may
> place point, e.g., somewhere in middle of the inserted text.
> Propably it's better to use after-change-functions.

If it is possible to reliably distinguish linear changes from
non-linear ones in `after-change-functions', we can choose the
appropriate representation accordingly. Any invocation of, e.g.,
`self-insert-command' will be represented as a linear change,
while anything that doesn't fit the linear mold is represented
as keystrokes.

> And sometimes it may be quite difficult to find out which
> key-sequence to drop - I thing of auto-completion packages where
> during completion the user executes several commands and only few of
> them (the final RET?) really inserts some text, and this completions
> are often triggered by a timer, not by an explicit command.

Provided the completion does work linearly -- i.e., it only changes
the text immediately before point -- the keystrokes shouldn't matter.

> I would prefer some good default rule which works in most cases
> (blacklists are okay for rare cases).

I agree; blacklists should be avoided if possible. For the rare,
intractable cases, though, we could add some variables for overriding
the default choice. These could then be adjusted in a `defadvice' of
the troublesome command. (We don't actually have to use advice, of
course; an internal data structure would work just as well.)

> Motions in vim-mode are represented by the vim:motion struct. This
> struct contains the two buffer positions along with the intended
> motion type (inclusive, exclusive, linewise, block). It is this
> data-structure that is passed to operators. So the single motion
> argument for operators in vim-mode is rather equivalent to the two
> parameters BEG END in vimpulse plus the type of the motion.

I see. We might have use of such a structure when passing values
around internally, but I think the commands themselves should do
things "out in the open" -- i.e., the first three arguments should be
(BEG END &optional TYPE), and maybe a fourth one for the register.
The define macros should let one specify further arguments, though.

> This can be done either implicit or explicit. Explicit means that
> the function body really creates the structure and returns it. This
> is usually the case for text-objects (they are just usual motions in
> vim-mode) or some special commands like , or ; which return motions
> of different types depending on the previous f F t T command.
> Implicit means that the function does not return an explicit
> vim:motion structure. In this case it is enough to move point.

Yeah, all of Vimpulse's motions and text objects are implicit. Text
objects are implemented as selection commands (since in Visual state,
they /are/ selection commands). I experimented with return values at
one point, but didn't find much use for it.

> I can think of defining operators with arguments
>   (BEG END &optional (type 'char) register)
> and an apropriate interactive specification. This would enable them
> to behave like normal region-operations but gives the additional
> possibilities to pass further important information. But then there is
> the problem how to pass the additional arguments interactively. But it
> sounds as if you have a good idea how to do this ;)

I can think of two approaches:

    1. The `with-register' command (bound to ") reads a register from
       the keyboard, reads an operator, reads a motion (if
       appropriate), runs the motion to get buffer positions, and
       passes the positions and the register to the operator.

    2. The current register is stored in a global variable which is
       reset in `post-command-hook'. The `with-register' command sets
       the variable, reads an operator from the keyboard, and executes
       the operator with `call-interactively'. The operator, as part
       of its interactive behavior, sets its REGISTER argument to the
       global variable.

Neither is very difficult to implement: the code for "converting"
motions to buffer positions can easily be reused in `with-register'.

>>> - What to do with non-vim commands and motions?
>>
>> Well, we will likely have one macro for defining operators, and
>> another for defining motions. Both can be set up to index all
>> defined operators/motions for later lookup, which will give us some
>> information to go on.
>
> As above, this would require a blacklist and have to keep the list
> up-to-date. And other packages may introduce new motions. So in all
> cases we need very good defaults so noone has to modify that list in
> almost all cases.

Yes. There is one case where I cannot see any other solution than a
blacklist, though: normalizing Emacs' region to the Visual selection.
We have to do this /before/ the next command is called (so we can't
inspect its behavior), and we don't want to do it if the next command
is a motion. So we can list either motions or non-motions, and
normalize or don't normalize based on that. I went with listing
motions because I figured the user is more likely to write a new
region command than a new motion -- and Vimpulse does, after all,
supply its own motions. If the documentation can convince the user to
always use the `define-motion' macro for defining new motions, there
shouldn't be much of a problem.

>>> - How to switch states and how to represent states (usually as
>>>   minor-modes, perhaps with some further meta-data like cursor,
>>>   info-message, ...)
>>
>> We'll have a macro for defining those too, of course. :)
>>
> Should we use standard minor-modes for this? I do this in vim-mode and
> it seems to be okay, but I wonder if there are any downsides.

I can't think of any. I think Viper solved this problem well enough.

> But if I understand correctly you just want a function which changes
> point and mark and toggle transient-mark-mode to behave like a
> text-object ... or to enable visual-mode. Sounds reasonable to me
> although I'm always afraid of compatibility issue between Emacs
> regions and visual-regions.

Nah, this problem is pretty much solved, too. Vimpulse enables
Transient Mark mode and activates the region when entering Visual
state (and it tries to restore everything to their old values when
exiting). This allows any region command to pick up on the selection,
including commands whose behavior is conditional on the activation of
the region, such as skeletons (defined with `define-skeleton').

>> I think the actual yank handlers should be specified by the type
>> system, so that `line' has one handler, `block' has another, and so
>> on. That way, we avoid hard-coding these associations into the
>> "yank" command.
>
> I prefer a type to be as simple as possible and to have as less
> additional functions and responsibilities as possible. The type
> should deal with its buffer positions. Not more. Anything else like
> handling of the visual-region or commands like yank/delete do not
> belong here.

Fair point. I agree that types should only specify a transformation
and the reverse transformation if possible, plus perhaps a function
for textual descriptions (for Viper-esque messages like
"Deleted 2 lines"). A definition of the `inclusive' type could be:

    (define-type inclusive
      :expand (lambda (beg end)
                (list beg (min (1+ end) (point-max))))
      :contract (lambda (beg end)
                  (list beg (max beg (1- end))))
      :describe (lambda (beg end)
                  (let ((width (- end beg)))
                    (format "%s characters" width))))

The words "expand" and "contract" refer to the fact that Vim's
"inclusive" selections are larger than Emacs' "exclusive" ones.
I'd prefer a more general pair of terms, though: maybe
"transform"/"inverse-transform", or "transform"/"inverse",
or "normalize"/"reverse", or something. Suggestions?

Oh, and you /can/ step through this thing with Edebug, which is
actually quite good at handling macros provided one tells it how
(with `declare'). But then there's cl.el. Vimpulse used cl.el when I
started out, but after various issues with `destructuring-bind' and
Edebug (among other considerations), I decided to revert to "vanilla"
Emacs Lisp. Would this be inconvenient to you? Are there some cl.el
macros which would be valuable when solving the kinds of problems
we're discussing here, or can we avoid this dependency?

> Other commands require additional character arguments like "r" or
> motions like "f". Of course those commands can read that character
> themselves. But I prefer passing the character as an argument
> because then this command can also be used non-interactively
> (useful, e.g., for commands like , and ;).

I agree. "f" should call `read-char' in its `interactive'
specification, while the body of the function should just concern
itself with finding CHAR in the buffer. This is very readable.

>>> The main problem is to cover all those funny special cases that
>>> arise (what happens at the end of the buffer, on empty
>>> lines, ...) - most bugs found in vim-mode are of this kind.
>>
>> This can be tested by executing a keyboard macro in a temporary
>> buffer.
>>
> Note that executing a keyboard macro makes testing certain aspects very
> difficult, namely those that behave differently when executed in a
> keyboard-macro or not. I think of the repeating system which must take
> care of keyboard-macros.

Yeah ... hm. Maybe we could invent our own variant of
`execute-kbd-macro' for testing purposes.

> I think errors should never be suppressed. The operator should just
> be canceled but the error should be shown. It is always dangerous to
> suppress errors (why does this not work? There is no error, right?)
>
> Note that there are other error-symbols in use like end-of-buffer and
> the like. I think we should use them whenever possible. But should we
> use special error-symbols for errors that occur in our code?

We could always create our own function for error reporting. For
example, less serious errors could be logged in a hidden buffer
(starting with a space), whence they're brought forth by calling a
special command. This could prove useful when helping users: "Please
do `M-x show-error-log' and post the output". The logging level is
the same, but the "noise level" is adjustable.

I don't have strong feelings about this. As long as I don't get
thrown into the debugger when I try to move past the end of the line,
I'm okay. :)

> Yes it is definitely a difference for commands to work on screen-lines
> or visual-lines. Note that deleting a visual line may have funny
> results because the other lines may change a little bit, too (long
> words deleted which caused line-breaks).

Yeah, I experimented a little bit with selecting screen lines in
Visual state, but called it off because of pasting issues. It might
work if one had a special yank-handler for such lines.

> Very good, texinfo allows for nice inline documentation via :help ;)

Good point. Perhaps :help could open our documentation, while :info
would go to the main Info index?

-- 
Vegard