Vimpulse and vim-mode
frank.fischer at mathematik.tu-chemnitz.de
Mon Feb 28 10:02:15 CET 2011
Am Samstag, 26. Februar 2011 schrieb Vegard Øye:
> On 2011-02-25 22:20 +0100, Frank Fischer wrote:
> > Gitorious should be okay.
> Then Gitorious it is! :)
> A big benefit of recording the keys is that it removes the need for
> repeat code in the commands themselves. For example, I have written
> some custom insertion commands for dealing with Lisp code (e.g.,
> "Insert after S-exp"); if I want them to repeat correctly under
> Viper, I have to edit the repeat history from within the commands,
> which is /extremely/ cumbersome. By contrast, a repeat system which
> monitors keystrokes and the current state would simply take note that
> I have entered Insert state, and update the repeat history
> As you point out, some keystrokes are troublesome: "M-/", which calls
> `dabbrev-expand', may expand to different words in different places
> in the buffer. One solution may be to define the repeat history as a
> collection of keystrokes and buffer insertions, e.g.,
> ([a b c] "inserted text" [d e])
> Most commands are remembered as keystrokes (vectors), but certain
> ones, like `dabbrev-expand', are remembered as insertions (strings)
> instead. The repeat system would use a "blacklist" of unsafe commands
> like `dabbrev-expand'; if the current command is listed, the system
> creates a marker at point (in `pre-command-hook'), lets the command
> execute, and then (in `post-command-hook') extracts the text between
> point and the marker, which goes into the repeat history like above.
I wonder if it is that easy. What kind of repeation is intended?
Sometimes you want to repeat the inserted text, but another person
wants to repeat the expansion context sensitive. I really have to think
about a good compromise (of course dabbrev-expand may yield different
results - but are they intendend? does this happend regularly or is it
just a rare case?).
Some further comments. First using markers does not work in all cases,
only if point is placed right after the inserted text. Other insertion
packages may place point, e.g., somewhere in middle of the inserted
text. Propably it's better to use after-change-functions. Second in
general I do not like blacklists. The reason is that there are many
Emacs packages out there each may define its own commands. Furthermore
each new Emacs version could introduce new commands and we have to keep
the blacklist up-to-date. Of course, each user can modify this list and
adjust it to the packages she uses. But I think this may be cumbersome
for the average user. And sometimes it may be quite difficult to find
out which key-sequence to drop - I thing of auto-completion packages
where during completion the user executes several commands and only few
of them (the final RET?) really inserts some text, and this completions
are often triggered by a timer, not by an explicit command.
I would prefer some good default rule which works in most cases
(blacklists are okay for rare cases). This does not mean I have a good
solution for these problems just a few things to consider (and I often
run into the auto-completion problem in programming, so this is not a
> > - undo, vim usually undos everything done in insert-mode at once
> > while emacs undos insertion in steps.
> This is not that difficult in itself, but becomes more complex when
> loading a custom undo package like undo-tree.el. (I would really like
> to use undo-tree.el in place of redo.el; it features a graphical
> interface which totally outclasses Vim's :undolist.)
I never used :undolist so I can't say anything useful to it. For me redo
has always been okay, so this is up to you ;)
> There is a case for making Vim-like "bulk undoing" toggleable:
> Emacs' fine-grained undoing can be valuable in some circumstances.
> The default should be Vim-like behavior, of course.
> > - catching of commands and motions. In each state the execution of
> > commands and motions has different effects and therefore some
> > special function has to be executed in each mode to deal with
> > commands and motions. In general I see two possibilities how this
> > special function can be executed. Either one hooks into pre-
> > and/or post-command hooks and checks whether the command/motion
> > about to be executed is a Vim operator and in this case do whatever
> > is required according to the current state. Or each command and
> > motion is defined in a special way in order to call the state
> > specific handler when it is executed. vim-mode uses the latter one
> > because I try to avoid pre- and post-command hooks if possible.
> Ah, the relationship between operators and motions. This is an area
> where Vimpulse and vim-mode differ somewhat. In vim-mode, as I
> understand it, motions are passed to operators by parameter (or the
> operator reads a motion from the keyboard when called interactively).
> The operator executes the received motion to determine what text to
> act on. The resulting pair of buffer positions is used internally.
That's wrong. Vim-mode almost behaves like what described below. Perhaps
it's time to tell you how the things work in vim-mode (and I think this
is not too bad, but certainly can be improved).
Motions in vim-mode are represented by the vim:motion struct. This
struct contains the two buffer positions along with the intended motion
type (inclusive, exclusive, linewise, block). It is this data-structure
that is passed to operators. So the single motion argument for
operators in vim-mode is rather equivalent to the two parameters BEG
END in vimpulse plus the type of the motion. Usually operators use the
motion type to behave differently (but this is not required). If we had
CLOS-like methods one would propably dispatch on the motion type.
A motion in vim-mode is function that returns a vim:motion structure.
This can be done either implicit or explicit. Explicit means that the
function body really creates the structure and returns it. This is
usually the case for text-objects (they are just usual motions in
vim-mode) or some special commands like , or ; which return motions of
different types depending on the previous f F t T command. Implicit
means that the function does not return an explicit vim:motion
structure. In this case it is enough to move point. Then the magic
within vim:defmotion ensures that a vim:motion structure is
automatically created depending on the default type of the motion and
the movement of point. Implicit is therefore enough for almost all
simple motions what makes them one-liners. Note that from the caller's
point of view it makes no different to call an implicit or explicit
returning motion - the caller always receives the vim:motion structure.
When executing an operator the operator is not executed immediately.
Instead the execution of its body is postponed until a motion has been
selected. When the motion has been selected, the handler of
operator-pending-mode first executes the motion to create the
vim:motion structure and then passes this structure to the operator
which is executed afterwards. In visual-mode the vim:motion structure
is not created by a motion but w.r.t. to the current visual selection.
All in all, the representation of a motion is very simple, just three
values, BEG END and TYPE.
I decided to use keyword arguments because in vim-mode commands and
motions may take a different set of arguments. E.g., a command may take
- additional argument (char, text, file, buffer, ...) whereas the
latter are only usefull for ex-commands
- force (only for ex)
Also note that in vim-mode each command can be bound in ex-mode AND
normal-mode/visual-mode. In my philosophy ex-mode is just another mode
with own keybindings, although they are read in a different way.
> In Vimpulse, the buffer positions /are/ the parameters (BEG and END).
> Operators work like region commands (i.e., (interactive "r")) in that
> they either receive the positions from the caller, or figure them out
> for themselves when called interactively. (Vimpulse also possesses
> some facilities for "passing a motion to an operator", which executes
> the specified motion and feeds its range to the operator. This is
> only used by the repeat system, though, and it would be totally
> superfluous if a keystroke-based system was in place.)
I really like the idea of making operators as close to usual emacs
commands as possible. This is certainly possible for character motions,
but how do you tell which kind of motion the region belongs to? Call
different commands depending on the type? Modify point and mark? The
latter is possible for character and linewise motions, but what about
block? Furthermore modifying point and mark may be problematic. Several
commands have rules where to place point after execution depending on
the original position of point (before modification to respect the
I can think of defining operators with arguments
(BEG END &optional (type 'char) register)
and an apropriate interactive specification. This would enable them
to behave like normal region-operations but gives the additional
possibilities to pass further important information. But then there is
the problem how to pass the additional arguments interactively. But it
sounds as if you have a good idea how to do this ;)
> For the sake of reusability, I recommend defining operator commands
> in the same way as region commands: with two parameters, BEG and END,
> and an `interactive' specification for determining their values by
> reading a motion from the keyboard.
> > - What to do with non-vim commands and motions? There should be a
> > simple rule so they work with vim in most cases as expected, but
> > if no meta-information is provided a non-vim command may always be
> > problematic (e.g., in operator-pending mode).
> Well, we will likely have one macro for defining operators, and
> another for defining motions. Both can be set up to index all defined
> operators/motions for later lookup, which will give us some
> information to go on. By the way, I also think that all of Emacs'
> standard movement commands should be regarded as motions.
As above, this would require a blacklist and have to keep the list
up-to-date. And other packages may introduce new motions. So in all
cases we need very good defaults so noone has to modify that list in
almost all cases.
> We thus get a large list of commands which move the cursor, and can
> restrict Operator-Pending state to those commands. Alternatively, we
> can maintain a "blacklist" of Emacs commands which should /not/ be
> executed. In any case, I don't think it's overly problematic if a
> nonsensical command should get called; any damage will be undoable,
> after all. It's probably better to err on the side of allowing too
> many commands than too few.
> > - What to do with commands that change the current buffer? This is
> > problematic if some buffer-local variables are involved.
> Right, which means that the repeat system cannot be buffer-local.
> I think it would just be confusing if it was, anyway.
Of course it is not, but there may be other buffer-local variables. In
vim-mode a tackle this problem by saving the new-buffer when a command
returns, switching back to the original buffer, do the necessary work
and switch to new buffer at the end. This seems to work quite well but
can certainly be improved.
> > - How to switch states and how to represent states (usually as
> > minor-modes, perhaps with some further meta-data like cursor,
> > info-message, ...)
> We'll have a macro for defining those too, of course. :)
> I suggest that each state's toggle function be a command; e.g., in
> Vimpulse's development code, vi state defines the command
> `vimpulse-vi-state' for entering that state, emacs state defines
> `vimpulse-emacs-state', and so on. The command also has an optional
> argument which, if negative, disables the state; this is used
> internally when switching from one state to another.
Should we use standard minor-modes for this? I do this in vim-mode and
it seems to be okay, but I wonder if there are any downsides.
> > - What is a motion? There are a few fundamental differences between
> > buffer-coordinates of Vim and Emacs. Vim known line/column, Emacs
> > knows just the offset. The choice how to represent coordinates
> > can be crucial. And of course there is that funny newline
> > character, which is usually invisible in Vim except for empty
> > lines. Because a motion is a important concept for many parts
> > including operators and visual-mode.
> I think all of Emacs' regular movement commands should be valid
> motions. Furthermore, all of Emacs' selection commands should be
> valid text objects. In Vimpulse, "y C-x h" and "C-x h y" are equally
> valid ways of copying the whole buffer, and "d M-e" works just as
> well for deleting a sentence as does "d)".
Hm I think d M-e is okay besides the fact that a sentence is slightly
different in Emacs and Vim (I think), but never mind. The first example
seems to be very special case. But if I understand correctly you just
want a function which changes point and mark and toggle
transient-mark-mode to behave like a text-object ... or to enable
visual-mode. Sounds reasonable to me although I'm always afraid of
compatibility issue between Emacs regions and visual-regions. Is it
easy to detect that a certain command behaves like this or is it
another blacklist variant? ;)
> Given the above, the only thing which separates Emacs' movement
> commands from the other motions is their lack of a /type/. For
> example, the motion "j" has a type of `line', which means that "dj"
> deletes two whole lines. What a type is, then, is a way to transform
> two buffer positions to two other positions -- in this case, to two
> whole lines.
> > Probably a motion is represented by some abstract data
> > type and this data-type is an important part of the interface a
> > user must understand in order to write new motions and commands.
> > Therefore it should be well-defined from the beginning because it
> > cannot be changed easily afterwards.
> I don't think it needs to be very abstract. I would rather try to
> keep things from getting too abstract, actually. One of the design
> rules I laid down in Vimpulse's new "TODO" list was that it should
> require as few packages as possible, and not invent new stuff unless
> it's absolutely necessary.
I think we both have the same feeling of it (see above). In fact,
vim:motion in vim-mode comes with a couple of functions which do
exactly this (returning appropriate buffer positions w.r.t. the motion
> The way I understand motions, they are just regular movement command
> plus a type. The type can be stored as a symbol property, e.g.,
> (put 'motion 'type 'line). The type itself, of course, must also be
> defined; for this, a minimal system may be warranted, since both
> Operator-Pending state and Visual state use types. I've pushed a
> currently unused "type system" to "development":
> Now, it's a bit more involved than it has to be; the routines for
> reselecting (e.g., two lines anywhere in the buffer) are unnecessary
> given a keystroke-based repeat system. The advantages of a systematic
> approach is that it can be reused by Operator-Pending state and
> Visual state, making the overall code simpler. The only heavy lifting
> Visual state ends up doing (once initialized) is highlight the
> different types, which is rather trivial (with the exception of
> `block' ...).
> > In fact, there are only two (or three) difficult commands: yank,
> > delete and paste, but I hope the implementation in vim-mode is
> > quite good know (it uses yank-handlers for all three kinds of
> > yanking and also provides "yank-pop" stuff).
> Yes, this is exactly the right way to do it. I think the actual yank
> handlers should be specified by the type system, so that `line' has
> one handler, `block' has another, and so on. That way, we avoid
> hard-coding these associations into the "yank" command.
At this point I completely disaggree. I don't like the idea of coding
behaviours of yank/delete and so on into the type system for two
reasons. First of all, yank/delete should not be different from any
other command. A special handling of them in the type system seems to
be confusing to me. The other reason is that there may be even more
commands that have different behaviour depending on the motion type,
not just yank/delete, even commands that we cannot think of because we
I prefer a type to be as simple as possible and to have as less
additional functions and responsibilities as possible. The type should
deal with its buffer positions. Not more. Anything else like handling
of the visual-region or commands like yank/delete do not belong here.
I think the main difference here is that you want to stick with the
standard interface of Emacs for operators (only BEG END) while I want
to provide additional information (especially motion type). So in my
world it would be the responsibility of the command to behave
differently on different motion types. Of course then your commands
need to be sensitive to the motion types - but you have to deal with
this sensitivity anyway, nothing prevents you from that. The question
Of course it would be nice if operators also behave like usual Emacs
commands when used in such a context. This was the reason of my
suggestion above (BEG END &optional type ...).
In fact, what I would need is a dispatch on the motion type, i.e., (yank
beg end 'char) calls another function than (yank beg end 'line). I do
not know of any such functionality in Emacs so propably it ends up with
manual dispatch (case type (line (yank-line ...)))) and so on. I do not
think this is too bad and certainly not more complicate than you
suggestion. But I think it provides a better separation of code.
Note that it is okay for me to have certain functions with update visual
region according to a motion type. But since visual-mode is the only
place where this is required it should go there. Of course in a
> > What could be useful are frameworks or perhaps only some functions
> > which help defining more involved motions like text-objects. We
> > should provide helper-functions for typical text-objects like
> > parentheses, blocks, whatever. vim-mode (and vimpulse, too)
> > contains a few of those functions which are again independent.
> Agreed; we should have a macro for defining text objects, too.
> To recap, we now have the following define macros:
> * `define-state'
> * `define-type'
> * `define-motion'
> * `define-text-object'
> * `define-operator'
> Do we need more than these?
Well there are also other commands that are not operators. E.g. paste
requires at least two arguments, the count and the register. Other
commands require additional character arguments like "r" or motions
like "f". Of course those commands can read that character themselves.
But I prefer passing the character as an argument because then this
command can also be used non-interactively (useful, e.g., for commands
like , and ;).
> >> http://gitorious.org/vimpulse/vimpulse/blobs/development/vimpulse-
> > I read your comments and they sound reasonable. Adding auxiliary
> > keymaps should not be that difficult. As I remember correctly,
> > local keymaps were the biggest problem in vim-mode, but major and
> > minor-mode specific should be relatively easy. Of course this is
> > very important and should be done as one of the first things.
> Actually, the current code already implements auxiliary keymaps,
> although I've not tested it thoroughly.
> > Note that vim-mode arranges keymaps in another hierarchy, too,
> > which allows to enable only few keybindings in certain major modes,
> > e.g., only movement and window-commands in info-mode (somehow like
> > viper-on-more-modes I think). The difference is that many
> > standard-keybindings are not available in this case, e.g.,
> > insert-mode.
> Yes, this is very sensible. Motions and text objects can be bound in
> their own keymap (or state), and that keymap is inherited by vi
> state, Visual state, and so on. Modes which do not work well in vi
> state, can come up in the motion state instead. Any shortcomings can
> be alleviated on a per-mode basis with mode-specific state bindings.
> >> Regarding test frameworks, I hear Christian Ohler's ert.el is now
> >> included with Emacs trunk, so we could go with that:
> > I never used any of those libraries but we should try. The main
> > problem is to cover all those funny special cases that arise (what
> > happens at the end of the buffer, on empty lines, ...) - most bugs
> > found in vim-mode are of this kind.
> This can be tested by executing a keyboard macro in a temporary
> buffer. A lot of Vimpulse's early tests do this, e.g.,
> (deftest test-change-undo
> "Change a word and undo."
> (execute-kbd-macro "wcwfoo^[u")
> (buffer-substring 1 51)
> ";; This buffer is for notes you don't want to save"))
Note that executing a keyboard macro makes testing certain aspects very
difficult, namely those that behave differently when executed in a
keyboard-macro or not. I think of the repeating system which must take
care of keyboard-macros.
> > This is another point which is currently very bad in vim-mode. How
> > to deal with errors, should there be a unique signalling scheme,
> > where to handle them, ...
> In "indirect" cases, like when executing a motion to get a text
> range, I think any errors should be suppressed. When executing the
> motions by themselves, however, we should signal any problems with
I think errors should never be suppressed. The operator should just be
canceled but the error should be shown. It is always dangerous to
suppress errors (why does this not work? There is no error, right?)
Note that there are other error-symbols in use like end-of-buffer and
the like. I think we should use them whenever possible. But should we
use special error-symbols for errors that occur in our code?
> > Btw, Emacs 23 contains visual-line-mode which is quite useful. But
> > most commands work on buffer lines not on screen lines. Should we
> > try to support visual-lines, too (optionally)? If yes this has to
> > be considered for motions and commands, especially for
> > operator-stuff, too, and we should think of it from the beginning,
> > because many commands have to be implemented differently for
> > visual-lines.
> Yes, we should definitely support screen lines; I have "j" and "k"
> bound to `next-line' and `previous-line' because they work better in
> visual-line-mode. In Vim, one uses "gj" and "gk" to move by screen
> lines. I think this is very inefficient and would prefer an option
> for the regular commands. It may of course be disabled by default.
Yes it is definitely a difference for commands to work on screen-lines
or visual-lines. Note that deleting a visual line may have funny
results because the other lines may change a little bit, too (long
words deleted which caused line-breaks).
Furthermore block-motions do not really make sense for visual lines. I
would definitely not use visual-line movements as defaults but document
very well how it can be configured, because dealing with visual lines
(as a user) may be very inconsistend.
> > Another thing somehow related to tests is documentation. We should
> > not only write developer-documentation but also user-documentation
> > right from the start. This should contain instructions for all
> > important concepts and also examples for new motions and commands
> > of all kind. Where should we put this documentation? EmacsWiki?
> > Hosting site? Perhaps as tex-info?
> Texinfo, definitely. I've had my share of arbitrary wiki syntax and
> never-ending comment blocks. A text file under version control is
> wiki-like enough for me.
Very good, texinfo allows for nice inline documentation via :help ;)
> > Hm, I think a new cool name would be good. I prefer short names or
> > at least names that allow short prefixes for function names ;)
> For the sake of brainstorming, I've grepped through /usr/share/dict
> for anything containing "vi". There isn't much, unfortunately, so
> I'll start with some acronyms comprising three letters:
> * avi, evi (augmented/extensible vi)
> * via, vie (vi augmented/extended)
> * vil (vi layer)
> * xvi (vi plus a cool letter)
> * yvi (yet a(nother) vi)
> Things get slightly more descriptive when we allow four letters:
> * evil (extensible vi layer; this almost works too well)
> * nuvi (new (unlimited) vi)
> * vibe (vi is beautiful?)
> * vici (veni, vidi ...)
> * vimu (vi(m) emulation; vim unlimited; vi made usable)
> * vini (vi new and improved)
> * viva (viva la vi!)
> * yavi (yet another vi)
"evil" really gave me a smile ;)
I thought of something like "VeM" but have no good idea for its
interpretation. "vimu" sounds good, too.
More information about the implementations-list