Indeed. Instead of listing all non-editing modes in `evil-emacs-state-modes', a better setting would in many cases be to make the mode come up in Motion state, but with priority /over/ Motion state's bindings. The keymap order then becomes:
<compilation-mode bindings> <Motion state bindings>
The point is that we get all of compilation-mode's bindings without having to disable Evil's keymaps entirely. This is actually how Dired is set up, although it comes up in Normal state instead of Motion state.
Different from Dired mode, the compilation-mode has very few key bindings and that's the reason why I thought Motion State is the best in this case. And by re-binding "recompile" to "c" (was "g") we can still use the extremely hany "gg" command. My two cents.
All of this this could be automated with a function (with the exception of replacement bindings, which have to be chosen carefully). The function would take the mode's keymap as an argument and set it up like above, and maybe provide an ADD-HJKL option for protecting the "core motions".
So, are you suggesting adding some new mechanism for this? That will be even better I believe.