joshtronic

in Command-line Interface #Vim / Neovim #Shell Script

How to tell if your shell is a subshell of Vim

If you're like me, you use Vim.

If you're like me, you use :sh[ell].

If you're like me, you forget that you used :sh and open Vim again.

I've been getting better about it, but every once in a while I end up with 2 or 3 nested Vim processes and subshells.

This tends to become apparent when I happen to open a file I already have open and Vim barks about there already being a swap file.

To help combat this situation, I've been running ps before running Vim again to see if Vim is already running in my current terminal and if so, I can simply exit and get back to Vim.

Problem there is that I end up running ps quite a bit and still forget to run it some of the time.

What if there was a way to show that I'm currently inside of a Vim subshell on my shell's prompt? That'd be spectacular!

But I wasn't sure how to accurately determine if I was in actually in a Vim subshell.

I've been researching this off and on for a month or so. Dabbled quite a bit with pstree and couldn't come up with anything that didn't seem a bit too fragile for day to day usage.

My recent bout of research led me to explore the environment variables that Vim sets at runtime. There are a few of them, $MYVIMRC and $VIMRUNTIME seemed like they could be the ticket.

Unfortunately, both variables provide mixed results.

$MYVIMRC is only present if you have a .vimrc file.

$VIMRUNTIME could be set in your environment and always be present.

With that, YMMV on which variable you would need to interrogate. For me, $VIMRUNTIME made the most sense as I've never set that value directly.

$MYVIMRC would have been sufficient as well as I always have a .vimrc file.

Now that I know what I needed to be checking, all I needed to do was wire it up to my prompt, similar to how I configured my minimalist git prompt:

zsh

vim_prompt() {
  if [ ! -z $VIMRUNTIME ]; then
    echo "vim ";
  fi
}

PS1='$(vim_prompt)# '

And for consistency with my bash prompt:

bash

vim_prompt() {
  if [ ! -z $VIMRUNTIME ]; then
    echo "vim ";
  fi
}

PS1='$(vim_prompt)$ '

Not much to it! You can configure the indicator however you'd like. For me, I just prefix my prompt with a little :sh (grey colon, green "sh").

You can check out the entirety of my prompt and other my other environment configuration and such over in my dotfiles!