Your system does not have to be in run level 5 to start the X Window System. If XFree86 is not running, you can start it at any time using startx.
startx has a much simpler startup sequence than the display managers, but it is basically very similar.
After starting XFree86 itself, startx runs /etc/X11/xinit/xinitrc. This script is very similar to Xsession, except that it will generally never start a desktop or window manager itself. It usually does not honor the user's $HOME/.xsession script, but it will honor $HOME/.Xclients. If the user does not have a .Xclients, then xinitrc will almost always run /etc/X11/xinit/Xclients, which will start a desktop or window manager.