The difference between the modes is that in user mode the machine's capabilities are limited compared to system mode. There are two types of limitations needed:
halt. Machines that use in and
out instructions will normally have these as
supervisor-mode instructions; machines that use memory-mapped IO will
normally have device registers in system space.
What I've described above is a very minimal set; it can be (and has been) expanded in nearly every direction you can think of. There have been machines that had more than two modes (VAX and Intel both have four levels, the intent being to have more than two levels of ``trustedness.''). Intel has a very elegant scheme allowing processes, on a process by process and IO-port by IO-port basis, to have access to IO on particular devices. And so forth.
Finally, the mode bit itself shouldn't be under the direct control of the user (if it were, you could just set the bit and circumvent everything). So, how do you change the mode bit? On most machines, the mechanism to do this is piggybacked onto the device interrupt mechanism. Remember, this (typically) looks something like this:
trap instruction
(aka syscall, software interrupt) which requests an interrupt from
software. Requesting a trap is the user's only way of setting the
mode bit - and when you do, you don't have control of where in the OS
your handler is located. So security is maintained. Incidentally,
one of the better ``pragmatic'' definitions of the kernel is that it
is the code you can only get to by way of interrupts or trap
instructions.
That said, there are two fundamental OS organizations: a system-call approach, and a message-passing approach (which might also be called a microkernel approach).
In a system-call approach, all OS functions are implemented directly in the kernel itself. The advantage of the system-call approach is efficiency.
In a message-passing approach, the only functions implemented directly in the kernel are message sending and receiving. Everything else is performed by trusted processes that execute in supervisor mode. The advantages of the message-passing approach are modularity and safety.
As you might expect, all real OS's are somewhere in between!