Symmetry brings harmony.
Encountered this problem for a while. It bothers me a lot although it does nothing harm. Well...the solution is quite simple, but hard to find by myself: just uninstall "Android File Transfer" app!! Don't know why this app causes the symptom. I am glad that I don't have to keep moving around on the touch panel to make sure I re-gain the control of mouse movement while pluging anything into usb port.