Improve check for when to ignore mouse events

The browser tells us which element was under the cursor, so we don't
have to calculate it ourselves.