Protect against focus lost before drawing/uesrs already tokenized.

Fixes bug:5339625 odd behavior on orientation change in the "to field"

Also, added more test cases around adding/ removing more chip.
Change-Id: I1b24526b5be82835b5ec7c0345b7d73a31169a8f