Yes I've noticed this as well. In the end the solution was to remove the keyboard in settings. Then remove the app after the update and then redownload and install. I did that a day or so ago with that update. Now after today's update the voice icon remained on the keyboard without all this hassle.