Yes, it is normal. The TCH uses the traction battery to assist when accelerating and to "bleed off" any extra energy stored from braking. Every time you slow down, the TCH uses regenerative braking to charge the traction battery. If you go down a long hill lightly braking, it is possible to "fill" the traction battery to the full level. Conversely, in slow moving congested traffic, it is possible to drain the battery down to one bar by driving a few miles in EV mode.

Don't worry about it, the TCH closely monitors the SOC of the traction battery. "Full" and "Empty" on the display are relative. The battery never charges over about 80% or discharges under about 40% of capacity. That is how Toyota gets such long life out of the battery.