During an active event, 50% of the required XP to level up the Castle is cut off. This visually influences the progress bar.
Imagine you need 10 XP to level up your Castle while you currently have only 2.5 XP. The progress bar will be filled by 25%. Then, after the 50% XP Event launch, you need only 5 XP to level UP.
As you have 2.5 XP, your progress bar will be filled by 50%, not 25%, as before. After the end of the temporary Event, the progress bar returns to normal, so it looks like you lost some XP points.