There are several possibilities.
For many people, their BG will rise first thing in the morning. As they go from sleeping to moving around, their liver releases glycogen to provide fuel for the body. So working out early in the morning can be part of the reason you see a BG spike.
Part of the spike can be from intense workouts. When you put sufficient physical stress on the body, your body releases stress hormones (epinephrine (adrenaline), norepinephrine, and cortisol). This is used to help your body handle the stress. The release of the hormones increases the release of glucose by the liver. Your heart rate goes up, more oxygen flows to the tissues increases, and liver glycogen stores are released for fuel. You’ve heard of the body’s “fight or flight” response. That’s part of what can cause a BG spike - fuel to fight or flee. Your liver is releasing the fuel. But this generally only happens when you exercise as sufficient intensity.
Disconnecting from the pump you are reducing your body’s available insulin supply, so that contributes to it as well. When you say “disconnect”, are you actually physically removing the pump, or just turning it off? Are you able to leave the pump on and just reduce basal?
Since you don’t always see it, is it only with intense workouts? When you are very tired at the end?
What is the typical pattern of your crossfit? What type of exercises do you do? I assume a general easy warm-up, followed by various other activities? Is there any pattern to what you are doing?
How long is your workout, and how long are you disconnected?
Generally, easy to moderate workouts will cause a drop in BG. Usually it will happen early in the workout as your body goes from inactivity to activity. Once your body has been moving for a while, a lot of the fuel for the activity is provided by muscle glycogen, which does not affect blood sugar at all. That’s what makes muscle glycogen great for us diabetics. It lacks the glucose-6-phosphatase enzyme, so it is not able to pass glucose into the blood! But it takes a while for that energy system to be enacted, so you may see a drop early in the workout before it is engaged.