I am an avid cyclist and triathlete. Competing since 2006. T1 for just over 18 years, pumping for just over 10 years, podding for just over 5.
Remember that any basal change you make will not really be "active" in your body for an hour or so. Meaning if you plan to exercise at 4pm and you change your basal at 4pm, you won't really have that basal as "active" until 5 pm. Well if you stop exercising by 5pm, you've totally missed your reduced basal and now you'll be chasing a high BG b/c the basal is not doing its job. This is perhaps what you are experiencing later in the evening?
SO, you need to make adjustments an hour or so prior to the exercise. Everyone is different, so saying to reduce your basal by "x %" for a certain activity is not necessarily going to work. However, a good starting point is maybe a 50% reduction for aerobic activity (which is what most people are likely doing). You may need a larger or smaller adjustment, but you won't know until you start somewhere.
If your exercise is for 1 hour, then perhaps make that adjustment for 1 hour. So exercising at 4pm you'd make your basal adjustment at 3pm, for a duration of 1 hour. At 4pm your basal profile goes back to normal as you start exercising...but remember that this insulin will not really be "on board" until after you finish your exercise. This should help prevent some of the high BG later.
Additionally, sometimes your muscles and liver realize that you are exercising (you're aerobic so you're burning muscle glycogen) and when you stop exercising, they--the muscles and liver--may not get the memo, so they continue releasing that glycogen for a short time...which causes a quick/dramatic rise in BG. You may be experiencing high BG later b/c of this issue as well.
Are you eating anything during your exercise? If not, then the 180 to 50 drop is probably a good indication that you need to be consuming something. It could be as little as some G2 gatorade, but you may need something more substantial (perhaps 30g of carbs for every hour of moderate activity?...that's just a starting point and may need adjusted on a case by case basis).
Lastly I would mention that you should WRITE IT DOWN. When you start making adjustments, look for patterns. Write down your last meal time and how many carbs, any active insulin still on board, your basal adjustments, how many grams of carbs you're consuming during exercise, what that exercise is, how intense that exercise is, the duration of your exercise, etc. Once you have a few weeks' worth of data, you can likely see that there is a pattern of "when I adjust by this much and I eat x amount of carbs, my numbers are more in the zone I want them to be". If you are still having highs/lows, then you would make adjustments from there.