I am a little confused by your rational. If you are going on medicare and medicare is your primary insurance, you really need Part A and Part B. People who don’t take Part B normally do so because they still have an employer based plan that overlaps with Part B benefits. It sounds like you can skip or take very low cost other additional plans such as Part D and you may be eligible for a Part B discount with low income status.
I have Parts A,B,D and only see my endo twice a year, take very low cost Part D and buy my insulin in Canada to bring back to US.