I understand your point, but saying "why not just get all needed carbs from sugar" is a bit extreme. Potatoes are healthier than refined sugar on a macro and micro level.
The GI index was created for those suffering from diabetes. All of the foods measured in the GI are tested in a fasted state under carbs alone.
(Incorrect. They've tested many foods that have more than just carbs in them. Ice cream, pizza, most foods reallly...) When carbs are consumed with fat, fiber and protein ther GI is lowered considerably. Furthermore, some low GI foods act as high GI foods after ingestion. Milk (30) is lower on the GI than white bread (70), but both produce the same insulin response. Ice cream (36) is much lower than an apple (80). This doesn't mean that ice cream is healthier than an apple. GI numbers are inaccurate.
(GI numbers aren't inaccurate, your understanding of them is inaccurate.)
I'm not saying to avoid high GI foods altogether.
(I believe you mean low GI foods.) As with all facets of life balance is key. People who pay special attention to the GI of a food and think that it will help their waistline don't realize that their efforts are better spent elsewhere.
(They should consider the GI index amonst other things.)
How the Glycemic Index Lies to You