It depends on what there is and what you are shooting. There is no one right answer. Contrary to popular opinion, photographers do not have an absolute right to shoot anything and everything in public. Just because there are 2,000 people shooting vacation shots at Yosemite Fall does not mean you can setup 5 strobes and pose 4 models while blocking traffic. The requirements for permits vary from location to location and the rules of what triggers the need for a permit varies. It does not have to be fair and the rules are often seemingly arbitrary. Too bad. If you don't like it, get elected to government and change it.
In general, if you blend in with what a tourist does, you probably don't need a permit to shoot. For example, it's common to see a tourist shooting a single person with a camera. What's not common is a tourist posing a model with lights, reflectors, and a makeup artist standing by. Fees for permits vary the the penalty for not having a permit when one is require is often very high. Enforcement is often arbitrary so just because you or 100 other people got away with it last time does not mean you'll get a free ride next time.
In general, all National Parks require permits. Most public beaches require permits and most cities require permits if you are going to obstruct the public in any way or use the images commercially.