The first thing you'll want to do is to see if you already have an older version of the software installed anywhere. If the software was previously installed under /usr/local, and you install the new package under /usr, not only will you needlessly consume disk space, but the version that is run will depend on the setting of your PATH environment variable. A user may report that he can't use a certain feature in the new version, and it may take you a while to notice that his environment variable differs from yours, and that he's still running the old software. So, find the name of an executable that you expect will be installed. For example, if you're installing the binutils software, you will expect that the ld binary should be installed somewhere. Next, type the command:
to see where it is currently installed. If you see it in "/usr/bin/ld", then you'll probably want to use a prefix of "/usr", so that your new versions install over top of the old ones. If, on the other hand, it's in "/usr/local/bin/ld", you'll want a prefix of "/usr/local".
Sometimes a package installs only one or a few binaries. You may decide to install this into its own directory. For example, I install firefox into the prefix /usr/local/firefox, SBCL into the prefix /usr/local/sbcl, and the apache httpd into /usr/local/apache2. These get their own directories because, while they may install a very small number of executables, they come with a large set of ancillary files. Rather than installing over top of the old directory, I move the old directory to a new location, say "/usr/local/sbcl.old", and then install and test the new version. If the new version doesn't work properly, I can revert to the old one by deleting the new install and renaming the ".old" directory. Alternatively, I can compare the two installations, the previously working one against the new one, and see if there are any obvious differences that could account for problems.
Of course, you probably won't be able to type the command firefox and expect it to run if it's installed in /usr/local/firefox/bin/. You will either want to add that directory to the PATH variable, or, more conveniently, put a symbolic link to the appropriate executable from a directory that is in your PATH. This command:
puts the firefox executable into your PATH, piggy-backing on the /usr/X11/bin entry that is probably there already. Note, however, that if you re-install X11 (we'll get to that in another posting), you might destroy this symbolic link, and you'll have to re-create it then.
ln -s /usr/local/firefox/bin/firefox /usr/X11/bin/firefox
So, you really have a couple of choices. Put the program into a standard place, like /usr or /usr/local (and if upgrading try to install over top of the old version by using the same prefix that was used then), or installing the software in its own dedicated directory, like /usr/local/firefox or /usr/local/sbcl.
Now, when you set the prefix in an autoconf configure script, it also sets a number of derived values which can be separately overridden. Configuration files are, by default, put in <prefix>/etc, libraries in <prefix>/lib, headers in <prefix>/include, man pages in <prefix>/share/man (sometimes omitting the 'share' component), log files in <prefix>/var/log, and so on. The configure program lets you override these defaults separately, so that you can put configuration files into, say, /etc/http with the option "--sysconfdir=/etc/http", and so on. Think carefully about whether you want these additional directories to keep their defaults. You probably don't want your X-server log to be in /usr/X11/var/log, nobody will know where to look for it.