First it should be noted that I am not a legal expert especially concerning areas where laws do not pertain to me. As such this will be a more general answer to the question and covering this question on several topics. Nothing in here should be considered legal advice and I would like to note that for civilians I consider any weapon as an adjunct to empty handed skills and for the most part unnecessary. Remember that knowledge weighs nothing, lower levels of force will more often be what are needed, and (unless you're an amputee) you can't forget your body parts at home.
Know The Law
If it is not legal to carry a weapon in your country, stop right now. Carrying something which is purpose made as a weapon in a region where it is prohibited can lead to jail time, even if you use it to save your own life. In some regions you may be able to legally carry an item which could be pressed into service as a weapon like a folding pocket knife, heavy duty pen, keychain, etc. More on these improvised weapons later.
If it is legal to carry purpose made weapons your search becomes much easier. Again be aware of what you can legally carry as some states in the U.S. for example will allow you to carry a firearm but not a knife. Our Legal Knowledge area has a section with prohibited (non-firearm) weapons in Canada.
For both groups of people be aware of how you carry. A knife legally carried but slipped under the shirt could be considered a concealed weapon and land you in hot water.
This section is very simple. If you don't have training with a particular item be it a gun, knife, taser, pepper spray, or any other tool... DO NOT CARRY IT! You are only creating more danger to yourself.
You need to be able to draw, retain, and use anything you carry effectively. That means consistent and regular training. Training once or twice a year is insufficient for something you carry everyday for intended use in a high pressure, high stakes, andrenaline fuelled situation.
If you don't have training with your force multiplier of choice you're also going to risk not deploying it when appropriate. This might mean drawing too early, drawing too late, or freezing under pressure after you've already introduced the weapon. The rule of thumb here is to only carry things with which you have enough training to make effective decisions on deployment and where you're able to use them confidently and competently under pressure.
Use the Right Level of Force
This is an issue of some debate and it goes right along with training. Some people think you should carry the highest level of force available to you while others think that is overkill. The level of force necessary certainly depends on your region and should be proportionate to the threats you're likely to encounter.
If you can't carry a weapon you may be tempted to resort to carrying a tactical pen or any one of the self defence keychains on the market; DON'T. If you found it any prosecuting attorney can find it online. The fact that is is marketed for self defence makes it a purpose built weapon which means your plea of "but it's just a pen/flashlight/keychain" will fall on deaf ears. The words tactical, weapon, protection, strike, and self defence are just some of the words you should avoid on any product you actually want to use for those purposes. Unless you can carry something which would not be questioned, don't carry it at all. Even something like a big carabiner could draw attention by a prosecutor when they ask why you had nothing clipped on it and had it in your pocket.
This brings me to my next point...
The best weapon is not a weapon at all
If you carry something whose only use is a weapon you are likely to forget it, begrudge it's weight and bulge, and probably never draw it day to day. Something which has a function as a tool is something you are likely to always pack, have in the same place, draw more often, and which you have legitimate reason to carry.
I use my pocket knife, a Spyderco Paramilitary 2, at least once a day to open a package or even just remove loose threads from my uniform. Every time I pull it out that is practice drawing it. Because I use it so often it's always in the same place and I never forget to put it there. I would bet money that I can draw that tool faster under pressure than anyone who carries something like a dagger and almost never pulls it out because they carry it for self defence.
More detailed considerations:
armor - take a look at the v-42 commando knife. It was designed to be long enough to pierce a Russian greatcoat. Now take a look at a karambit. If you live in a cold and snowy place like I do you have to consider that much of the year people will be wearing heavy coats and/or heavy clothing. Impact tools, short/curved blades, and tasers are going to be a whole lot less effective in reaching their way through these layers.
environment - if you're likely to be shot at a knife isn't going to help you much unless you're close and even then it's not a great option. If you're using pepper spray in a region where it tends to be windy you could be getting a face full instead of your attacker(s). And again for cold regiond you will want something you can operate while wearing gloves.
weight & bulk - if it's big and heavy you won't want to carry it and will probably eventually get rid of it anyway. No one is going to carry those huge Cold Steel folders; not for long anyway.
carry & draw - where can you carry it, how easy is it to draw and deploy, will it be comfortable to carry, and will it be concealed while carrying.
ergonomics - is it comfortable for you personally to hold and operate
handedness - yes lefties some things are just not made for you