Can I use in file name?

When I try to save a file, if I use special characters in the name (such as a colon, question mark, or exclamation point) in Windows I get a message saying that the file name is invalid.

Why is that?