Generally, men’s clothing. In British English, it can mean “small wares,” usually things like buttons, ribbons, zips, handkerchiefs, etc. – things used in sewing.
However, in most commercial usage, it refers to mens’ clothing stores or people who sell men’s clothing (a “haberdasher”).
I have seen some usage implying that “haberdasher” means a hat seller, specifically, but I can’t find any official definition of that (other than that a hat is a common article of men’s clothing). The female version of this usage is “milliner.”
I’ve also seen some usage of the word to mean poor or boorish behavior, supposedly stemming from the historical behavior of traveling clothes salesmen.