This usually means the window at a theater or event where you pick up tickets that you reserved in advance.
Wikipedia sums the etymology nicely:
The word “call” is a shortened form of “call for”, which means “to come and get”, so “will call” literally means “(the customer) will call for (come and get) the goods.” In a linguistic process similar to initial-stress derived nominalization, the first syllable of the noun phrase is usually stressed (“will call”) rather than the second syllable in the verb phrase (“will call”).
Why I Looked It Up
I had always wondered – it seems like such a weird phrase for a noun: “the will call window.” I was waiting for my wife at a movie theater, and the lady at entrance kept asking everyone, “Do you have will call tickets?” I heard it enough times that I wanted to know the origin.