Fashion is a complicated business involving long and varied supply chains of production, raw material, textile manufacture, clothing construction, shipping, retail, use and ultimately disposal of the garment. While Fisher’s assessment that fashion is the second largest polluter is likely impossible to know, what is certain is that the fashion carbon footprint is tremendous.
The vast majority of us shop at the giant fashion retailers, which have the biggest carbon footprint—and many of them specialize in fast fashion. Swedish giant H&M is the current largest clothing retailer in the world at $20.2 billion in sales (as of January 2015) followed by Zara, another fast fashion specialist.
The fashion industry by design is constantly changing with the seasons, but fast fashion can change weekly, summed up by a sign in H&M, “New stuff is coming in each and every day. So why not do the same.” It's not uncommon for shoppers to wear an item once or twice before throwing it away for next week’s style, aided by the poor quality of many of the clothes causing them to fall apart after several washes. [emphasis mine]
Related from The Economist:
Faster, cheaper fashion
"A rapidly rising, super-cheap Irish clothes retailer prepares to conquer America. Rivals should be fearful"
"For many shoppers, Primark has an irresistible offer: trendy clothes at astonishingly low prices. The result is a new and even faster kind of fast fashion, which encourages consumers to buy heaps of items, discard them after a few wears and then come back for another batch of new outfits."
(There's a new(ish) Primark near me; I went in out of curiosity, looked at a few shirts, and was unimpressed with their quality. I previously mentioned this article in "Learn from marketing to save us all", related to "break[ing] the grip of fossil fuel civilization's disposable lifestyle.")