Grailed Scraper
"I wanted sweaters"
Tour of my virtual closet (p5js)
Grailed.com is a website where users trade designer and vintage apparel. Due to the sheer volume of listings, the website can be viewed as the largest archive of high fashion clothing in existence, curated by the internet.
This collection offers a wealth of data about the market value of certain brands, insight into current clothing trends, as well as a peek into the absurdity that is high fashion.
Unfortunately, Grailed does not offer a public API.
To tap into this unique collection of data, a few unsanctioned scrapers have been created that are able to retrieve meta-data, including price, list date, and brand name. However, there are currently no scrapers that download images.
In order to aggregate a large collection of outerwear clothing, I created Grailed-Scraper. There are a number of technologies that exist for web scraping such as Beautiful Soup, Scrapy, and Selenium. I ultimately settled on Selenium because Grailed delivers content through JavaScript and lazyloads their images, and Selenium allows for interaction with the DOM. The URL for every listing image includes parameters for width, height, and fit, so Grailed-Scraper uses this to download images at user specified dimensions.
For ease of use, I parameterized the search term, number of images, and image dimensions as arguments to the script.
Now, getting six thousand vintage sweaters with a dimension of 200px by 200px is as easy as:
python grailed_scraper.py "vintage sweater" --num 6000 --width 200 --height 200