You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Gary dos Santos c194dcea8d 🎮 Added DoomWorld scraper 2 months ago
IJSRD Added IJSRD scraper 3 years ago
IJSSB 💩 Added IJSSB scraper 2 years ago
artic 🎨 Corrected readme 3 years ago
bensound 🎵 Added bensound scraper 3 years ago
betterttv 😃 Added Betterttv scraper 3 years ago
boorus Added function to filter path names to work in Windows. Reset the incrementer upon each search. 3 years ago
calibre-server 📚 streaming download added 3 years ago
dafont 🗛 Added dafont scraper 4 years ago
doomworld 🎮 Added DoomWorld scraper 2 months ago
iconmonstr 🔖 Fixed iconmonstr scraper 3 years ago
impawards 🎞️ Updated impaward scraper 2 years ago
khinsider 🎧 Added khinsider scraper 2 years ago
linfoxdomain 🎮 Added linfoxdomain scraper 3 years ago
memoryoftheworld 📚 memoryoftheworld automatically creates folders 3 years ago
open3dlab Updated Open3DLab scraper 6 months ago
oppetarkiv 📽️ Added argument to specify output for Öppet arkiv 2 years ago
riksdagen 🏛️ Changed dokid API and error check for speakers 3 months ago
shzm 📈 Top 50 songs by city 4 years ago
thecoverproject 🎮 Added thecoverproject scraper 4 years ago
theoatmeal 🥣 Added theoatmeal scraper 3 years ago
unicode-emoji-chart Added Unicode Emoji scraper 3 years ago
vulnhub 👩‍💻 Added vulnhub scraper 3 years ago
wad-archive 🕹Updated WAD-archive scraper 6 months ago
wallhaven 🖼️ Skips downloaded images 1 year ago
wallpaperflare 🖼️ Added Wallpaper Flare scraper 2 years ago
waset Updated waset scraper for new site 3 years ago
webtoons 💬 Added Webtoons scraper 2 years ago
xkcd Added xkcd scraper 4 years ago
zenpencils 📗 Added zenpencils scraper 3 years ago
LICENSE Initial commit 4 years ago
README.md 🆕 Updated README 2 years ago
requirements.txt requirements file 3 years ago

README.md

Scrapers

This is going to be a repository for random scrapers I've come to develop over the years. These scripts have been published in the intention of data preservation.

Description

All scrapers in this repository with a brief description

  • artic - Grabs art along with metadata from artic.edu
  • bensound - Grabs royalty music from bensound.com
  • betterttv - Grabs emotes from betterttv.tv
  • calibre-server - Grabs ebooks from a calibre-server
  • dafont - Grabs fonts uploaded to dafont.com
  • iconmonstr - Grabs icons in all formats from iconmonstr.com
  • ijsrd - Grabs papers published to ijsrd.com
  • ijssb - Grabs papers published to ijssb.com
  • impawards - Grabs all movie posters from impawards.com
  • linfoxdomain - Grabs all flash games from linfoxdomain.com
  • memoryoftheworld - Grabs all the books from memoryoftheworld.org
  • open3dlab - Grabs all assets and metadata from Open3DLab and Smutbase(NSFW)
  • oppetarkiv - Grabs programmes from oppetarkiv.se
  • riksdagen - Grabs debates and documents from riksdagen.se
  • shzm - Grabs the top 50 songs by city from shazam.com
  • thecoverproject - Grabs all the game posters from thecoverproject.net
  • theoatmeal - Grabs comics from theoatmeal.com
  • unicode-emoji-chart - Grabs emoji and metadata from unicode.org
  • vulnhub - Downloads items and all metadata on each item from vulnhub.com
  • wad-archive - Grabs WADs with metadata from wa-archive.com
  • wallhaven - Grabs all the images for a given search query from wallhaven.cc
  • wallpaperflare - Grabs all or some wallpapers with search from wallpaperflare.com
  • waset - Grabs papers published to waset.org
  • xkcd - Grabs comics posted to xkcd.com
  • zenpencil - Grabs all comics from zenpencils.com

Others

Scrapers in other repositories