Topic: E621 tag based downloader

Posted under e621 Tools and Applications

I made an e621 downloader program that uses java to download based on tags. Allows for up to 6 tags, and has a few built in presets(order by score, nsfw only, sfw only). If the tags aren't working for you make sure you have enabled checked by them.

note!
the downloader can't download .swf files, and will skip over them, but it retries once (in case it's the servers causing the problem), so expect a 1 second delay for every swf file if you don't search for images without the tag type:swf

NO LONGER WORKING DUE TO E621 UPDATE, SORRY!

Updated by onewolf

I get the same error as the above.

Updated by anonymous

Skufer said:
Not working for me, I get:

page url: https://e621.net/post/index/<NUMBER>/%20<TAG>
Error: Server returned HTTP response code: 403 for URL: https://e621.net/post/index/<NUMBER>/%20<TAG>
got page, parsing
finished getting page links
we finished the page and still have 10 images left

<NUMBER> increments by "1" every time and <TAG> is the tag I was searching for. Any ideas?

sorry, just checked the forums for the first time in a while, i'll try and get it fixed

Updated by anonymous

jonjetjon said:
sorry, just checked the forums for the first time in a while, i'll try and get it fixed

It's returning a 403 error because e621 now blocks connection from crawler bots like this one(for a fair reason), sorry guys, but it looks like this is a dead thread.

Updated by anonymous

jonjetjon said:
It's returning a 403 error because e621 now blocks connection from crawler bots like this one(for a fair reason), sorry guys, but it looks like this is a dead thread.

They might let this in, I mean, how else is the unofficial Android app still working?

Updated by anonymous

TonyCoon

Former Staff

Try adding a user agent string to the requests.

Updated by anonymous

TonyLemur said:
Try adding a user agent string to the requests.

the request already had a user agent string, if you were wondering the current source code (in java) for grabbing the pages is:

org.jsoup.nodes.Document doc = null;
try{
doc = Jsoup.connect(url)
.data("query", "Java")
.userAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36")
.cookie("auth", "token")
.timeout(3000)
.get();
}
catch (Exception e)
{

}
return doc;

and is returning:

Error: Server returned HTTP response code: 403 for URL: [url here]

Updated by anonymous

i kinda made this app for personal use(kinda why it looks so shitty and is very un-user-friendly) and when it stopped working i just left it, but if other people actually care enough i'll see if i can get it working again

Updated by anonymous

well I personally would LOVE a tag-based downloader. i've never used one before but it would save me a lot of time.

question: i assume that you can limit it from downloading everything with that tag based on criteria, can you limit by date or by number of pages or something?

also, can it check for existing filenames to prevent duplicates?

also, you mentioned that it skips flash, can it report the files that it skipped?

and thanks for letting people use your softs, super cool and i know how time consuming writing stuff can be o.o

Updated by anonymous

  • 1