Topic: How do you search decriptions and notes for multiple words

Posted under Tag/Wiki Projects and Questions

I've been trying to search for phrases in a description, but have been unsuccessful. It is possible to search for a single word by using description:, but using underscores only returns underscores, not spaces. You can search for multiple unrelated words by using multiple description tags, but nothing for phrases.

lafcadio said:
It's not possible.

Well, if you really want to, it is possible. Just not through the web interface.
You'd have to download the posts dump from https://e621.net/db_export/, which contains all the metadata of every post, including descriptions. From there you can load it with a CSV library and search it.

It's way too large to do with Excel, unfortunately. It's the largest text file I've ever seen, at 3.13GB.

scth said:
Well, if you really want to, it is possible. Just not through the web interface.
You'd have to download the posts dump from https://e621.net/db_export/, which contains all the metadata of every post, including descriptions. From there you can load it with a CSV library and search it.

It's way too large to do with Excel, unfortunately. It's the largest text file I've ever seen, at 3.13GB.

You can also use a trick with something like grep, where you filter out lines that contain word A into a second file, then find all the lines in the resulting file that also have word B. Assuming the CSV does the sane thing and has new lines for end of a row... The nice thing is that this can run on very little RAM, and is pretty much linear in time complexity, as well.

alphamule said:
You can also use a trick with something like grep, where you filter out lines that contain word A into a second file, then find all the lines in the resulting file that also have word B. Assuming the CSV does the sane thing and has new lines for end of a row... The nice thing is that this can run on very little RAM, and is pretty much linear in time complexity, as well.

Unfortunately, when searching through descriptions, newlines are a real possibility.
Haven't tried it, but topic #37429 looks good; it only goes one line at a time and deals with stuff like newlines inside a field. Of course, requires some python knowledge to use.

There's also (not making a complete CSV) partial parsing at binary level, if you have two different kinds of new lines, or double-double quotes. Those should be sanitized to something like &quot or the like. I don't know of any specific good CSV sanitizers.

spongebob69420 said:
I've been trying to search for phrases in a description, but have been unsuccessful. It is possible to search for a single word by using description:, but using underscores only returns underscores, not spaces. You can search for multiple unrelated words by using multiple description tags, but nothing for phrases.

I wish, I mean Furaffinity has it!

  • 1