Topic: Subtract Blacklist from Tags

Posted under Site Bug Reports & Feature Requests

I want to know the real number of posts I will be seeing with the blacklists enabled. There have been countless times I have searched something and there were way fewer results than what was advertised.

Tl;dr, Even if we changed that, it'd both dramatically degrade the user experience & destroy our server's performance.

Because of the aforementioned client-side blacklist, in order to do this, we'd have to do one of the following:

  • Modify the search query users submit to automatically exclude results that match your blacklist behind the scenes
    • We limit the number of terms you can use in a search (currently to 40 or less) to prevent the server from being overwhelmed, but we don't limit the number of terms in the blacklist, both overall (as long as it's less than or equal to 150,000 characters) & in a single line*. If we added your blacklist to your query, one of a few things would happen:
      • We'd have to explicitly limit the number of overall terms added to your blacklist (very bad)
      • We'd have to only add as much of your blacklist to the query that would fit in the remaining space in your query (defeating the purpose)
      • People with huge blacklists can make functionally infinitely large queries, completely destroying the server's performance (worst case scenario)
    • This also isn't reactive; it cannot dynamically adjust to what filters in your blacklist are currently enabled or disabled
      • This means we'd either have to
  • Make an additional modified search query for every user-submitted query that specifically excludes results that match your blacklist used solely to retrieve a more accurate number of results
    • This would, at best, double the load on the server solely to retrieve a more accurate number of results
    • Everything said about the first scenario applies to this second query.

Additionally, the "number of results" is already an estimate, due to complicated optimization stuff too dense for even me to try explaining. Going through the programming & computational work to improve that estimate specifically for blacklists without addressing the underlying inaccuracy isn't really worth it.

* This is something we can do specifically because the blacklist is processed client-side (instead of server-side), & only on one page (with at most 320 posts on it) at a time; because each client's browser handles filtering these posts, we don't have to handle the strain of filtering it on the server, & because of the reduced number of results the client has to filter, it's not that demanding on their machine either.

Original page: https://e621.net/forum_topics/60391