Topic: Find user's most favourited tags through API

Posted under e621 Tools and Applications

Hello everyone!
Is it possible to get a user's most favourited tags through the api?
I know it is possible to get a user's favourites data with https://e621.net/favorites.json?search[user_id]=USERNAME, do you think there is a way to count the times a tag appears in a user's favourites?

Thanks in advance!

There isn't an API endpoint for it to my knowledge. Of course if you process the json returned by the above link, it is possible.
Here is a quick experiment, which will work on Linux providing that a) you have jq installed, b) you have downloaded the favorites data into /tmp/favorites.json.

jq -r '.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta ' < /tmp/favorites.json | less

This is not complete, this gets you part of the way. What it does is return one record for each item in 'posts' (so, one record for every favorite, assuming that the above url returns complete rather than paginated data), where that record is a list of all the tags of all different types applied to that post. The step I haven't figured out yet is how to concatenate those records into one long list of strings.

To illustrate:

"some sample output"
["4_toes","anal","anal_penetration","animated_sketch","anthro","anus","blonde_hair","blue_eyes","blush","bouncing_breasts","breasts","butt","claws","clothing","collar","collar_only","cowgirl_position","crouching","eyes_closed","feet","fellatio","female","foot_focus","from_front_position","genitals","hair","jacket","jacket_only","kneeling","leash","legs_up","lying","male","male/female","nipples","nude","on_back","on_bottom","on_hands_and_knees","on_top","open_mouth","oral","paws","penetration","penile","pink_anus","pink_nipples","pink_nose","pink_paws","pink_pussy","pussy","sex","sitting","smile","soles","solo","spread_legs","spreading","toe_claws","toe_curl","toes","tongue","tongue_out","topwear","vaginal","vaginal_penetration","flummery","mammal","mouse","murid","murine","rodent","nihilochannel","animated","hi_res","short_playtime","sketch"]
["3_toes","after_sex","anus","barefoot","bodily_fluids","butt","claws","cum","cum_drip","cum_on_penis","cum_on_pussy","cum_on_tongue","cum_pool","cum_string","cum_taste","dripping","duo","erection","feet","feral","fur","genital_fluids","genitals","herm","herm/herm","intersex","intersex/intersex","leaking_cum","licking","licking_lips","mutual_masturbation","nude","open_mouth","orgasm","penis","presenting","presenting_partner","presenting_penis","presenting_pussy","pussy","pussy_juice","pussy_juice_drip","pussy_juice_string","saliva","saliva_string","self_lick","sharp_claws","sitting","soles","spread_legs","spread_pussy","spreading","symmetry","take_your_pick","toe_claws","toes","tongue","tongue_out","video_games","dragon","generation_5_pokemon","legendary_pokemon","pokemon_(species)","reshiram","scalie","zekrom","bamia","nintendo","pokemon","2014","digital_media_(artwork)","portrait","three-quarter_portrait"]
["big_breasts","big_butt","black_body","bodily_fluids","breasts","busty_feral","butt","eyelashes","eyeshadow","female","feral","freckles","half-closed_eyes","head_markings","horn","huge_breasts","huge_butt","hyper","hyper_breasts","hyper_butt","leg_markings","looking_at_viewer","looking_back","makeup","markings","multicolored_body","multicolored_horn","narrowed_eyes","orange_eyes","overweight","overweight_feral","pink_body","pink_horn","pink_tail","sigh","socks_(marking)","solo","sweat","tail_markings","two_tone_body","two_tone_horn","two_tone_tail","video_games","generation_5_pokemon","pokemon_(species)","scolipede","conditional_dnp","inkplasm","nintendo","pokemon","hi_res"]
["anatomically_correct","anatomically_correct_anus","anatomically_correct_genitalia","anatomically_correct_pussy","animal_genitalia","animal_pussy","anus","big_tail","blep","blush","bodily_fluids","butt","canine_pussy","claws","female","feral","fluffy","genital_fluids","genitals","looking_at_viewer","musk_clouds","pawpads","paws","presenting","pussy","pussy_juice","simple_background","solo","tongue","tongue_out","white_background","canid","canine","canis","fox","mammal","k0ira","2022","monochrome","sketch"]
["5_fingers","abdominal_bulge","areola","balls","bangs","bat_wings","bedroom_eyes","big_areola","big_balls","big_breasts","big_penis","big_pussy","black_hair","blue_eyes","blue_hair","bodily_fluids","breasts","clitoris","conjoined","cum","cum_in_pussy","cum_inside","egg","ejaculation","excessive_cum","excessive_genital_fluids","eyelashes","eyes_closed","eyewear","female","fingers","flared_penis","flying","gaping","gaping_pussy","genital_fluids","genitals","glasses","grey_hair","group","gynomorph","gynomorph/gynomorph","hair","hands_behind_head","horn","huge_balls","huge_breasts","huge_penis","humanoid_pointy_ears","hyper","hyper_balls","hyper_genitalia","hyper_penis","hyper_pussy","intersex","intersex/intersex","lips","macro","male","male/female","membrane_(anatomy)","membranous_wings","merging","multi_breast","multicolored_hair","narrowed_eyes","navel","nightmare_fuel","nipples","nude","open_mouth","oviposition","ovipositor_penis","penis","pink_body","pink_penis","pink_skin","precum","pseudo_hair","pussy","pussy_nipples","seductive","sharp_teeth","sweat","tattoo","teeth","tentacle_hair","tentacles","thick_thighs","tongue","two_tone_hair","unbirthing","unusual_anatomy","unusual_genitalia_placement","unusual_pussy_placement","vaginal","vein","veiny_penis","vore","what","what_has_science_done","where_is_your_god_now","why","wide_hips","wings","alien","humanoid","taur","modeseven","1:2","2022","absurd_res","digital_media_(artwork)","hi_res","signature"]

If given a list of strings that possibly repeat, sort | uniq -c will count for you how often each string occurs. Then you could optionally sort that by the count -- so sort | uniq -c | sort --reverse --numeric-sort.

Someone who is more familiar with jq than me might know how to fill in the missing step. it probably uses reduce.

EDIT:

jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' < /tmp/favorites.json

produces the required list of strings.

So a full command would be
jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' < /tmp/favorites.json | sort | uniq -c| sort -rn | less

EDIT2: note that the data returned by the above link is paginated, via page POST key. So the data for each page needs to be downloaded and processed into a list of strings, which need to be concatenated and only then fed into the sort | uniq | sort pipeline. Otherwise you will only get a histogram of the tags of the first 250 results.

EDIT3: Assuming you have downloaded every page of the favorites into /tmp/favoritesN.json, the following command should work to calculate a total histogram:

jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' /tmp/favorites*.json | sort | uniq -c| sort -rn | less

Giving a result like this:

"sample overall histogram"
749 mammal
    706 genitals
    636 female
    578 solo
    561 anthro
    479 pussy
    478 breasts
    460 hi_res
    459 bodily_fluids
    457 hair
    420 nude
    385 genital_fluids
    384 penis
    365 nipples
    365 fur
(further data omitted)

I'd recommend using curl to do the download since it supports number range expansion, eg https://e621.net/favorites?format=json&user_id=12345&page=[1-4]. Various download managers do too.

Updated

savageorange said:

There isn't an API endpoint for it to my knowledge. Of course if you process the json returned by the above link, it is possible.
Here is a quick experiment, which will work on Linux providing that a) you have jq installed, b) you have downloaded the favorites data into /tmp/favorites.json.

jq -r '.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta ' < /tmp/favorites.json | less

This is not complete, this gets you part of the way. What it does is return one record for each item in 'posts' (so, one record for every favorite, assuming that the above url returns complete rather than paginated data), where that record is a list of all the tags of all different types applied to that post. The step I haven't figured out yet is how to concatenate those records into one long list of strings.

To illustrate:

"some sample output"
["4_toes","anal","anal_penetration","animated_sketch","anthro","anus","blonde_hair","blue_eyes","blush","bouncing_breasts","breasts","butt","claws","clothing","collar","collar_only","cowgirl_position","crouching","eyes_closed","feet","fellatio","female","foot_focus","from_front_position","genitals","hair","jacket","jacket_only","kneeling","leash","legs_up","lying","male","male/female","nipples","nude","on_back","on_bottom","on_hands_and_knees","on_top","open_mouth","oral","paws","penetration","penile","pink_anus","pink_nipples","pink_nose","pink_paws","pink_pussy","pussy","sex","sitting","smile","soles","solo","spread_legs","spreading","toe_claws","toe_curl","toes","tongue","tongue_out","topwear","vaginal","vaginal_penetration","flummery","mammal","mouse","murid","murine","rodent","nihilochannel","animated","hi_res","short_playtime","sketch"]
["3_toes","after_sex","anus","barefoot","bodily_fluids","butt","claws","cum","cum_drip","cum_on_penis","cum_on_pussy","cum_on_tongue","cum_pool","cum_string","cum_taste","dripping","duo","erection","feet","feral","fur","genital_fluids","genitals","herm","herm/herm","intersex","intersex/intersex","leaking_cum","licking","licking_lips","mutual_masturbation","nude","open_mouth","orgasm","penis","presenting","presenting_partner","presenting_penis","presenting_pussy","pussy","pussy_juice","pussy_juice_drip","pussy_juice_string","saliva","saliva_string","self_lick","sharp_claws","sitting","soles","spread_legs","spread_pussy","spreading","symmetry","take_your_pick","toe_claws","toes","tongue","tongue_out","video_games","dragon","generation_5_pokemon","legendary_pokemon","pokemon_(species)","reshiram","scalie","zekrom","bamia","nintendo","pokemon","2014","digital_media_(artwork)","portrait","three-quarter_portrait"]
["big_breasts","big_butt","black_body","bodily_fluids","breasts","busty_feral","butt","eyelashes","eyeshadow","female","feral","freckles","half-closed_eyes","head_markings","horn","huge_breasts","huge_butt","hyper","hyper_breasts","hyper_butt","leg_markings","looking_at_viewer","looking_back","makeup","markings","multicolored_body","multicolored_horn","narrowed_eyes","orange_eyes","overweight","overweight_feral","pink_body","pink_horn","pink_tail","sigh","socks_(marking)","solo","sweat","tail_markings","two_tone_body","two_tone_horn","two_tone_tail","video_games","generation_5_pokemon","pokemon_(species)","scolipede","conditional_dnp","inkplasm","nintendo","pokemon","hi_res"]
["anatomically_correct","anatomically_correct_anus","anatomically_correct_genitalia","anatomically_correct_pussy","animal_genitalia","animal_pussy","anus","big_tail","blep","blush","bodily_fluids","butt","canine_pussy","claws","female","feral","fluffy","genital_fluids","genitals","looking_at_viewer","musk_clouds","pawpads","paws","presenting","pussy","pussy_juice","simple_background","solo","tongue","tongue_out","white_background","canid","canine","canis","fox","mammal","k0ira","2022","monochrome","sketch"]
["5_fingers","abdominal_bulge","areola","balls","bangs","bat_wings","bedroom_eyes","big_areola","big_balls","big_breasts","big_penis","big_pussy","black_hair","blue_eyes","blue_hair","bodily_fluids","breasts","clitoris","conjoined","cum","cum_in_pussy","cum_inside","egg","ejaculation","excessive_cum","excessive_genital_fluids","eyelashes","eyes_closed","eyewear","female","fingers","flared_penis","flying","gaping","gaping_pussy","genital_fluids","genitals","glasses","grey_hair","group","gynomorph","gynomorph/gynomorph","hair","hands_behind_head","horn","huge_balls","huge_breasts","huge_penis","humanoid_pointy_ears","hyper","hyper_balls","hyper_genitalia","hyper_penis","hyper_pussy","intersex","intersex/intersex","lips","macro","male","male/female","membrane_(anatomy)","membranous_wings","merging","multi_breast","multicolored_hair","narrowed_eyes","navel","nightmare_fuel","nipples","nude","open_mouth","oviposition","ovipositor_penis","penis","pink_body","pink_penis","pink_skin","precum","pseudo_hair","pussy","pussy_nipples","seductive","sharp_teeth","sweat","tattoo","teeth","tentacle_hair","tentacles","thick_thighs","tongue","two_tone_hair","unbirthing","unusual_anatomy","unusual_genitalia_placement","unusual_pussy_placement","vaginal","vein","veiny_penis","vore","what","what_has_science_done","where_is_your_god_now","why","wide_hips","wings","alien","humanoid","taur","modeseven","1:2","2022","absurd_res","digital_media_(artwork)","hi_res","signature"]

If given a list of strings that possibly repeat, sort | uniq -c will count for you how often each string occurs. Then you could optionally sort that by the count -- so sort | uniq -c | sort --reverse --numeric-sort.

Someone who is more familiar with jq than me might know how to fill in the missing step. it probably uses reduce.

EDIT:

jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' < /tmp/favorites.json

produces the required list of strings.

So a full command would be
jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' < /tmp/favorites.json | sort | uniq -c| sort -rn | less

EDIT2: note that the data returned by the above link is paginated, via page POST key. So the data for each page needs to be downloaded and processed into a list of strings, which need to be concatenated and only then fed into the sort | uniq | sort pipeline. Otherwise you will only get a histogram of the tags of the first 250 results.

EDIT3: Assuming you have downloaded every page of the favorites into /tmp/favoritesN.json, the following command should work to calculate a total histogram:

jq -r '[.posts | .[] | .tags.general + .tags.character + .tags.species + .tags.artist + .tags.copyright + .tags.invalid + .tags.lore + .tags.meta] | reduce .[] as $item ([]; . + $item) | .[]' /tmp/favorites*.json | sort | uniq -c| sort -rn | less

Giving a result like this:

"sample overall histogram"
749 mammal
    706 genitals
    636 female
    578 solo
    561 anthro
    479 pussy
    478 breasts
    460 hi_res
    459 bodily_fluids
    457 hair
    420 nude
    385 genital_fluids
    384 penis
    365 nipples
    365 fur
(further data omitted)

I'd recommend using curl to do the download since it supports number range expansion, eg https://e621.net/favorites?format=json&user_id=12345&page=[1-4]. Various download managers do too.

Oh wow, I did not expect such a detailed answer, thank you so much!
I really appreciate you being so thorough with your explanation. Not only did I get the data I was looking for, i now also know how to use jq AND cURL!

Next project, doing something fun with said user data :D
Thank you again, cheers!

Updated

No problem, it's actually something I had tried to do previously but didn't understand jq well enough to achieve at that time.

  • 1