Topic: Decensor GIMP script live development

Posted under e621 Tools and Applications

What is it?

Some Japanese artists have helpfully censored using transparent black rectangles. It should be possible to completely decensor if you can guess the exact color and opacity of the rectangle. For example, 50% black aka rgba(0,0,0,0.5).

pool #5397

The script will have to work with any color and any opacity, not just 50%. One test showed a value closer to 52%. The edges of the rectangle may be feathered. Meaning you might have to make multiple selections.

Adding color

rgb(128,0,128) + rgba(255,192,0,0.6) = rgb(204,115,51)

(128 * 0.4) + (255 * 0.6) = 204.2
(0 * 0.4) + (192 * 0.6) = 115.2
(128 * 0.4) + (0 * 0.6) = 51.2

Subtracting transparent color (decensor)

((204.2 - (0.6 * 255)) / 0.4) = 128
((115.2 - (0.6 * 192)) / 0.4) = 0
((51.2 - (0.6 * 0)) / 0.4) = 128

Code

It doesn't work but it will look like this:

(define (script-fu-decensor img
							drawable
							mask-color
							mask-opacity)
							
	(define color-get-red car)
	(define color-get-green cadr)
	(define color-get-blue caddr)
	
	(gimp-context-push)
	
	(gimp-image-set-active-layer img drawable)
	
	(gimp-image-undo-group-start img)
	
	;; Do stuff here
	
	(let* ((selection-bounds (gimp-selection-bounds img))
        (select-offset-x (cadr selection-bounds))
        (select-offset-y (caddr selection-bounds))
        (select-width (- (cadr (cddr selection-bounds)) select-offset-x))
        (select-height (- (caddr (cddr selection-bounds)) select-offset-y))
		(y select-offset-y)
		(x select-offset-x)
	)
	
	(gimp-selection-none img)
	
	(gimp-palette-set-background (list 255 255 255))
	(gimp-palette-set-foreground (list 0 0 255))
	(gimp-brushes-set-brush "Circle (01)")
	
	(while (< y (+ select-offset-y select-height))
	(set! x select-offset-x)
	(while (< x (+ select-offset-x select-width))
	
	(let* ((in-color (gimp-drawable-get-pixel drawable x y))
	(out-color (list (cons (aref (aref in-color 4) 0) (aref (aref in-color 4) 1)) (cons (aref (aref in-color 4) 2) (aref (aref in-color 4) 3)) (cons (aref (aref in-color 4) 4) (aref (aref in-color 4) 5))))
	)
	
	;;CHANGE FILL COLOR
	
	(gimp-rect-select img x y 1 1 REPLACE 0 0)
	(gimp-edit-fill drawable 0)
	(gimp-selection-none img)
	
	)
	
	(set! x (+ x 1)))
	(set! y (+ y 1)))

	(gimp-image-undo-group-end img)
	(gimp-context-pop)
	(gimp-displays-flush)
	)
)

(script-fu-register "script-fu-decensor"
    "Decensor"
    "Subtracts a partially transparent color \previously added to the image."
    "Anonymous"
    "Anonymous"
    "2015"
    "*"
	SF-IMAGE      _"Image"          0
	SF-DRAWABLE   _"Drawable"		0
	SF-COLOR      _"Color"			'(0 0 0)
	SF-ADJUSTMENT _"Opacity"		'(50 0 100 1 10 0 1)
)

(script-fu-menu-register "script-fu-decensor" "<Image>/Filters/Enhance")

Updated by savageorange

Can be done by hand too of course, just takes more time

Updated by anonymous

memeboy said:
I guess that's censoring?

You guessed right.

Imagine if every single Japanese xxx artist used this technique of slapping a 50% black rectangle on the naughty bits. We could just script all the censorship away. Too bad that won't happen. But if someone can actually read and write Japanese and bothers Pixiv artists they should share this idea.

Updated by anonymous

In cases where the censor is RGB(0,0,0) at some level of opacity 0-1, you can just multiply the pixel values by (1/(1-opacity)) -- for example 2 for the case you gave of 0.5 opacity.

Divide layer mode implements this -- it multiplies the underlying pixels by 255/value, so eg. 127 == multiply by 2, 63 = multiply by 4 (appropriate if the opacity was 75%), 192 = multiply by 1.33 (appropriate if the opacity was 25% -- 1/ .75 == 1.33). Hopefully it is clear from this that the censoring process -does- irrevocably destroy some data -- number of RGB levels is reduced from 256 (0% opacity) to 128 at 50% opacity, 64 at 75% opacity. Past 75% opacity, the permanent color degradation will become increasingly obvious, especially given that JPG compression purposefully reduces precision in darker areas of the image.

For colors other than black, I haven't worked out the math (though chances are, there is an existing layer mode that can do it for you..). But the above works, for any opacity of black.

EDIT: testing with white 50%, it seems the thing to do is to subtract 50% of the white (per your algorithm AFAICS) and then multiply by 100/50 ( ie. 2). This can be done with a 100% opacity Subtract layer containing #808080, below a 100% opacity Divide layer also containing #808080.

Poking values around:

  • white 75% -> subtract 75% of the white, multiply by 100/25 (ie. 4). This can be done with a Subtract layer containing #bfbfbf, below a Divide layer containing #404040.
  • #8899aa 50% -> dunno. Tried, haven't got a sensible looking formula yet.

So, unless I misunderstood your algorithm, it contains one half of the above, the subtraction only.

Updated by anonymous

Subtracting 50% black:

5f3a42 -> BE7484
(95,58,66) -> (190,116,132)

663e47 -> CC7C8E
(102,62,71) -> (204,124,142)

50% looks typical. So it's 16,777,216 colors down to 2,097,152. It sounds bad but that's still a lot of colors. It's hard to spot the difference unless it's a 2 color image. Try making an image half #be7484 rgb(190,116,132) and half #bf7585 rgb(191, 117, 133). You can spot it but it's subtle. That's ±1 to R, G, and B, the greatest deviation possible, in the most visible way possible.

I couldn't get the divide blend mode to work on my own. I created a transparent layer with black on the censored area, but I didn't get a correct result.

Updated by anonymous

Lance_Armstrong said:
Subtracting 50% black:

5f3a42 -> BE7484
(95,58,66) -> (190,116,132)

663e47 -> CC7C8E
(102,62,71) -> (204,124,142)

I agree these results are correct, so perhaps I misunderstood your algorithm.

50% looks typical. So it's 16,777,216 colors down to 2,097,152. It sounds bad but that's still a lot of colors.

Yes, this is why I gave that 75% cutoff point. 64 levels of RGB is equivalent to old MCGA video mode, and begins to display some visible banding and oversaturation. Though considering the small areas usually involved, you may be able to go down to 32 (87.5% opac) or even 16 (93.75% opac) levels before things start looking icky. Encoding this in a JPEG will throw some of the remaining available precision away, though.

(you can see the effects very directly and simply by selecting an uncensored area and using the Colors->Posterize tool with the appropriate number of levels (eg. 64). Since this has a real-time preview, it shows very well the gradual progression of colors towards oversaturation and banding.

It's more of an academic concern in this area, possibly -- just my interest in restricted colorspaces and pixelart showing -- since
transparent censors tend to not go above 80% opacity, IME.

Try making an image half #be7484 rgb(190,116,132) and half #bf7585 rgb(191, 117, 133). You can spot it but it's subtle. That's ±1 to R, G, and B, the greatest deviation possible, in the most visible way possible.

I'm not sure why you are citing those colors, as with 128 levels, the minimum amount of distance is 2. #be7484 rgb(190,116,132) and, say, half #c07686 rgb(192, 118, 134). If you allow hue-distortion you can get closer than that of course -- #be7486 rgb(190, 116, 134) or other variations.

I couldn't get the divide blend mode to work on my own. I created a transparent layer with black on the censored area, but I didn't get a correct result.

That's not how divide mode works (although I suspect 0 is a special case, to avoid dividing by 0.)
255 / 0 is (undefined), probably treated as 1. This means that all input pixels will be multiplied by 1, therefore passing through unchanged.

I'll run through it in a slightly more explicit way:

  • You are working with 50% black (0,0,0) as the censor color
  • You have already subtracted out the necessary values (nothing, in this case, since 0*anything is still 0) with a Subtract layer
  • Your intensities therefore currently range 0-127; you want them to range 0..255.
  • Divide mode makes this simple: you just specify the intensity you want to end up being the new 255 (ie. 127). You would therefore fill the divide layer with #7f7f7f (or #808080, close enough) and this would multiply all intensities by 2 (normalizing the range 0..127 -> 0..255).
  • If you filled with #3f3f3f, that would multiply by 4 (normalizing the range 0..63 -> 0..255). That would be appropriate if the censor was 75% opacity.

HTH.

Updated by anonymous

I'll try it. The way I have my script set up to work it reads in the color of every pixel in a rectangle selection, will perform the subtraction using the formula (color - (opacity * censor))/(1-opacity), once for red, blue, and green components, setting the pixel to the result color.

Typically (color - (0.5 * black))/0.5

And it will work if I can figure out the annoying syntax. The broken script I posted changes every pixel in the selection to blue one at a time.

Updated by anonymous

And it will work if I can figure out the annoying syntax. The broken script I posted changes every pixel in the selection to blue one at a time.

There are two points worth making about the script IMO:

  • You may find using Python-fu easier (and if you bring up the console (Filters->Python-fu->Console), you can test stuff interactively. I'll mention gimp.image_list() to you as it will help you if you do that.)
  • The operation you are doing actually, AFAICS, needs no pixel-level scripting. Since each RGB channel is treated independently, you can build a 256-entry array (ie lookup table) for each of r,g,b using the formula you stated, and simply call pdb.gimp_curves_explicit() three times, once for each channel -> all selected pixels are processed. If you look gimp-curves-explicit up in the PDB browser, it's pretty straightforward.

Here's an example array, usable for all three channels on a 50% black censor (since all channel values are the same in black):

  • Python code, compact:

[min(255, v*2) for v in range(256)]

  • Python object resulting from the above:

[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 186, 188, 190, 192, 194, 196, 198, 200, 202, 204, 206, 208, 210, 212, 214, 216, 218, 220, 222, 224, 226, 228, 230, 232, 234, 236, 238, 240, 242, 244, 246, 248, 250, 252, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255]

Applying to all three channels, given a drawable 'drawable' and the above table stored in 'lut' (and bullet points standing in for indentation):

for channel in (1,2,3):

  • pdb.gimp_curves_explicit(drawable, channel, 256, lut)

(verified in the python-fu console; the only lines I typed before this were:

  • i = gimp.image_list()[0]
  • drawable = i.layers[0]
  • lut = [min(255, v*2) for v in range(256)]

)

Yeah.. it can be a lot simpler than that script fu currently is.

Updated by anonymous

savageorange said:
I'm not sure why you are citing those colors, as with 128 levels, the minimum amount of distance is 2. #be7484 rgb(190,116,132) and, say, half #c07686 rgb(192, 118, 134). If you allow hue-distortion you can get closer than that of course -- #be7486 rgb(190, 116, 134) or other variations.

I think I rounded wrong but it's still sound.

rgb(191,117,133) can't exist in the 128x128x128 color space. It needs to be rounded.

If you round it to rgb(192,118,134) it's (+1,+1,+1).
If you round it to rgb(190,116,132) that's (-1,-1,-1).

No matter what you will hit the nearest valid color within (±1,±1,±1).

Since you gave me so much help I will come back to the thread with a working python script that decensors any rgba color.

Updated by anonymous

Lance_Armstrong said:
I think I rounded wrong but it's still sound.

rgb(191,117,133) can't exist in the 128x128x128 color space. It needs to be rounded.

If you round it to rgb(192,118,134) it's (+1,+1,+1).
If you round it to rgb(190,116,132) that's (-1,-1,-1).

No matter what you will hit the nearest valid color within (±1,±1,±1).

Oh, now I understand what you meant. Yeah, I agree.

I look forward to trying your script.

Edit: Now that I think about it, this could also be implemented pretty easily as a GMIC filter, which would give it an interactive realtime preview. That could be pretty sweet, but it would depend on GMIC for GIMP already being installed.

Updated by anonymous

I think what would be really nice is if they were all just layered PNG images where you could delete the layer with the censoring. Though I am sure that this isn't entirely possible for whatever reason. It would just be nice.

Updated by anonymous

Pasiphaë said:
I think what would be really nice is if they were all just layered PNG images where you could delete the layer with the censoring. Though I am sure that this isn't entirely possible for whatever reason. It would just be nice.

I would assume that would probably get the artists in trouble.

Updated by anonymous

Pasiphaë said:
I think what would be really nice is if they were all just layered PNG images where you could delete the layer with the censoring. Though I am sure that this isn't entirely possible for whatever reason. It would just be nice.

PNG doesn't support layers, nor does APNG or JPG.

You could sort of hack it in in APNG though:
The first frame records the censored image. Its frame delay (see the APNG spec ) parameter is set to 65535/1 (65535 seconds, which is 18.2 hours).
Second layer records only the original pixels of the censored areas (pixels where the 'censored' image differs from the uncensored one), and is displayed with 0/0 delay (ie. go to next frame ASAP).
Result: image displays censored in both browsers that support APNG and those that don't. After sitting for 18.2 hours (shyeah right) in front of an APNG-supporting browser, the uncensored version is displayed for probably about 0.05s, then switches back to the censored version.

Opening it in (GIMP / whatever APNG-supporting image editor) and merging the layers uncensors it.
This would allow any possible censor method, without major overhead.

The perceived legality of this is probably in inverse proportion to how easy it is for the average person to actually perform the 'cancel censoring' operation.

Something similar could be done with WEBP, with the delay of frame one maxing out at 4.66 hours/16777215 milliseconds.

The only genuinely 'layered' image format for the web is SVG, AFAIK.
With better SVG support, this would be the most suitable IMO : overhead is reasonable, no 'animation timeout' issue, censoring can be implemented via native SVG blur or native SVG rectangles if desired (saving space compared to the rasterized censoring that is currently common).
This can reasonably be viewed as a genuine censor.

However, currently SVG support is not that good -- particularly hosting, most image hosts do not support SVG even though most browsers have some level of SVG support .
this is the only SVG-supporting image host I know of for sure.

Updated by anonymous

Pasiphaë said:
I think what would be really nice is if they were all just layered PNG images where you could delete the layer with the censoring. Though I am sure that this isn't entirely possible for whatever reason. It would just be nice.

It is quite literally against the law to draw uncensored pornography. The fact that the censor makes it legal is a loophole, which Japan is very fond of. They've got a loophole for the gambling law, as well--They don't get money for winning games, just more pachinko balls, which they exchange for prizes, go next door to the pawn shop, and sell those for money. Think if Chuck-E-Cheeses had a pawn shop next to it, so you could exchange tickets for prizes for money. Not technically gambling, but you're risking money for a chance to end the day with more.

Updated by anonymous

Pasiphaë said:
I think what would be really nice is if they were all just layered PNG images where you could delete the layer with the censoring.

That reminds me of accidental leaks where some govt agency released a PDF with redactions, but the redactions were just a layer on top of the text that you could delete.

savageorange said:
PNG doesn't support layers, nor does APNG or JPG.

You could sort of hack it in in APNG though:

Furrin_Gok said:
It is quite literally against the law to draw uncensored pornography. The fact that the censor makes it legal is a loophole, which Japan is very fond of.

This transparent censor used by j7w and baneroku is the closest I've seen to noncompliance while still censoring. It still permanently degrades the image quality. You may still have to redraw it after correcting the color, but it saves you work.

The APNG idea is clever. If it landed an artist in court maybe they could say it was "simulated blur". But I'm going to make 2 generalizations here.

1. We (baka gaijin) can't effectively communicate anti-censor tricks to a critical mass of Japanese artists.
2. Japanese people give deference to authority, and although artists (especially smut ones) love free speech, they will continue to obey the law. Meaning solid black bars forever.

Updated by anonymous

Lance_Armstrong said:

Meaning solid black bars forever.

And cute little heart shapes and well-placed kanji.

Updated by anonymous

Lance_Armstrong said:
We (baka gaijin)

I-idiot! It's not like I'm a foreigner or anything...!

Updated by anonymous

IMHO such script isn't feasible. JPEG artifacts (underlying tiling used by JPEG) and rescaling interpolations will give you some grayish borders around the decensored areas, which you will have to manually fix them, and that defeats the purpose of an automated tool for this.

Updated by anonymous

Lizardite said:
IMHO such script isn't feasible. JPEG artifacts (underlying tiling used by JPEG) and rescaling interpolations will give you some grayish borders around the decensored areas, which you will have to manually fix them, and that defeats the purpose of an automated tool for this.

Certainly there is some degradation -- JPG (or any other lossy format) is a bad source format. That said, I don't agree that being able to automatically do the majority of the decensoring is valueless; I'm sure that anyone who wants to decensor stuff would appreciate this time saver.

(otherwise, you're likely to end up fiddling with brightness/contrast controls, obtaining a rather imperfect result that then needs a lot of touchup. This would be particularly the case when the censor color is not something simple like white or black)

If you then select the heavily artefacted pixels, GMIC Inpaint filter can automatically clean them up with decent accuracy, leaving perhaps a dozen lines that need manual repainting.
Then you should save in PNG or another non-lossy format to avoid adding more JPG artifacts.

I don't personally have any interest in decensoring except at the 'image processing puzzle' level, but people are clearly gonna decensor, so why not save them some time?

(we could also do even better than this.. Computer vision techniques -- see GIMP FG/BG select, GMIC interactive foreground extraction -- could, with a single click on each censored area, detect the censored area and locate any feathered borders if applicable. Applying Lance's algorithm using this smooth mask, and then selecting the bordering areas and GMIC Inpainting that area automatically, this would accomplish 95% of the work of decensoring in a few clicks)

(For other methods that don't completely destroy the source data, there are also ways of automation. Mosaic censorship, you can automatically detect the presence and area of, and semi-intelligently smooth using GMIC smart upscale, leaving the decensorer only to resolve higher resolution details from the overly smoothed base.)

Updated by anonymous

Furrin_Gok said:
It is quite literally against the law to draw uncensored pornography. The fact that the censor makes it legal is a loophole, which Japan is very fond of. They've got a loophole for the gambling law, as well--They don't get money for winning games, just more pachinko balls, which they exchange for prizes, go next door to the pawn shop, and sell those for money. Think if Chuck-E-Cheeses had a pawn shop next to it, so you could exchange tickets for prizes for money. Not technically gambling, but you're risking money for a chance to end the day with more.

The thing that I find totally bat shit insane about the law is how oppressive it is. I mean I can't imagine an entire modern culture that still makes it illegal for an artist to draw nude images. I mean you go back a few hundred years and drawing people naked was normal and in fact encouraged as the best way to learn how to properly learn size and scale and become a better artist.
Also at some point during the creation of the censored image, unless the artists draws the censor in first the image is technically uncensored and illegal. What if the poor guy gets caught mid drawing by the police? Can't he just say: "Hey I was gonna draw the bars in but you guys interrupted me." lol

Updated by anonymous

What if the poor guy gets caught mid drawing by the police? Can't he just say: "Hey I was gonna draw the bars in but you guys interrupted me." lol

Japanese, at least, have a great cultural propensity for ignoring things. Or at least pretending to ignore them. IMO your described case falls firmly in the '-actually arresting- someone for this mainly proves you are a frivolous wanker'.
It's the same story for many questionable Western laws -- if you don't parade your violation of them up and down, it's less embarrassing for everybody concerned to just leave it be.

Updated by anonymous

Lance_Armstrong said:
Any more features and I might as well write an AI instead

Haha, I'm not proposing you include those things... They are interesting things I already know for sure are possible, that's all.

How

Autodetecting pixelation:

Oddly enough, you can do this with repeated applications of the 'Pixelate' filter. You just keep trying different sizes (up to a max of 32, perhaps) and calculate the average difference for each cell versus the original image (which, funnily enough, you can also approximate using a Pixelate filter). Cells where the difference is very low are part of the mosaiced area. After you've tried all the different sizes, you pick the size with the lowest difference in the mosaiced area; this is the correct mosaic size, and you already calculated the mosaic mask.
This will work for mosaics which are not offset, which AFAIK is 99% of them.

GMIC inpainting -- well, that's just calling a filter, after using 'border selection' or selection feathering and thresholding to get the area around the censored area, and filling that area on a new transparent layer with white. Basically just needs experimentation to find optimal filter parameters.

Smart upscaling --similar.
Given a mosaiced source area (with transparency where the mosaic isn't):
First, you store a copy of the alpha channel as a new greyscale image (because Smart Upscaling doesn't preserve alpha). Then you scale the mosaiced image down so the mosaic becomes pixels (6x6 mosaic tile; downscale image to 1/6th size, nearest neighbour -> each mosaic tile is now a single pixel). Then you hit Smart Upscaling, with a high anisotropy -- probably 1.0, and a scaling factor no greater than 200%. Repeat until you reach the original scale (eg. for 6x6 mosaic tiles, you want 600% upscaling in total, which is achieved by 200% (->2x) + 200% (->4x) + 150% (->6x)). Perform the same process on the alpha channel image you made, Normalize it, and add it back into the smoothed version of the mosaiced areas. You can now overwrite the original mosaic with this, or put it on its own layer for any further editing. Example result (no touchup has been done) : here . Original is post #574434

The feathering-detection is another mask-based trick, based on the fact that censors are typically a distinct, contiguous region -- really just a variant on floodfilling/magic wand, just doing it with multiple thresholds and then combining the generated masks to figure out what is censor, what is feather, and what is non-censor. Very similar in concept to the pixelate detection.

Some minimal level of user interaction will always be required, of course, since flat rectangles are a legitimate image element in other cases. These techniques just help reduce 'mark out the exact bounds' to 'click somewhere inside each censored area'

Updated by anonymous

  • 1