Using AI to make pictures ‘better’

Using AI to make pictures ‘better’

There are some things that computers are good
at. There are other things where humans are better. But not for long! AI is here. Computers
are learning how to do human stuff… but BETTER.
They will be faster. Smarter. WAYYYY more rational. If this sounds scary, that’s good!
Because it is. …But for now we may as well enjoy stuff
it can do like image upscaling! If you asked me a few years ago if you could
make an image more detailed by making it bigger, I would laugh in your pathetic little face!
Those CSI shows where they used ENHANCE to uncover details in images were internet memes.
Nobody took it seriously. Shows how much we know. Because AI upscaling is here and it’s
terrifyingly good. I’m using Topaz Gigapixel AI to upscale
the images in this video. I did actually buy it and everything because this program has
given me as much pleasure as shooting palm trees in Crysis, though they do have a 30
day trial on their site. The way AI upscaling works is… well, nobody
knows. It’s magic! It learns what images should look like, then applies that knowledge
to new stuff. If I upscale this tree, it almost becomes lots and lots of little trees joined
together to give the impression that it’s actually a more detailed tree. It’s super
effective! AI upscaling really loves foliage. You know
something it doesn’t like? Faces. URGH!
I did wonder if this video would get demonetized for showing this, but then I remembered that
AI algorithms thought this was what a face looked like in the first place… so Youtube’s
filters will probably like it. AI isn’t actually bad at faces. It’s just
that we’re very good at faces. We know what they’re meant to look like, and can detect
when something’s wrong with them. We’re not so good with trees, so this looks perfectly
acceptable to us. That’s why there’s a phenomenon known
as ‘uncanny valley’, which is when digital or drawn people look ALMOST real, but not
quite, which makes them look really creepy to us. I guess I’ll get the obvious out of the
way: you can’t keep upscaling forever. It can’t get details that weren’t captured
in the original image. It can only fake it by adding small touches that it thinks should
go there. My friend got bullied by a van driver who
kept braking and swerving, seemingly to cause a crash. They came to me asking if I could
somehow retrieve their numberplate from THIS. Uh, yeah… I don’t think so. I’m not
proud of myself but I did try running it through this program in the hope of a miraculous ‘ENHANCE!’
moment. Obviously it didn’t work! But I had to try
it didn’t I. So let’s get to some examples, of what works
and what doesn’t. I tried on a few old family photos. It would be great if I could some
how make these crisper and clearer… and a few of them, I did!
Here’s one of my MUM. Now, I’ve blurred her face out cos I don’t want thousands
of people staring at my MUM, but also because we’ve established already that AI does terrifying
things to faces. Actually, I will show her face, but horrendously upscaled from a tiny
image. There. Here’s before…
…Here’s after. What a difference! Something I learned from the hours of fun
I had messing around with this program is that, for best results… you have to shrink
the image. That’s right: to upscale, you must first downscale!
Bear with me here! It obviously depends on the image. But for these old ones, the original
scans aren’t the clearest. They have a lot of blur, which the AI must look at and go
‘hmm I had better keep this blur!’ While if there’s greater contrast between
the pixels, it enables the AI upscaling to splurt its creative juices all over the picture
in the most delicious of ways. So here’s the original, here it is AI upscaled
normally. Here’s the half-res original… and here’s the upscaling for that.
And now you can see the upscaled versions compared.
This makes me feel good for humanity that AI still needs a helping human hand for best
results! Although impressive, it isn’t magic. I was
a bit late with this video because I spent the best part of a week messing about with
it on all kinds of stuff and I’ve gotten pretty good at knowing how good results will
be before I even try them now. AI upscaling can do 2 things: it can make
a horribly pixelated picture a lot better, or it can make an already decent picture…
more decent. Let’s start with pixelated images. Here
is the skyline of San Francisco. I got this original image, squished it down to just 384×216
pixels… and then gave that to the AI upscaler and it produced THIS. Impressive, isn’t
it. It really adds clarity to the outlines of the buildings, as well as making sense
of the mess of pixels on the walls and windows. But it’s not ‘perfect’. You can see
lots of phantom colours creeping into this one- the buildings almost look like they’ve
been drawn in crayon! The pyramid building looks great in my opinion, having successfully
made the sides smooth from what was about 6 pixels of steps before. And the windows
were each about a pixel large, which I think makes this upscaling very impressive. Something
interesting to note here is the jpegging artifacts in the sky- the AI has taken those and has
painted a larger part of the picture with them! The Salesforce tower has been strangely
smoothened- if we look at the original image you can see it should actually have very clear
rows of windows. But from the low res image it’s become a huge rounded dildo. As usual
I feel it’s done a good job with the trees- especially when all it looks like before is
a jumble of random brown pixels. Yosemite! Again, trees look more tree-y and
I think it’s nicely textured the mountains as well. To think that from just a few pixels
it can conjure up a convincing rock texture. Imagine if it got it wrong and made them look
like soup or something! As always, peoples’ faces turn into nightmare fuel. I mean what’s
going on with this cartoon-girl here? Now the Grand Canyon! Looks like my face is
trapped in a plastic bag, but the rest of the image is greatly improved- I’d say it
looks closer to the original image than it does to the pixelated one. But you’re free
to disagree. Now to the tallest building in Devon. AI upscaling
makes the clouds look very much like clouds, the grass like grass, the pavements also look
a hell of a lot better… but again I think the trees are the best bit of the picture!
If AI could have a favourite, you know it’s to upscale branches.
I feel like a broken record here but it continues to make everything look a lot more like what
they are. Let’s just forget about the pixelated image here and compare the upscaled with the
original- I can’t even tell from the sky which is the original and which is upscaled,
even though everything ends up looking quite different. For once I don’t think it does
a very good job on the grass- it makes it look more like branches, when in the original
you can clearly see they’re blades of grass. Ha, stupid AI.
This beautiful view of the Cornish coastline was unfairly crushed down to just 240 pixels
wide- the lowest I went in this test! And when it’s shrunk down this far, the AI begins
to struggle. It manages what looks like a 10-year old phone-photo: blurry, lacking detail,
horrible halos around everything… but I think you can agree it still looks a helloolwollalot
better than THIS. Now onto a graveyard shot, which I didn’t
pixelate AS much as with the previous pictures. With this extra detail, the AI has a field
day, lavishing the grave stones with fake writing, nice bumpy stone textures, the trees
look great, the only thing that’s a bit off is the grass. The AI got a little too
excited about this and made them look like dancing snakes, without anywhere near as many
strands as in the original. But I think you can agree that as a whole, this upscaled image’s
quality is very acceptable. And lastly we’ll see what it does to this
adorable doge. It correctly makes it look very hairy. But I’m more impressed with
the background- it has successfully identified that it’s meant to be blurry, so instead
of adding more detail like it has to the dog, it’s nicely blurred it so that it looks
out of focus again, just like with the original image. That in my opinion is really impressive,
and shows intelligence. When we compare the upscaled dog with the original you can see
the hair has lost a bit of clarity. It’s almost like comparing low and high texture
detail in a videogame, isn’t it. Integrated graphics… tress FX! Integrated graphics…
tress FX! And so on. So upscaling from a pixelated image is something
AI really excels at. I guess these look so bad in the first place that AI can ONLY make
it look better. The upscaling might not necessarily make them look more like the original picture
again, but it’s worth it for the de-blurring alone and more often than not it works a small
miracle by nicely texturing various surfaces in a convincing manner.
But what about making a decent picture look… more decent? I mean, surely it’s the same
goal, but just starting with more pixels? I first tried upscaling the original images
but it resulted in some crazy 30,000 pixel-wide photos which killed my PC, so instead I’ll
just get small bits from the original images to upscale.
Let’s do the salesforce tower justice. But I don’t know if it has- it looks disappointingly
similar in my opinion, minus the pixellation. I mean don’t get me wrong I think it’s
an improvement, but it just isn’t as impressive as drawing a whole picture from just a few
pixels. I decided to zoom in on Yosemite’s famous
half-dome, and again… not as impressed with the result, which appears too pastel-y. I
blame this on the original image’s blur and haze- it’s not like it’s upscaling
pixels with great contrast… which makes the resulting harsh tones all the more mysterious
in my opinion. I don’t think the AI knew what it was doing here.
Just out of curiosity I halved the resolution of the before image, to see if increased contrast
would lead to better results. You could say it’s just a quarter dome now! But this didn’t
particularly improve the upscaling. In fact, it introduced more oddly coloured halos around
stuff. The Grand Canyon: one of the wonders of the
world. A spectacular sight! …and so from this image I chose to upscale my FACE. And
again, it’s almost like it doesn’t know what to do with the image. I swear it’s
added some wrinkles that weren’t there before. But it doesn’t even make the hair look hairier-
something that AI upscaling usually loves! Again, I tried halving the resolution of the
original image. And this time I think it really improved the result. I look a lot more crisp…
at least the hair now looks hairier. And my bag strap looks like it’s gotten an HD makeover.
Well done, AI. You’ve redeemed your face-upscaling in this example.
As a treat I fed it this tree image. AND IT WENT TO TOWN ON IT! Beautiful. Thousands of
creeping, criss-crossing twigs and branches. You can tell it enjoyed upscaling this. I
think the original image was taken on a Google Pixel, which likes to over-sharpen its images,
so maybe that’s why this result is so good. We’ve learned that it likes high-contrast
images. We know it often upscales better if the original image is shrunk down a bit. So
that’s what I chose to do with this shot of the sea. I shrunk the original to a third
of the size and got that to be AI-upscaled. Is the result better than the original image?
I think it is. Some of the grain is lost. Some of the clarity in the darker areas also.
But the whole image looks more in-focus, the distant sea looks a lot clearer, and I also
like what it’s done to the grass just here. It’s pretty much behaved like a denoiser,
only it’s noisened up the bits that had no noise before.
I did the same with this shot, and again I think upscaling from an image one third the
size results in something better than the original! Make no mistake about it- you can’t
get closer to reality than the original image. But there’s no mistaking that the upscaler
has made the image crisper and in parts, seemingly more detailed.
If nothing else, I reckon this AI upscaling could be used to compress images. Shrinking
them to a third will make the filesizes 9x smaller, and it generally upscales them to
close to the original quality again. Of course I’m sure it would sometimes horribly mess
it up. But it has potential! I got to this bit of the video and wondered
if it was a bit disrespectful to upscale grave stones- especially if I single one out in
particular. Almost seems like bullying. So I’m going to use this picture instead.
Believe it or not, no filters were used here. Just my Google Pixel’s powerful post-processing
magical HDR effects. Oh and a lens flare. I’ll just get rid of that. I think we should
try and upscale the JOLLY MILLER! There’s a lot of noise in this original image- let’s
see what the upscaling AI thinks of that. Honestly, not too impressed with the result.
It denoises it somewhat, and of course gets rid of pixellation on the signs. Jazzercise
is looking pretty damn hot there. But I feel maybe this image was too muddy to do much
to the building itself. …But then I halved the original image’s
size… and this happened. Oh my God, this is a true ‘enhance!’ moment. It’s removed
the noise and has pretty much drawn a door that wasn’t really there before. And look
at the fence in the window’s reflection! And the tiles on the roof! And the bricks
themselves! I know this isn’t NEW detail- it’s just the AI doing stuff with what was
there already. But it’s good enough for me, even if the result is a bit like a water
colour at times. Also in case you were wondering, the gravestones
ended up looking sharper and the jpeg artefacts disappeared.
How about the dog’s eye. I was unimpressed with this- it doesn’t look quite so much
like ‘hair’ any more- and in parts it looks like it’s attempted to split hairs
into smaller ones! Which I did wonder if it would try. I halved the resolution of the
before image and this led to a clearer result- but in my opinion it looks fake and has too
much contrast, and looks less like hair than it did previously. Sorry, dog. AI doesn’t
like you… but you’ll forever be man’s best friend. Or at least until AI takes your
job as well. If nothing else, I reckon this AI upscaling
could be used to compress images. Shrinking them down to a third will make the filesizes
9 times smaller, and it generally upscales them to close to the original quality again.
Of course, I’m sure it would sometimes horribly mess it up. But it has potential!
We’ve seen real life images. But what about games? And could this technology be used to
upscale videos? I hope to answer that, and many more things, in future videos. I’ll
just end this with some terrifyingly upscaled faces. Have fun!

Only registered users can comment.

  1. Notice how, at 1:24, he said "EWWWW" while he shows the original image of Trump's face, not the upscaled version.

  2. The photos shows how AI is been trained, that means you can upgrade it by train more, and add more complex stuff, like AI face recognition that can upscale faces depence on real ones, or AI that can recognize and upscale number or letters so can fix fake letters or blur numbers , and AI consists sometimes on 2-3 AI 🤷‍♂️

  3. Start at 100×100 and upscale to 200×200, upscale result to 400×400, upscale result to 800×800.. Repeat until 4kx4k

  4. you could have made one hell of a jump scare by using that picture of the jolly miller. Just add one of those horrifyingly creepy faces in that dark window so its barely visible and i would have shit in my pants.

  5. I'm pretty sure my Samsung A50 does this all the time when shooting with the regular lens and it's impossible to turn off. My theory is they integrated it into the camera to keep the quality up with the high MP.

  6. Downscaling as a preprocessing step is equivalent to low pass filtering that removes high frequency noise. Good find.

  7. It's always the same with such applications of ML, impressive, but not close enough to be actually usable for anything.

  8. Now it seems to have fixed the face glitch, all photos I used are perfectly enhanced, almost no visible glitch, it's insanely crazy how it works well I'm scarred

  9. Couldn't you repeat this process with faces, take a picture of a face, scale it down, run the AI on it, then compare it to the original, AI that get's closest result wins,. Repeat until the issue is fixed ?

  10. I believe that computers can do certain things much better and faster than us. But I also believe that when developing computers we have no way to make them "better" than us in a way that would let them rule the world or something (you know what I mean). I believe that what limits us in the end must limit the computers somehow as well, since if we are not perfect, our creations wont be either.

    Im not a computer scientist or anything so if you are (or just know deep shit about them, thats fine too) please help me then.

    Am I wrong? Am i right? What's your opinion please tell 😀

  11. Still waiting for AI that can redraw over-exposed skies. HDR and photoshop are obvious solutions, but it would be great if I didn't have to take multiple pictures or paint in a skyreplasement. The task itself seem pretty trivial – something AI would be great at.

    Adding detail to shadows or fixing reflections that are burned would be another great task. For these, there usually is some reference available in the picture. especially acoustic guitars seem to act like mirrors under stagelights, resulting in pictures that are painful to grade.

  12. Shit, I loved how you included a photo of Zuckerbot when describing "uncanny valley" 🤣🤣 Sometimes I wonder myself if the kid's real or just an animated puppet.

  13. betcha we'll look at this phase of AI upscaling development the way we do at the classic 1980s vhs aesthetic today

  14. I don't think this AI interpolation better than “Photo zoom”.
    If you are interested in enlarge the images, you can compare the results and time-consuming, program size, etc.

  15. I have a question, If I take an image and copy it, Topaz ai will obviously give me identical results for both, If I then run the copy through Photoshop detail preserve 2 (which Sometimes interprets edges much better, clearer, sharper) when I put both images back in Gigipixel it will only interpret, re-render the original image.. Is there anything I can do to the photoshopped image to get the topaz ai to interpret, re-render it?? I don't get why the ai skips or ignores the image?

    Help Please, Any ideas would be great…

  16. So the next step would be to get the program to crawl the internet for public pictures with similar parameters (name, colors, etc) to derive the missing information

  17. Calling it upscaling is really not a good terminology. It implies that it is indeed doing a sci-fi "enhance".

    It's better to think of it as giving a low res picture to an artist and asking them to paint a large scale version. The artist knows that building have straight walls and what foliage looks like, and recognising these things in the image, paints a high res image based on that. But it isn't a modification of the original any more, it's a completely new image that was inspired by the original. This will have lots of uses, but it's dangerous if people think they are seeing "truth" rather than merely an AI's "opinion" of what the picture might have looked like were it captured in greater detail.

  18. The real shocker is: Each and every digital photo recorded with a Bayer filter sensor, that is, basically every digital photo the last twenty years, has some kind of “intelligent” guessing at the details and colours in the photo.
    The apparent cleanness of digital photos is a sham. And a lot of the detail is invented.
    That’s one reason why film looks better.

  19. This is awesome you could download a movie in low resolution making the file smaller and convert to a good resolution in real time, same thing could be apply to photo storage.

Leave a Reply

Your email address will not be published. Required fields are marked *