Super-resolution update

Hi Folks,
This is my first post on this section of the forum. I posted on the Introduce Yourself section, so I won't bore you all again with the details. Anyway, my question - I came across some software called PhotoAcute. It's based on a concept called super-resolution which I've found references to all over the web. Does anyone here have any experience with this? I downloaded the trial of PhotoAcute and so far I am pretty impressed, but I'd like to know more. Is this the real deal, or just some fancy way of sharpening? Can I stop drooling over that 1DS MkII and just buy some software? I'd love to hear people's thoughts on this.

Thanks for any replies,
Paul

If you use good technique, low ISO & shoot from a tripod, I don't see that this software does much for you. DXO sounds better. If you really want more resolution, you could stitch or get into 4x5 for quite a bit less than a 1DsMkII.

Or buy Genuine Fractals. Good write-up on it in the latest Digital Photo Pro.

Paul,
If your goal is larger prints take a look at this thread on NPC, http://www.npcolorado.net/forums/showthread.php?t=196 , several software solutions are mentioned, including the free DOP method which seems to do as well as anything. The fact that Darren and others are getting 20x30 prints that look great from 8MP images is encouraging.
The super resolution stuff sounds interesting but I don't know that I want to be limited to a series of shots taken in continuous mode. What happens to moving elements? But it might be worth a look. Let us know what you find.

Also, I second the recommendation on DxO. If you haven't seen it, download the demo and run some RAW images through in full auto mode. It's pretty amazing. Full auto probably isn't what you'll end up using but it makes for a good demo.

The "super-resolution" processing algorithm has some intrigue. Unfortunately they don't support my camera, so I couldn't give it a quick test. Certainly their "before" and "after" results show obvious differences. However, I'd like to see a comparison of their output with just standard sharpening and contrast adjustments to a RAW file.

Can anyone explain the basic concepts behind this "super-resolution" algorithm. I'm puzzeld how processing multiple, identical pictures can increase spatial resolution. Each image has exactly the same information content.

Bill, I think they rely on very subtle differences from one exposure to the next to "fill in the blanks". Keep in mind that a digital photosite size is measured in microns so presumably even a light breeze or camera shake resulting from the mirror or shutter will produce some tiny amount of variance.

I'm going to give it a try this weekend. My camera and 2 lenses are supported and I need some desert time anyway

Guy

For those of you using DxO, do you use it in place of the Adobe Raw, etc? Is it your perfered RAW converter?

Alex

Alex,
I use DxO as my first workflow step with Raw files. I use it primarily for lens specific corrections (distortion, abberation, and some sharpening). To be honest, I've let it process batches of Raw files in full auto mode with good results but I'm working on refining what I let it do. I just started working with it a couple of months ago. I have DxO process the Raw files to DNG files that I import into Lightroom. I keep the Raw files in case I want to start over or work with them independent of DxO. In Lightroom I do the sorting, global corrections, and cropping.

I've found that DxO does a great job of cleaning up the shots in terms of perceived sharpness and detail, especially for wide angle shots with some distortion. I ended up putting DxO at the front of my workflow because I found if made a big enough difference in my images that what might have been throw-aways became candidates for keepers. (I'm not sure what that says about the quality of my images ).

I guess some might use it for the whole Raw conversion and just let it produce a Tiff or Jpeg, but I still prefer ACR or Lightroom for most of the adjustments.

Thanks for the input so far. For those of you interested in PhotoAcute, although it supports my 20D, it doesn't support my camera/lens combos but, I tried it anyway and the results are impressive. To Guy Tal; I had the same thoughts as you regarding the photosites. I look forward to hearing what you think after your desert time.
Paul

Another way to "gain" data from multiple exposures is that noise is not constant across those exposures; this feature of PhotoAcute will work even with camera/lens combos that aren't supported for superresolution, producing a much cleaner image, which allows sharpening techniques to work better. PhotoAcute recommends taking your shots at ISO 400 to decrease motion blur - they'll take care of the noise issue... They also recommend at least 4 consecutive shots, which is a significant amount of data from which to work.

Paul,

In what way is the trial version limited? Will it not save pictures, or does it add a watermark?

Sampling theory holds that if a analog signal is sampled at high enough frequency, the original waveform can be reproduced from the samples perfectly. I'm geussing the idea of PhotoAcute is this: since digital cameras sample at too low a frequency to capture every detail, slight offsets between a series of shots lets the software try to fill in the gaps between the sample points (pixels) of any individual image. Imagine two images, offset by half a pixel pitch. This is why I think a series of images with exactly the same framing will show little resolution improvement, but could show improved noise.

However, I believe that each pixel records an average of the light intensity across it's entire area, and is not a 'point sample'. Given this, I suspect the PhotoAcute process may have some success at filling in 'gaps' between pixels, to the extent that such gaps exist, but will not yield results better than those achieved through good technique: tripod mounted, mirror lock-up, etc. The PhotoAcute website says as much, "You will get clear image, as if you took the photograph using a tripod".

To Les - The trial is fully functional, but it doesn't let you save. I find this unfortunate since I can't do a really good side by side comparison with an image I've fullly processed myself. Also, they don't support tif input yet.
To Tony - My first thoughts were the same as yours, (I always use a tripod, mirror lock-up, low iso) so I was really surprised when I processed some images and they looked so much better than what I input. I'm one of those people who HAS to know how things work so I've been scratching my head over this one. The only explanation that makes sense to me right now is Guy Tal's comment on the photosites. I did email PhotoAcute and I'll share whatever they tell me with you folks.
Thanks for the comments everyone.
Paul

Hi Guys,
I emailed PhotoAcute support and asked how the product could work using a series of registered images taken on a tripod. Here is their response -
"The photos can be displaced a little because due to
mirror and shutter shake. The other effect may be the reduction of the
atmosphere fluctuations which are simply averaged on the resulting
image."

Paul

Can you show us examples?

Tony,
Unfortunately I can't post any examples. The limitation of the trial version is that you can't save. I originally posted here to try to get some more input since I didn't understand how this software could possibly work. I'm hesitant to pay $119 just to experiment. Maybe after I get my tax refund!
Paul

Paul, to get around that limitation just take a screen shot and post that instead. Should be enough to get a good idea of what is going on. You can even pan around and take several screen shots posting them all.

I'd be interested in seeing real life samples myself,

Craig

Craig,
I initially thought I wouldn't do just a screen shot since it might not look good, but what the heck.
The image on the left is from an unsharpened jpeg converted from RAW with Capture One LE. The image on the right is the output of PhotoAcute produced from an input of 5 jpegs. Other than the jpeg conversion and resizing in PS, I did nothing to either image, as you'll see when you find the dust spots (ugh!). There's definitely noise in the sky on the PhotoAcute version and some artifacts in both images that don't show up in the "real" versions. Noise is not as apparent if you process all in RAW (PhotoAcute will use RAW files as input. It uses the DNG converter).
That's it. (BTW, these are both heavily cropped)
Paul

The multi-image shot definitely looks clearer to my eyes; sharper crags, not so fuzzy on the pine boughs. Double the res in each direction is bound to produce some noise with only 5 images, especially pulling from JPEG instead of RAW. It also looks a bit more contrasty.

I might have to try it out, though I don't have many multi-image shots outside of HDR and pano captures.

Paul, thanks for the post with samples. I took them into PS and played around for a bit, but I was hampered somewhat by the jpg artifacts in your posted images. Anyway, the interesting thing I noticed was that the PhotoAcute image is missing some of the snow or it at least appears that way, but perhaps the snow was never really there and the multiple images made that apparent? To see what I mean load this image into PS and look at pixel position x=455, y=166. What happened to the snow here??? This happens all over your shot. A good way to make it really apparent is to create a duplicate of the background layer, line up the original and PhotoAcute versions, then turn the top layer off and on. Notice how the snow seems to shrink in almost all areas? It's a very interesting affect and not easily noticed until you do this.

Next, I tried to determine if this is really higher res or just a fancy sharpening technique by trying to simulate it with a really low capture sharpen of USM 80,.3,1 It seemed to look fairly identical at that point, but when I did a final sharpen of both and the PhotoAcute version look more detailed. Nothing mind boggling, but apparent none the less. It seems this might actually work somewhat, but to really test it we would need to have a real high res shot to compare it with to see if it's just noise or real additional resolution. In any case, it doesn't look like noise, so nobody would be the wiser.

Looking forward to what Guy finds out in his testing. This seems to hold some promise of getting more apparent resolution than your camera really has.

BTW, which version are you testing with?

Craig

Craig, now that you mention it, it's easy to see small snow-specks disappearing in the PA version. I noticed something similar on one of their samples; a small shadow was missing in their upsampled version; I put it down to differences in frames (since one of the other things the program is supposed to do is to fill in things that are moving through the view...). You'd have to look at the hi-res version to see if it's still there but not taking up as much space, or whether it got removed...

Craig,
Good eye! I reprocessed the images in PhotoAcute and don't see the missing snow problem anymore, but there are some strange artifacts at high magnification. I'm guessing the missing detail has something to do with what I did. If you guys are interested in trying the trial of PhotoAcute, I'll be happy to let you pull these images off my ftp site.
Just shoot me an email. BTW, I sent PhotoAcute an email and they asked to have a look at the images. I'll let you know what they say.
Thanks,
Paul

Well, I used 6 raw captures from the 5D to test it. In order for it to work with raw files you need to have the DNG converter on your system path.
Anyway, the results are very good and very clean. I tried to compare it to a single file with converted with ACR and enlarged to the same proportions in PS and the PhotoAcute file does seem to have a very slight edge in detail.
If they were selling the pro version for $50 it would be a no-brainer for me, but at $119 I'll have to think about it some more.

Guy

So, Guy, if you could have the software for FREE, do the results justify routinely shooting 6 exposures - for someone using a tripod and low iso.

Bill, I think it would be for me. I don't make a lot of images on a trip so making the extra exposures is not a big deal. Also, CF cards are cheap (I heard that you can now buy a 4GB card, fill it and throw it away for less than what it would cost you to buy slide film for that amount of images!).
It did take about 12 minutes to process the 6 images though so I won't use it for every shot.

Guy

Guy,
Thanks for the feedback. I was kinda hoping you would have said, "don't waste your money". They do offer a deal where if you help them make a profile for a new camera/lens combo, they'll give you a free license. Maybe I'll look into that. I hope your desert shooting went well. I had wind and blowing snow
Paul

Guy, could you be so kind to post some before/after sampels similar to Paul. I'd be curious if you see disappearing info in your images similar to what I saw with Paul's posted samples.

While getting higher res this way seems interesting, I'd think this would fall down miserably with any wind what so ever. Even with high ISO so individual frames don't show movement, the combination of several frames will none the less still show differences as trees/grass/water/clouds will not be in the same positions under windy conditions.

Craig

Craig,
The developers anticipated your concern. There are two settings that remove moving objects. One setting removes them completely, and the other lets you use moving objects from just the first image. I used this last setting in my photo post. The wind was blowing really hard that day and those trees were all over the place. Have you checked out their website? They have a cool demo photo where they actually remove people from an image.
Paul

Paul, thanks for the info. I could see removing moving people easy enough as the rest of the scene would be static, but you don't want to remove trees/grass/clouds from your shots. Nice to see that the app can use a single shot for moving objects if you desire and it seems to work as I didn't notice any blur on the trees in your shot.

Craig

Hmm, can't seem to find that reference to moving objects on their web site. Can you point me to the page it's on?

If it's not too much work I will probably sign up to create profiles for my Canon XTi and once they are created I will test PA out and compare a super-resolution result to a multiple frame frame pano of the same scene. That should be a good comparison point as the pano will have 2-2.5 times the resolution. I will probably download it and play with the other features that don't require a profile to see if they work as advertised in the meantime.

Craig

Craig,
It's under Examples -
http://www.photoacute.com/studio/examples/sandsculpture/index.html
I emailed them about making a profile for some of my lenses. It does require a lot of shots, but the free license is probably worth it.
I'd like to hear about your results.
I can't see this software outdoing a good multi-frame stitched shot, but who knows. I'd love to be wrong about that!
Paul

One setting removes them completely, and the other lets you use moving objects from just the first image.

Oh, I thought they had and example or something in their online manual that mentioned or showed the various settings in action. I couldn't find mention of the "use first image" for moving objects anywhere.

BTW, can I download your raw images to test with?

Craig

Craig,
Send me an email and I'll give you the ftp info. Also, I sent these images to PhotoAcute and they processed them and sent them back to me as a DNG file. It might take me a couple of days, but after I process them I'll give some feeback. I'm interested in doing a test like Guy did, where he did some up-rezzing.
Paul