Question: This is an important slide. Why is there a need for "deomosaicing" the output of an image sensor?
kapalani
The raw output from the Bayer filter only contains responses of each pixel to a particular color. So if you didn't do the demosaicing then you'll have an image where every other pixel has only a shade of green, and each pixel in every other row having a shade of red or a shade of blue respectively. A good example to understand the need for demosaicing is I think if we are trying to take a picture of a pure blue object, if we didn't do demosaicing, then our image would be more black than blue with all the surrounding pixels not responding at all!
Wikipedia also has a really nice image about the intermediate images
Good! But let's be a little more precise about what you mean by "color". Really the raw output of the sensor (when fronted by a Bayer filter) contains the response of the pixel to the input spectrum, where the response curve is determined by the Bayer filter. An example of the three response curves was shown on slide 7. Note how these camera pixel response curves bare some similarity to the response curves of the human eye's cones.
Further comments about response curves are given here.
kapalani
When it says find and avoid interpolation across edges is it something like the following scenario. Say for interpolating blue's we had the following values
255
255 ?? 50
255
It would seem like the column is a blue edge, but it could also be a part of a lighter blue shade?
kayvonf
@kapalani: The pattern you drew is the green pattern. Not blue. Let's draw a blue pattern:
It looks an awful lot like there might be a edge here. So when interpolating blue values onto aaa, bbb, and ccc it might be better to use 255 or 0, rather than some interpolation of the two. However, for yyy and zzz the measurements suggest that intepolation of the surrounding blue values might very likely give us the values the camera would have been measured at these locations.
Question: This is an important slide. Why is there a need for "deomosaicing" the output of an image sensor?
The raw output from the Bayer filter only contains responses of each pixel to a particular color. So if you didn't do the demosaicing then you'll have an image where every other pixel has only a shade of green, and each pixel in every other row having a shade of red or a shade of blue respectively. A good example to understand the need for demosaicing is I think if we are trying to take a picture of a pure blue object, if we didn't do demosaicing, then our image would be more black than blue with all the surrounding pixels not responding at all!
Wikipedia also has a really nice image about the intermediate images
https://en.wikipedia.org/wiki/Bayer_filter#/media/File:Colorful_spring_garden_Bayer.png
Good! But let's be a little more precise about what you mean by "color". Really the raw output of the sensor (when fronted by a Bayer filter) contains the response of the pixel to the input spectrum, where the response curve is determined by the Bayer filter. An example of the three response curves was shown on slide 7. Note how these camera pixel response curves bare some similarity to the response curves of the human eye's cones.
Further comments about response curves are given here.
When it says find and avoid interpolation across edges is it something like the following scenario. Say for interpolating blue's we had the following values
It would seem like the column is a blue edge, but it could also be a part of a lighter blue shade?
@kapalani: The pattern you drew is the green pattern. Not blue. Let's draw a blue pattern:
It looks an awful lot like there might be a edge here. So when interpolating blue values onto
aaa,bbb, andcccit might be better to use 255 or 0, rather than some interpolation of the two. However, foryyyandzzzthe measurements suggest that intepolation of the surrounding blue values might very likely give us the values the camera would have been measured at these locations.