Adding Support for New Raw Formats - Revision history

DrSlony at 21:27, 9 December 2019

2019-12-09T21:27:27Z

← Older revision		Revision as of 21:27, 9 December 2019
Line 1:		Line 1:
			<div class="pagetitle">Adding Support for New Raw Formats</div>

	== Introduction ==		== Introduction ==

DrSlony: rearranged, and rewrote some parts

2019-11-22T07:26:33Z

rearranged, and rewrote some parts

← Older revision		Revision as of 07:26, 22 November 2019
Line 1:		Line 1:
	== Introduction ==		== Introduction ==

	Supporting a raw format requires ~~generally two things~~:		Supporting a raw format requires the following:
	* Being able to decode the raw file, so that the program has access to the image data and metadata stored within each raw file. This is handled by a program embedded inside RawTherapee called "dcraw".		* Being able to decode the raw file, so that the program has access to the image data and metadata stored within each raw file. This is handled either by a program embedded inside RawTherapee called "dcraw", or by custom code.
	* Being able to interpret the image data. This can be further broken down into ~~more items~~:		* Being able to interpret the image data. This can be further broken down into:
	** Measuring the black ~~and white~~ levels of the decoded image data, as every camera's idea of what is the blackest black ~~and whitest white~~ is different. ~~This article covers this point~~.		** Measuring the "white levels" (and sometimes the "black levels") of the decoded image data, as every camera's idea of what is the whitest white (and sometimes the blackest black) is different. White levels are measured using photos called "white frames".
	** Determining where the ~~valid~~ image lies on the "canvas" - the "raw crop".		** Determining where the actual image lies on the raw canvas - the "raw crop".
	** ~~Correctly interpreting the colors. This is done by generating~~ an input ~~color~~ profile ~~(DCP or ICC), as described in the~~ "~~[[How_to_create_DCP_color_profiles\|How~~ to ~~Create DCP Color Profiles]]" article~~.		** Creating an "input profile" to accurately reproduce the colors.

	You will have to shoot the photos, but you do not heave to understand or carry out the measurement - we can do that for you.		You will have to shoot the photos, but you do not heave to understand or carry out the measurement - we can do that for you.

			=== Storing and Reading This Information ===

			RawTherapee looks for information regarding how to interpret the image data (the black and white levels, the color matrix, and some other details, but not the input profile) in three places:
			* In the dcraw code which is embedded inside RawTherapee,
			* In the raw file itself,
			* In a text file on your system which is installed with RawTherapee called <code>camconst.json</code>

			Information is gathered from all three places, and values from <code>camconst.json</code> are prioritized above those from other sources. There is an exception for the [[Color_Management#Input_Profile\|input color matrix]], in that if the raw file is in the DNG format and the <code>Software</code> Exif tag (<code>0x0131</code>) does not begin with the string <code>Adobe DNG Converter</code> and the file does contain a <code>ColorMatrix2</code> tag, then the value from this tag is prioritized.

			If you made any changes to <code>camconst.json</code> while RawTherapee was running, restart it for the changes to take effect.

			== White Levels ==

	What are black and white levels? A sensor is made of millions of tiny photo-sensitive elements called photosites or sensels. Each one measures the intensity of the light which falls upon it, and records that intensity as a number - the more light, the higher the number. The bulk of the raw file consists of these recorded measurements. Each photosite has a level below which it cannot sense any light - there might be some light falling upon it, but this light is too weak to register a signal. This is called the black level, and it is not always 0. There is also a level beyond which the photosite will not register a change in light intensity even if the light does keep getting brighter - this is the white level. A photosite which cannot record any brighter light is said to be fully saturated, and in post-processing this state is called clipping.		What are black and white levels? A sensor is made of millions of tiny photo-sensitive elements called photosites or sensels. Each one measures the intensity of the light which falls upon it, and records that intensity as a number - the more light, the higher the number. The bulk of the raw file consists of these recorded measurements. Each photosite has a level below which it cannot sense any light - there might be some light falling upon it, but this light is too weak to register a signal. This is called the black level, and it is not always 0. There is also a level beyond which the photosite will not register a change in light intensity even if the light does keep getting brighter - this is the white level. A photosite which cannot record any brighter light is said to be fully saturated, and in post-processing this state is called clipping.

			The white levels are measured based on completely overexposed photos called "white frames".

	Some camera models use the same white and black levels regardless of other settings, while for other models these levels depend on other factors, such as ISO sensitivity. We need photos from across the whole range of ISO values to determine this.		Some camera models use the same white and black levels regardless of other settings, while for other models these levels depend on other factors, such as ISO sensitivity. We need photos from across the whole range of ISO values to determine this.
Line 18:		Line 33:
	The white levels for some camera models change depending on the aperture, but generally this only happens for wide-open apertures. To avoid this being a problem, set the aperture to f/5.6 or higher unless instructed otherwise.		The white levels for some camera models change depending on the aperture, but generally this only happens for wide-open apertures. To avoid this being a problem, set the aperture to f/5.6 or higher unless instructed otherwise.

	== ~~What and~~ How to Shoot ==		For more documentation detailing the required photos and instructions how to measure them, read the comments inside the <code>camconst.json</code> file:
			https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json

			=== How to Shoot White Frames ===

	Shoot raw photos in manual (M) mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.		Shoot raw photos in manual (M) mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.
Line 26:		Line 44:
	It does not matter what lens you use, but it will be easier to make the whole image overexposed if you do not use a wide angle lens.		It does not matter what lens you use, but it will be easier to make the whole image overexposed if you do not use a wide angle lens.

	== Needed Photo Sets ==		=== Needed Photo Sets ===

	We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets. All sets involve photographing completely overexposed white frames.		We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets. All sets involve photographing completely overexposed white frames.
Line 41:		Line 59:
	Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with LENR disabled weigh 234MB but if you ZIP them you get a 1MB file.		Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with LENR disabled weigh 234MB but if you ZIP them you get a 1MB file.

	== Renaming ==		=== Renaming ===

	In order to simplify working with these images, the filenames should segregate the photos by LENR, aperture and ISO. ExifTool can rename them automatically:		In order to simplify working with these white frame images, the filenames should segregate the photos by LENR, aperture and ISO. ExifTool can rename them automatically:

	<code>exiftool '-FileName<${make}_${model}_${LongExposureNoiseReduction}_${aperture}_${iso}%-c.%le' dir</code>		<code>exiftool '-FileName<${make}_${model}_${LongExposureNoiseReduction}_${aperture}_${iso}%-c.%le' dir</code>

	== ~~More Information~~ ==		== Raw Crop ==

	~~For documentation detailing the required~~ photos ~~and instructions how to measure them, read the comments inside the camconst.json file:~~		The raw crop can be determined from any photograph, no extra photos are needed.
	~~https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst~~.~~json~~

	== ~~Storing and Reading This Information~~ ==		== Input Profile ==

	~~RawTherapee looks for information regarding how~~ to ~~interpret the image data (the black and white levels, the color matrix, and some other details) in three places:~~		An input profile is required in order to reproduce colors accurately. One is needed per camera model. Read the "[[How_to_create_DCP_color_profiles\|How to Create DCP Color Profiles]]" article to learn about the types of input profiles and how to shoot photos of a color target so that we may create an input profile for your camera model.
	* In the dcraw code which is embedded inside RawTherapee,
	* In the raw file itself,
	* In a text file on your system which is installed with RawTherapee called <code>camconst.~~json</code>~~

	~~Information~~ is ~~gathered from all three places, and values from <code>camconst~~.~~json</code> are prioritized above those from other sources. There is an exception for~~ the [[~~Color_Management#Input_Profile~~\|~~input color matrix~~]]~~, in that if~~ the ~~raw file is in the DNG format~~ and ~~the <code>Software</code> Exif tag (<code>0x0131</code>) does not begin with the string "Adobe DNG Converter" and the file does contain~~ a ~~<code>ColorMatrix2</code> tag, then the value from this tag is prioritized.~~

	~~If you made any changes to <code>camconst.json</code> while RawTherapee was running, restart it~~ for ~~the changes to take effect~~.

DrSlony at 22:31, 11 October 2019

2019-10-11T22:31:17Z

← Older revision		Revision as of 22:31, 11 October 2019
Line 19:		Line 19:

	== What and How to Shoot ==		== What and How to Shoot ==

	Shoot raw photos in manual (M) mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.		Shoot raw photos in manual (M) mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.

Line 53:		Line 54:
	== Storing and Reading This Information ==		== Storing and Reading This Information ==

	RawTherapee looks for information regarding how to interpret the image data (the black and white levels and some other details) in three places:		RawTherapee looks for information regarding how to interpret the image data (the black and white levels, the color matrix, and some other details) in three places:
	* In the dcraw code which is embedded inside RawTherapee,		* In the dcraw code which is embedded inside RawTherapee,
	* In the raw file itself,		* In the raw file itself,
	* In a text file on your system which is installed with RawTherapee called <code>camconst.json</code>		* In a text file on your system which is installed with RawTherapee called <code>camconst.json</code>

	~~The needed information~~ is gathered from all three places, and <code>camconst.json</code> is prioritized ~~above~~ the ~~others~~.		Information is gathered from all three places, and values from <code>camconst.json</code> are prioritized above those from other sources. There is an exception for the [[Color_Management#Input_Profile\|input color matrix]], in that if the raw file is in the DNG format and the <code>Software</code> Exif tag (<code>0x0131</code>) does not begin with the string "Adobe DNG Converter" and the file does contain a <code>ColorMatrix2</code> tag, then the value from this tag is prioritized.

			If you made any changes to <code>camconst.json</code> while RawTherapee was running, restart it for the changes to take effect.

DrSlony: /* Needed Photo Sets */

2019-10-01T19:39:55Z

Needed Photo Sets

← Older revision		Revision as of 19:39, 1 October 2019
Line 27:		Line 27:
	== Needed Photo Sets ==		== Needed Photo Sets ==

	We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets.		We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets. All sets involve photographing completely overexposed white frames.

	The sets:		The sets:
	# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.		# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.
	# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.		# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.
	# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.		# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo as described above but using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.

	At the very least, you should end up with a series of about 8 photos from point 1. It is recommended that you take photos for both points 1 and 2, leading to 16 or more photos, plus the one raw scaling test photo from point 3. If it is found that your camera performs raw scaling, you could additionally take the needed series described in point 3, but since this could potentially mean many photos (over 50) it is not expected.		At the very least, you should end up with a series of about 8 photos from point 1. It is recommended that you take photos for both points 1 and 2, leading to 16 or more photos, plus the one raw scaling test photo from point 3. If it is found that your camera performs raw scaling, you could additionally take the needed series described in point 3, but since this could potentially mean many photos (over 50) it is not expected.

DrSlony at 21:05, 4 September 2019

2019-09-04T21:05:54Z

← Older revision		Revision as of 21:05, 4 September 2019
Line 26:		Line 26:

	== Needed Photo Sets ==		== Needed Photo Sets ==

	We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets.		We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets.

Line 38:		Line 39:

	Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with LENR disabled weigh 234MB but if you ZIP them you get a 1MB file.		Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with LENR disabled weigh 234MB but if you ZIP them you get a 1MB file.

			== Renaming ==

			In order to simplify working with these images, the filenames should segregate the photos by LENR, aperture and ISO. ExifTool can rename them automatically:

			<code>exiftool '-FileName<${make}_${model}_${LongExposureNoiseReduction}_${aperture}_${iso}%-c.%le' dir</code>

	== More Information ==		== More Information ==

	For documentation detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:		For documentation detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:
	https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json		https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json

	== Storing and Reading This Information ==		== Storing and Reading This Information ==

	RawTherapee looks for information regarding how to interpret the image data (the black and white levels and some other details) in three places:		RawTherapee looks for information regarding how to interpret the image data (the black and white levels and some other details) in three places:
	* In the dcraw code which is embedded inside RawTherapee,		* In the dcraw code which is embedded inside RawTherapee,

DrSlony: /* Needed Photo Sets */

2019-09-04T20:56:09Z

Needed Photo Sets

← Older revision		Revision as of 20:56, 4 September 2019
Line 29:		Line 29:

	The sets:		The sets:
	# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second~~, using an aperture of f/8~~. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.		# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.
	# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.		# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.
	# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.		# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.

DrSlony at 12:09, 7 September 2018

2018-09-07T12:09:38Z

← Older revision		Revision as of 12:09, 7 September 2018
Line 29:		Line 29:

	The sets:		The sets:
	# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second, using an aperture of f/8. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.		# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second, using an aperture of f/8. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.
	# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.		# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.
	# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.		# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.

DrSlony: Revised, updated

2018-09-02T12:32:23Z

Revised, updated

← Older revision		Revision as of 12:32, 2 September 2018
Line 1:		Line 1:
	~~Adding perfect support for new~~ raw ~~formats in~~ RawTherapee ~~is easy~~. ~~You~~ can ~~do it yourself~~, or ~~you can take~~ the ~~needed~~ photos ~~and send them~~ to ~~us so~~ we can do the measurements ~~ourselves~~ and ~~add support for your camera~~.		== Introduction ==

			Supporting a raw format requires generally two things:
			* Being able to decode the raw file, so that the program has access to the image data and metadata stored within each raw file. This is handled by a program embedded inside RawTherapee called "dcraw".
			* Being able to interpret the image data. This can be further broken down into more items:
			** Measuring the black and white levels of the decoded image data, as every camera's idea of what is the blackest black and whitest white is different. This article covers this point.
			** Determining where the valid image lies on the "canvas" - the "raw crop".
			** Correctly interpreting the colors. This is done by generating an input color profile (DCP or ICC), as described in the "[[How_to_create_DCP_color_profiles\|How to Create DCP Color Profiles]]" article.

			You will have to shoot the photos, but you do not heave to understand or carry out the measurement - we can do that for you.

			What are black and white levels? A sensor is made of millions of tiny photo-sensitive elements called photosites or sensels. Each one measures the intensity of the light which falls upon it, and records that intensity as a number - the more light, the higher the number. The bulk of the raw file consists of these recorded measurements. Each photosite has a level below which it cannot sense any light - there might be some light falling upon it, but this light is too weak to register a signal. This is called the black level, and it is not always 0. There is also a level beyond which the photosite will not register a change in light intensity even if the light does keep getting brighter - this is the white level. A photosite which cannot record any brighter light is said to be fully saturated, and in post-processing this state is called clipping.

	~~== Introduction ==~~		Some camera models use the same white and black levels regardless of other settings, while for other models these levels depend on other factors, such as ISO sensitivity. We need photos from across the whole range of ISO values to determine this.

			Some cameras have built-in noise reduction, often called LENR - Long Exposure Noise Reduction. It could affect the white and black levels. If you enable it, typically it only kicks-in when the exposure time is over 1s. The steps explained further on will explain when to turn it on and off.

			The white levels for some camera models change depending on the aperture, but generally this only happens for wide-open apertures. To avoid this being a problem, set the aperture to f/5.6 or higher unless instructed otherwise.

	~~Reading a~~ raw ~~file properly requires knowing some things about it~~, ~~and RawTherapee looks for this information in three places:~~		== What and How to Shoot ==
	* In the ~~embedded dcraw code~~,		Shoot raw photos in manual (M) mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.
	* In the raw file itself,
	* In a text file called camconst.~~json~~

	~~The needed information is gathered from~~ all ~~three places~~, ~~and camconst~~.~~json~~ is the ~~most important source of it. If any information overlap occurs, camconst.json takes priority~~.		Each photo must be completely overexposed all across the frame. As such, it does not matter what you shoot since everything will be white anyway, but its easiest to achieve this while pointing the camera at the sky or at a white light bulb. It does not matter whether the sky is sunny or overcast, but don't point it at the sun as you might damage the sensor.

	~~camconst.json allows one to set the following:~~		It does not matter what lens you use, but it will be easier to make the whole image overexposed if you do not use a wide angle lens.
	* Color matrices for [http://en.wikipedia.org/wiki/Illuminant_D65 Illuminant D65], ~~in dcraw format,~~
	* white and black levels,
	* raw crop size and offset to ~~remove spurious rows and columns,~~
	* masked area size and offset from which black levels can be calculated
	~~for each combination of color channel, ISO and aperture for a given camera~~ make ~~and model. For this reason, camconst.json serves both as~~ a ~~place for quickly adding detailed support for new raw formats as well as a place for improving support for cameras already but imperfectly handled by dcraw~~.

	== Needed ~~Photos~~ ==		== Needed Photo Sets ==
	~~To make~~ the ~~required measurements for perfect~~ support, you ~~will~~ need to ~~take several series of photos with specific settings:~~		We have broken down the image requirements into three sets, where each subsequent set would improve the quality of support, but would also require more effort from you and from us. In most cases, you only need to photograph the first two sets.

	* Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped. Of course shoot in raw mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.		The sets:
	# If your camera has ~~built-in noise reduction~~, turn it ~~off~~. Take a series of photos ~~as described above~~, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of ~~0.5s~~, using an aperture of f/8. As an example, ~~for a typical camera~~ you ~~would~~ end up with about 8 photos~~: ISO100~~, ~~ISO200~~, ~~ISO400~~, ~~ISO800~~, ~~ISO1600~~, ~~ISO3200~~, ~~ISO6400, ISO12800~~. ~~New~~ cameras ~~often~~ include intermediate ISO values, e.g. ISO160, ISO320, ~~etc. If your camera includes such ISO~~ values~~, it is important that you shoot them~~ as well.		# If your camera has LENR, turn it OFF. Set the aperture to f/5.6 or higher. Take a series of photos, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 1 second, using an aperture of f/8. As an example, you could end up with about 8 photos for ISO 100, 200, 400, 800, 1600, 3200, 6400 and 12800. Most cameras include intermediate ISO values, e.g. ISO160 or ISO320, so if you wanted to improve white level accuracy you would need to photograph these intermediate values as well.
	# If your camera has ~~built-in noise reduction~~, turn it on. Take a second series of photos as described above~~, one photo for every ISO value your camera supports 1 stop apart,~~ making sure that the exposure time in all cases is at least 2 seconds, not less~~, using an aperture of f/5.6~~. That's another ~~about~~ 8 photos.		# If your camera has LENR, turn it ON. Take a second series of photos, as described above but this time making sure that the exposure time in all cases is at least 2 seconds, not less. That's another 8 or more photos. In most cases these two sets should be enough.
	# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with ~~long exposure noise reduction~~ turned ~~off~~, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos ~~at every ISO value one stop apart, with~~ an exposure time less than ~~0.5s~~, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.		# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with LENR turned OFF, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos using an exposure time of less than 1 second, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.

	At the very least, you should end up with a series of about 8 photos from point 1. It is recommended that you take photos for both points 1 and 2, leading to ~~about~~ 16 photos, plus the one raw scaling test photo from point 3. If it is found that your camera performs raw scaling, you could additionally take the needed series described in point 3, but since this could potentially mean many photos (~~e.g.~~ 50 ~~or more~~) it's not expected.		At the very least, you should end up with a series of about 8 photos from point 1. It is recommended that you take photos for both points 1 and 2, leading to 16 or more photos, plus the one raw scaling test photo from point 3. If it is found that your camera performs raw scaling, you could additionally take the needed series described in point 3, but since this could potentially mean many photos (over 50) it is not expected.

	Compress all these photos, upload them to [http://filebin.net/ filebin.net] and send us the full link either through our [https://github.com/Beep6581/RawTherapee/issues/new GitHub] page or in the [http://rawtherapee.com/forum Forum].		Compress all these photos, upload them to [http://filebin.net/ filebin.net] and send us the full link either through our [https://github.com/Beep6581/RawTherapee/issues/new GitHub] page or in the [http://rawtherapee.com/forum Forum].

	Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with ~~long-exposure noise reduction~~ disabled weigh 234MB but if you ZIP them you get a 1MB file.		Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with LENR disabled weigh 234MB but if you ZIP them you get a 1MB file.

	== ~~Details~~ ==		== More Information ==
	For ~~exact~~ documentation, detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:		For documentation detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:
	https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json		https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json

			== Storing and Reading This Information ==
			RawTherapee looks for information regarding how to interpret the image data (the black and white levels and some other details) in three places:
			* In the dcraw code which is embedded inside RawTherapee,
			* In the raw file itself,
			* In a text file on your system which is installed with RawTherapee called <code>camconst.json</code>

			The needed information is gathered from all three places, and <code>camconst.json</code> is prioritized above the others.

DrSlony: Updated camconst.json link from master to dev

2017-05-13T23:00:27Z

Updated camconst.json link from master to dev

← Older revision		Revision as of 23:00, 13 May 2017
Line 33:		Line 33:
	== Details ==		== Details ==
	For exact documentation, detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:		For exact documentation, detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:
	https://github.com/Beep6581/RawTherapee/blob/~~master~~/rtengine/camconst.json		https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/camconst.json

DrSlony: Added note on intermediate ISOs and compression

2017-01-22T14:49:52Z

Added note on intermediate ISOs and compression

← Older revision		Revision as of 14:49, 22 January 2017
Line 21:		Line 21:

	* Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped. Of course shoot in raw mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.		* Each photo must be completely overexposed everywhere! Do this by pointing your camera at a bright light (e.g. the sky, a lamp), zooming in as needed, and increasing exposure time until everything is absolutely clipped. Of course shoot in raw mode. If your camera has several raw modes, use the full one, uncropped, lossless compression if possible.
	# If your camera has built-in noise reduction, turn it off. Take a series of photos as described above, one photo for every ISO value your camera supports ~~1 stop apart~~, making sure not to exceed an exposure time of 0.5s, using an aperture of f/8. As an example, for a typical ~~modern~~ camera you would end up with about 8 photos: ISO100, ISO200, ISO400, ISO800, ISO1600, ISO3200, ISO6400, ISO12800.		# If your camera has built-in noise reduction, turn it off. Take a series of photos as described above, one photo for every ISO value your camera supports, making sure not to exceed an exposure time of 0.5s, using an aperture of f/8. As an example, for a typical camera you would end up with about 8 photos: ISO100, ISO200, ISO400, ISO800, ISO1600, ISO3200, ISO6400, ISO12800. New cameras often include intermediate ISO values, e.g. ISO160, ISO320, etc. If your camera includes such ISO values, it is important that you shoot them as well.
	# If your camera has built-in noise reduction, turn it on. Take a second series of photos as described above, one photo for every ISO value your camera supports 1 stop apart, making sure that the exposure time in all cases is at least 2 seconds, not less, using an aperture of f/5.6. That's another about 8 photos.		# If your camera has built-in noise reduction, turn it on. Take a second series of photos as described above, one photo for every ISO value your camera supports 1 stop apart, making sure that the exposure time in all cases is at least 2 seconds, not less, using an aperture of f/5.6. That's another about 8 photos.
	# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with long exposure noise reduction turned off, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos at every ISO value one stop apart, with an exposure time less than 0.5s, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.		# Some cameras scale raw values for larger apertures, particularly Canon and Nikon models. The only way to know whether your camera does this for sure is to take a photo and measure it. Take one photo using your lens's widest aperture, e.g. f/1.7, at ISO100 with long exposure noise reduction turned off, and send it to us along with the rest of the shots. If we detect that there is raw scaling (or if you detect it yourself if you do your own measurements) then we will ask you shoot a series of photos at every ISO value one stop apart, with an exposure time less than 0.5s, from the widest aperture your lens supports down every 1/3 of a stop until such an aperture where raw scaling is no longer performed. This could mean many photos. Handling raw scaling caused by large apertures is not very important so don't feel daunted by it, you don't need to do it even if your camera does do raw scaling, but if you have the time and bandwidth then it would be better to check for it.
Line 28:		Line 28:

	Compress all these photos, upload them to [http://filebin.net/ filebin.net] and send us the full link either through our [https://github.com/Beep6581/RawTherapee/issues/new GitHub] page or in the [http://rawtherapee.com/forum Forum].		Compress all these photos, upload them to [http://filebin.net/ filebin.net] and send us the full link either through our [https://github.com/Beep6581/RawTherapee/issues/new GitHub] page or in the [http://rawtherapee.com/forum Forum].

			Completely clipped photos can have amazing compression, don't forget to compress them (7-Zip, ZIP, bzip2, whatever) before uploading! As an example, 10 completely clipped Sony 7M2 raw files with long-exposure noise reduction disabled weigh 234MB but if you ZIP them you get a 1MB file.

	== Details ==		== Details ==
	For exact documentation, detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:		For exact documentation, detailing the required photos and instructions how to measure them, read the comments inside the camconst.json file:
	https://github.com/Beep6581/RawTherapee/blob/master/rtengine/camconst.json		https://github.com/Beep6581/RawTherapee/blob/master/rtengine/camconst.json