PROPixx Demo 7 – Understanding CLUTs, M16 Mode, and Color Transparency

Color look-up tables (CLUTs) are a part of the video pipeline. You can have CLUTs for color correction, gamma correction, color removal, etc. A CLUT is usually a 256 by 3 matrix, where each column represents a color channel (Red, green, blue (RGB)) and each row represents an operation or a new color value for that color. A non modifying CLUT is defined as follow in MatLab: linspace(0, 1, 256)' * [1, 1, 1]. Colors in MatLab are usually represented as a double going from 0 (no color) to 1 (full intensity). Colors usually have an 8 bits representation, and therefore can vary from 0 to 1 in 256 steps.

In this demo, we use CLUTs to define a transparency color and to have different colors on the console display from the main display.

The transparency color is useful if you want to display a message to yourself but having it invisible to the subject. Different colors on the two displays can be useful if you want different shades or a different gamma on the two displays.

As stated before, colors are usually represented using 8 bits of information. Our device are capable of displaying 10-12 bits of colors. However, a video signal can only send 24 bits (8 per color) of information. To overcome this limitation, we have to manipulate the data and interpret it differently in our device.

In mode M16, we calculate a 16-bit shade of gray and send that information using the 8 bits of red and 8 bits of green. The VPixx device combines it to display a 10-12 bit closest value if the blue component is zero. If the blue component of the pixel is not zero, then it is used as an index to a CLUT to display the pixel as an overlay. That overlay will have the color defined in the CLUT.

There are two look-up tables on our device, one for the main device and one for the console display. This means you can have different color overlays on the console and on the main device.

It is also possible to define a transparency color to the device. If the CLUT table has that color at the given blue index, there will not be an overlay, the pixel will be drawn as if the blue value had been zero (using the gray color provided by the combined Red and Green).

This demo uses the PsychImaging pipeline and sets it up to use 32 bits precision for calculations, enables the M16 video mode with overlay, and sets a simple gamma correction. We specify the color correction gamma to be 1/2.2, such that this example has a gamma of 1.

Screen('LoadNormalizedGammaTable', win, linspace(0, 1, 256)' * [1, 1, 1]); is run to prevent a bug on MAC which always require a CLUT loaded on the graphic card.

Since we set up the M16 mode with an overlay, we must create the overlay and set its CLUT. Datapixx('SetVideoClutTransparencyColor', transparencyColor); sets up the device to use a certain color as transparent (green in this case), and we must enable the transparency color mode Datapixx('EnableVideoClutTransparencyColorMode');.

To use this transparency color in M16, we create two CLUTs that only contain the transparency color. clutTestDisplay = repmat(transparencyColor, [256,1]);. After that, we must change the index of the color we do not want to be transparent to a color we choose. clutTestDisplay(242:246,:) = repmat([1, 0, 0], [5,1]); defines that anything with a blue index from 243 to 247** will be drawn as red ([1,0,0]).

We must then load the CLUTs on the device Datapixx('SetVideoClut', [clutTestDisplay;clutConsoleDisplay]); (We load the main display CLUT then the console CLUT).

The range of color that was changed on the main display CLUT will not be transparent, but it will be transparent on the console display, since the range was not modified and it leads to the transparent color.

** On some devices, there is rescaling of the color, in which case we use a 242 to 246 range instead of just the value 247. This varies from system to system and, for example, our test window machine for 247 we had to set the color to 246, while on the Mac it was 245. With range 242 to 246, you should aim to use color 243.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
function DatapixxM16Demo()
% DatapixxM16Demo()
%
% A demonstration of the DATAPixx M16 (16-bit monochrome) video mode using the PsychToolbox imaging pipeline.
%
% History:
%
% July 21, 2010  paa     Written
% Oct 29, 2014   dml     Revised

AssertOpenGL;
% Screen('Preference', 'SkipSyncTests', 1);

% Configure PsychToolbox imaging pipeline to use 32-bit floating point numbers.
% Our pipeline will also implement an inverse gamma mapping to correct for display gamma.
PsychImaging('PrepareConfiguration');
PsychImaging('AddTask', 'General', 'FloatingPoint32Bit');
PsychImaging('AddTask', 'General', 'EnableDataPixxM16OutputWithOverlay');
PsychImaging('AddTask', 'FinalFormatting', 'DisplayColorCorrection', 'SimpleGamma');

% Open our window.
% We are assuming that the DATAPixx is connected to the highest number screen.
% If it isn't, then assign screenNumber explicitly here.
screenNumber=max(Screen('Screens'));
oldVerbosity = Screen('Preference', 'Verbosity', 1);   % Don't log the GL stuff
[win, winRect] = PsychImaging('OpenWindow', screenNumber);
Screen('Preference', 'Verbosity', oldVerbosity);
winWidth = RectWidth(winRect);
winHeight = RectHeight(winRect);

% Specify the window's inverse gamma value to be applied in the imaging pipeline
gamma = 2.2;
PsychColorCorrection('SetEncodingGamma', win, 1/gamma);

% Ensure that the graphics board's gamma table does not transform our pixels
Screen('LoadNormalizedGammaTable', win, linspace(0, 1, 256)' * [1, 1, 1]);

% define a 2D plaid with 100% contrast and 256 pixel period, with value in the range 0-1.
% Make a 32-bit floating point monochrome texture out of it.
[wx,wy] = meshgrid(1:winWidth, 1:winHeight);

plaidMatrix = (sin(wx*pi/128) + sin(wy*pi/128)) / 4 + 0.5;
plaidTexture = Screen('MakeTexture', win, plaidMatrix, [], [], 2);

% Draw the floating point texture.
% Specify filter mode = 0 (nearest neighbour), to ensure that GL doesn't interpolate pixel values.
Screen('DrawTexture', win, plaidTexture, [], [], [], 0);

% DATAPixx overlay window can hold a seperate 255-colour image.  We'll use a blue ramp.
overlay = PsychImaging('GetOverlayWindow', win);
% Screen('LoadNormalizedGammaTable', win, linspace(0, 1, 256)' * [0, 0, 1], 2);

% We'll arbitrarily use full green as a CLUT's "transparent" color
transparencyColor = [0, 1, 0];
Datapixx('Open');
Datapixx('SetVideoClutTransparencyColor', transparencyColor);
Datapixx('EnableVideoClutTransparencyColorMode');
Datapixx('RegWr');

% On some systems (Win?) LoadNormalizedGammaTable doesn't support 512 CLUT entries,
% so we'll use our own CLUT load function.
clutTestDisplay = repmat(transparencyColor, [256,1]);   % By default, all overlays are transparent
clutConsoleDisplay = repmat(transparencyColor, [256,1]);   % By default, all overlays are transparent

% ! ON WINDOWS, DrawFormattedText scales the color by 255/256, therefore
% the color is off by 1 for the upper half of the CLUT 
% On OS-X, DrawFormattedText seems to apply a grossly non-linear mapping
% between the argument intensity and the actual draw intensity.
% Other draw commands like DrawRect do not seem to show this bug.
% For the purposes of this demo, we will draw the text in the center of a
% 5-colour span, at the top of the 256-entry CLUT.
% This seems to work for all systems tested so far.
clutTestDisplay(242:246,:) = repmat([1, 0, 0], [5,1]);   % Items drawn with 255 show on test display as blue % FOR MAC
clutTestDisplay(252:256,:) = repmat([0, 0, 1], [5,1]);   % Items drawn with 255 show on test display as blue % FOR MAC

clutConsoleDisplay(247:251,:) = repmat([1, 1, 0], [5,1]);   % Items drawn with 255 show on test display as blue % FOR MAC
clutConsoleDisplay(252:256,:) = repmat([0, 0, 1], [5,1]);   % Items drawn with 255 show on test display as blue % FOR MAC

Datapixx('SetVideoClut', [clutTestDisplay;clutConsoleDisplay]);

% Draw some text onto overlay window
Screen('FillRect', overlay, 0);
Screen('TextSize', overlay, 36);
Screen('Preference', 'TextAntiAliasing', 0);    % Overlay looks best w/o antialiasing
%DrawFormattedText(overlay, 'Test Display', 'center', 40, 252);      % Overlay only visible on test display
%DrawFormattedText(overlay, 'Console Display', 'center', 80, 253);   % Overlay only visible on console display
%DrawFormattedText(overlay, 'DATAPixx 16-bit monochrome demo\nHit any key to exit.', 'center', 'center', 255);

DrawFormattedText(overlay, 'Test Display', 'center', 40, 243);      % Overlay only visible on test display
DrawFormattedText(overlay, 'Console Display', 'center', 80, 248);   % Overlay only visible on console display
DrawFormattedText(overlay, 'DATAPixx 16-bit monochrome demo\nHit any key to exit.', 'center', 'center', 253);

% Show resulting image, wait for keystroke, then terminate demo
Screen('Flip', win);
KbStrokeWait;
RestoreCluts;       % Restore any system gamma tables we modified
Datapixx('DisableVideoClutTransparencyColorMode');
Datapixx('RegWr');
Screen('CloseAll');
return;