May I stress again that colors don't really exist? It's just coming from a fabricated resonance of 2 resonators (one for blue wavelength and one for red wavelength) in our eyes. The real world is black, white and shades of grey, only defined by the intensity of light.
But as far as I can see through my human eyes the decoding of the light combination I receive is translated in blue, green, yellow, etc. depending on the overall light intensity, its blue component level and its red component level.
Trying to translate this to sound you realize that our eye is a very poor sensor: its like hearing a noise and being able to say how loud it is, how much of a middle C it contains and how much of the next G key it contains - no more. Well the eye is actually very powerful as it can do this crude analysis for any point in the field of view at the same time, and that's a lot of information. But as a frequency analyzer it is not the best tool we have.
Going back to colors, the perception we have can be modeled in the YCrCb space (Y is the luminance or overall light intensity, Cr is the level of the red component of light and Cb the blue component). A zero value for both Cr and Cb correspond to all shade of grey (from black to white). Any non zero value of Cr or Cb will add hue to the perceived color. Please refer to
http://en.wikipedia.org/wiki/YCbCr if you're curious (Stephan is right, lots of good stuff there

)
Black and white belong to this space, as much as all shades of grey. And all of those are only singularities of the color space we can perceive.
If we were missing one of the eye pigments (extreme case of color blindness) we would see all shades of gray, including black and white and anything going from bright red to teal, but no blue, green yellow or orange in the case blue was missing, or from vivid blue to yellowish orange, but without purple, green, or red in the case red was missing.
If we had one more pigment type, TV's would need one more color to display beyond RGB. Its like seeing space in the 4 dimensions compared to what we are used to. Hard to comprehend.
All this to say (again): colors are only created by physiological benefits of 2 frequency resonators and have no physical reality. The real world is just grey. Our preceptions of pitch black and bright white are very subjective, reflecting our limitation as a light sensing machine. We see black when the light level is just too low for our sensor to feel. We see white when all our sensors (luminance, red an blue) are saturated in the narrow spectrum we are receptive to.
One more thing, from a very different angle: when a kid asks for the color of an object and s/he's told it's black or white he understand exactly what the answer is. A kid's learning about black, white, red, green or pink is the same. We all know what a black cat or a white T-shirt look like. These are perfect descriptors of the intensity of light and color saturation our eyes would receive when we see one. This are perfect descriptors of what our brain integrates and of how our data processing is functioning.
I think white and black belong to the color space
