Image Signatures

For now, one should think of the attribute as color. The A_I are points in a color space, the P_I are points in the image plane, and the W_I are areas in the image. An example is shown below.

In general, the signel (signature element) ((A_I,P_I),W_I) indicates the presence of a region of color A_I with area W_I and centroid P_I. In the above image, we see, for example, that there is yellow (A₄) region at P₄ with area W₄ (perhaps W₄=2% of the total image area), and purple regions at P₁ and P₃ with areas W₁ and W₃, respectively.

The reason that attribute is used instead of color in the above description is that the SEDL framework can also be applied to the shape pattern problem. When this is done, the attribute used by the system is the orientation of ink along curves in an image. In the shape case, the signel ((A_I,P_I),W_I) indicates the presence of a piece of an image curve with length W_I, average orientation A_I, and average position P_I.

Distributions in Attribute x Position space are used by the initial placement and verification phases. Both of these phases use position information. In the color pattern problem setting, the initial placement phase needs to know whether a uniform color region occurs inside or outside of a rectangle. This allows the system to compute the local image color histogram inside of a rectangular region. The verification and refinement phase uses absolute region positions to check that an image area is visually similar to the query (which may not be the case if there is simply a good match between color histograms), and to improve its estimate of the scale, orientation, and location of the pattern occurrence.

The scale estimation phase does not use the positions of the colors within the image. The inputs to the scale estimation phase are distributions of mass in Attribute space, not Attribute x Position space. For example, if the Color x Position distribution of an image notes 10% red at the top of the image and 20% red at the bottom, then the Color distribution notes only that there is 30% red in the image. The input Attribute distribution to the scale estimation phase is obtained by marginalizing out the position information from the Attribute x Position distribution, as in the color example just given.

top	Title, Table of Contents, Introduction
prev	Overview
next	The Scale Estimation Phase

The ideas and results contained in this document are part of my thesis, which will be published as a Stanford computer science technical report in June 1999.

S. Cohen. Finding Color and Shape Patterns in Images. Thesis Technical Report STAN-CS-TR-99-?. To be published June 1999.

Email comments to scohen@cs.stanford.edu.