Word bursts reveal hot topics

Cornell University researcher Jon Kleinberg has shown that you can learn a lot about what topics are becoming important at any given time by analyzing large collections of documents for sudden changes in content.

Kleinberg tested the theory by searching his email using an algorithm that examined the frequency of words over time. These provided clues about the hot topics of the moment. For example, the searches made it apparent when exam time was near -- the word "prelim" suddenly became more frequent.

Searching American presidents' state of the union addresses produced a list of words that neatly summarized the important topics in american politics at any given time: gentleman was popular in the late 1700s, militia burst onto the scene in 1801, bank showed up in 1833, California in 1848, slaves in 1859, rebellion in 1861, emancipation in 1862, paper in 1867, coinage in 1877, forest in 1901, interstate in 1907, marketing in 1919, atomic in 1947, inflation in 1971, oil in 1974, crime in 1991, businesses in 1990, health in 1992, and teachers in 1996.

Kleinberg presented his work at the American Association for the Advancement of Science (AAAS) annual meeting in Denver on February 18. -TECHNOLOGY RESEARCH NEWS