Strong profiling is not mathematically optimal for discovering rare malfeasors (on rare event detection)

Just in time for the latest Christmas terror scare, I came across an interesting paper: “Strong profiling is not mathematically optimal for discovering rare malfeasors” (William H. Press; PNAS 106(6), p. 1716-1719 www.pnas.org/cgi/doi/10.1073/pnas.0813202106). In the paper, the author investigates whether profiling by nationality or ethnicity can be justified mathematically and tries to answer the question of how much screening must we do, on average, to catch the bad guys in the crowd. Rare events detection is hard as it is, and it’s interesting to see a look from the sampling perspective. It’s an interesting and short read. Long story short, it shows that using an indiscriminate feature like nationality or ethnicity is not optimal (as is any screening at least in proportion to a prior probability) and wastes resources.

This entry was posted on January 10, 2010 5:37 pm at 5:37 pm (January 10, 2010) and is filed under Math, Society, Statistics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.

| Print