As someone who’s interested in the use of Census data for analysis, I couldn’t resist attending the first inaugural ACS Data Users Conference, held in DC May 29-30. The conference brought together over 300 people who use the ACS in various applications, from research, to policy, and to business. On hand were a number of people from the Bureau of the Census, giving presentations and being available for questions. It was a great experience for someone who hasn’t had much formal experience working with the ACS but who preaches the utility of the data it provides.
The most valuable thing I took away from the first day was the extended discussion on how to use the margin of error reported with the ACS. Since the ACS is a sample of the population (with detailed responses from approximately 3 million households a year) aggregated over various time frames (1-, 3- and 5-years), there is a measured margin of error for the weighted estimate of the actual value measured in the larger population. The Census reports the estimated value in one file and reports the margin of error (as a number + or -) for that estimate in a separate file.
One thing I learned during the conference was that I wasn’t alone in basically ignoring the margin of error when I use ACS data. Particularly when dealing with people unfamiliar with Census data and the ACS in particular, the margin of error is confusing and distracts from the larger message being discussed with the data. I definitely don’t condone ignoring the margin of error for rigorous statistical analysis where accuracy is important, but particularly in applications where the data is being mapped, there aren’t really any good ways of displaying the error in any way that makes it comprehensible. Outside of mapping applications, the coefficient of variation (CV), which expresses the standard error as a percentage of the estimate, gives a sense of how reliable the estimate is for decision making purposes. The consensus seemed to be that a CV of less than 15-20% was generally good, with anything above that questionable for analysis.
There were many great presentations on this, but one of the best I thought was a presentation from researchers working at the Cornell University Program on Applied Demographics. Another interesting presentation on techniques to create more accurate estimates came from a researcher at Nielson.
There were many great presentations from the ACS Data Users Conference and I highly recommend perusing the conference schedule and downloading the presentations (registration required). There is a wealth of information on lessons learned and best practices for making use of this valuable data.