Statistical confidentiality:
how to avoid the “big brother” effect of statistics while meeting governments’
and researchers’ needs for detailed data?
How to find a balance between providing an increased access
to very detailed data (microdata) without producing a negative effect on the
protection of privacy and confidentiality of individuals – this was the main
topic for discussion of Heads of national and international statistical offices
at the Seminar Session on “Statistical Confidentiality and Microdata” which
took place at the Conference of European Statisticians1 this June.
Various aspects of statistical data confidentiality – legal, organizational,
methodological and technical – were considered.
The confidentiality principle is one of the cornerstones
of official statistics. Statistical offices collect large amounts of data
through surveys and censuses and make use of various administrative records.
When collecting data through surveys, the offices make a promise to respondents
to protect their privacy: whatever the respondents reply in the questionnaires
is to remain confidential. Traditionally, only aggregate statistics were disseminated
where information about individuals, be it persons or enterprises, cannot
be disclosed to any third party – and in particular not to tax authorities
or other government agencies.
However, there is a wealth of detailed data available in
statistical offices that could be invaluable for tailor-made statistics necessary
for research, policy-making, local planning, etc. If these data can be used,
while still protecting privacy, the economy, society and citizens would gain.
Alongside research, a strong driving force for more data is the increasing
demand for accountable government and transparency. Developing evidence-based
policies and monitoring their impact requires analysis on a much more detailed
level than presently published by statistical agencies.
The challenge is to strike the right balance between increased
data dissemination and confidentiality protection. This dilemma lies at the
heart of official statistics. On the one hand, official statistics should
serve the government, economy and public with data, and it is in the interest
of statistical offices to disseminate as much as possible. On the other hand,
they have to guarantee the confidentiality of individual data collected to
maintain the trust of respondents. An optimal solution can be found only by
combining the different measures and actions of confidentiality protection:
legal, organizational, methodological and technical. Individual data (microdata)
can be released in such a way that it is virtually impossible to recognize
a concrete individual from the data.
In general, statistical offices take quite a conservative
attitude to dissemination of microdata but along with the improvement of technological
methods to guarantee confidentiality, they are increasingly opening up to
a greater dissemination of detailed data.
The Heads of statistical offices also discussed the specific
concerns of central and eastern European and the CIS countries about data
confidentiality. EU accession countries have mostly brought their legal framework
on confidentiality protection in line with the EU requirements. Some of the
south-east European and CIS countries are in the process of establishing the
required legal basis. In some CIS countries, problems still arise because
of the pressure from other government bodies to gain access to microdata.
Emerging from a past where official statistics followed a completely different
paradigm, its role is often not fully understood by other government agencies.
Unresolved technical aspects, lack of special software and the low level of
technological equipment in local statistical offices sometimes make it difficult
to ensure data protection at the required level. However, the statistical
offices of these countries are committed to resolve the problems of confidentiality
protection.
The methods demonstrated show that it is possible to allow access to microdata
and protect the privacy of individuals at the same time. Developing international
standards for a wide variety of possible methods of access will be beneficial
both for the research community and the countries that are setting up their
legal and technological basis. Official statistics are dealing seriously with
any confidentiality concerns, and we can expect an increasing use of microdata-based
research and policy making in the near future.
All papers from the Seminar session are available at the
web site:
http://www.unece.org/stats/documents/2003.06.ces.htm
For more information, please contact:
Heinrich Brüngger
Director of the Statistical Division
UN Economic Commission for Europe
Palais des Nations
CH – 1211 Geneva 10
Tel: +41 (0) 22 917 12 71
Fax: +41 (0) 22 917 00 40
E-mail: [email protected]
__________
1 The Conference of European Statisticians (CES) is composed of
the Heads of national statistical offices of UNECE member countries and international
statistical organizations who meet annually to discuss the most important
problems and developments in statistics.
Ref: ECE/STAT/03/N01