Register-based Statistics
Registers and the National Statistical System
Wiley Series in Survey Methodology
3. Edition March 2022
288 Pages, Hardcover
Wiley & Sons Ltd
ISBN:
978-1-119-63237-5
John Wiley & Sons
Further versions
REGISTER-BASED STATISTICS
Rediscover this authoritative guide to register-based statistics filled with significant new improvements
In the newly revised Third Edition of Register-based Statistics: Registers and the National Statistical System, Anders Wallgren and Britt Wallgren deliver a robust exploration of how register-based statistics can be used to its fullest potential. The authors describe how statistical institutes can work on long-term projects to improve administrative systems, as well as estimation methods that can improve the quality of statistical estimates based on registers with quality problems. Readers will also discover how to improve the ways register-statistical issues are introduced, as well as how to create population registers.
Finally, the authors draw on their experience from teaching and consulting in several countries to explain how to implement register-based statistics.
Key features of the third edition:
* Discusses the problems new register countries face
* Explains how registers will improve the efficiency of the national statistical system
* Clarifies the importance of the system approach
* Describes how a statistical population register can be created
* Registers-based statistics require new skills and understanding of new concepts
* Many important quality indicators are described
* Explains difficult topics in a pedagogic way
Perfect for staff at national statistical institutes and administrative and ministerial authorities belonging to national statistical systems, Register-based Statistics will also prove to be an indispensable resource for undergraduate and graduate students in statistics programs and courses, as well as survey researchers and practitioners.
Index to Charts xi
Preface xix
Chapter 1 Censuses, Sample Surveys and Register Surveys 1
1.1 The national statistical system 2
1.2 The traditional census-based system 3
1.3 New sources: Administrative registers and Big data 5
1.4 Basic concepts and terms 7
1.4.1 What is a register? 7
1.4.2 Databases, records and observations 8
1.4.3 What is a register survey? 10
1.4.4 A register survey: The Income and Taxation Register 12
1.5 New demands and opportunities require new methods 14
1.5.1 A new paradigm is necessary 14
1.5.2 New statistical methods 15
1.5.3 The basic principles of register-based statistics 18
1.6 Preconditions for register-based statistics 19
1.6.1 Reliable administrative systems 20
1.6.2 Legal base and public approval 21
1.6.3 Political support to strengthen the statistical system 23
Chapter 2 The Transition to a Register-based Production System 25
2.1 First obstacle: How to gain access to microdata? 26
2.2 Protection of privacy and confidentiality 26
2.3 Second obstacle: How to take care of dirty data? 29
2.4 The new production process 30
2.4.1 Contacts with administrative authorities 31
2.4.2 Metadata have a new role 31
2.4.3 Anonymisation of identity numbers 32
2.4.4 Editing of a single administrative register 33
2.4.5 Organising the work with administrative registers 36
2.5 Third obstacle: The national registration system 37
2.5.1 Legislation governs access to data 38
2.5.2 Too many registers, but no good registers - what to do? 38
2.5.3 Legislation rules obligations to report and what to report 40
2.6 Why has the census been so important? 41
2.7 Creating the register system 42
2.7.1 Where do you live? 43
2.7.2 Where do you work? 45
2.7.3 With whom do you live? 46
2.7.4 A centralised or decentralised national system? 47
2.8 Register surveys and estimation methods 48
2.9 A traditional census or a register-based census? 49
Chapter 3 The Nature of Administrative Data 51
3.1 Comparing questionnaire and register data 51
3.1.1 A questionnaire to persons compared with register data 51
3.1.2 An enterprise questionnaire compared with register data 54
3.2 Enterprise registers for combined use 56
3.2.1 Corrections in accounting data 57
3.2.2 Missing values in accounting data 58
3.2.3 Administrative and statistical information systems 59
3.3 Measurement errors in questionnaire and register data 60
3.3.1 Measurement errors 61
3.3.2 Taxation errors 62
Chapter 4 Building the System - Record Linkage 65
4.1 Record linkage 65
4.2 Record linkage in the Nordic countries 66
4.3 Deterministic record linkage 68
4.4 Creating variables by adjoining and aggregation 70
4.5 Probabilistic record linkage 73
4.6 Four causes of matching errors 79
4.7 The statistical system and record linkage 82
Chapter 5 Building the System - Quality Assessment 85
5.1 Four quality concepts 85
5.2 Making an inventory of potential sources 87
5.3 How can a source be used? 87
5.4 Quality assessment in a register-based production system 90
5.4.1 Analysing metadata 91
5.4.2 Analysis and data editing of the source 92
5.4.3 Comparing a source with the base register 92
5.4.4 Comparing a source with surveys with similar variables 93
5.5 Output data quality and quality of estimates 94
5.5.1 Analysing quality with a test census 94
5.5.2 Analysing quality with samples from the new register 95
5.5.3 Analysing quality with area samples 96
5.5.4 Measuring quality of basic register variables with the LFS 98
5.6 A coordinated system of registers 98
5.6.1 Are the base registers a coordinated system? 98
5.6.2 Quality indicators at the system level 99
5.7 Using the quality indicators 101
Chapter 6 Building the System - Editing Register Data 107
6.1 Editing in register surveys 108
6.2 Editing of a single administrative register 109
6.3 Consistency editing 110
6.3.1 Consistency editing - is the population correct? 111
6.3.2 Consistency editing - are the units correct? 118
6.3.3 Consistency editing - are the variables correct? 120
6.4 Case studies - editing register data 121
6.4.1 Editing work within the Income and Taxation Register 121
6.4.2 Editing work within the Income Statement Register 123
6.4.3 What more can be learned from these examples? 124
6.5 Editing, quality assessment and survey design 125
6.5.1 Survey design in a register-based production system 125
6.5.2 Survey design - management problems 127
6.5.3 Total survey error in a register-based system 128
Chapter 7 Building the System - The Population Register 129
7.1 Inventory of sources 131
7.1.1 Time references 131
7.1.2 Activities or 'signs of life' 131
7.2 The Population Register based on full information 133
7.2.1 Object types - Changing and unchanging registers 133
7.2.2 Variables with different functions in the system 134
7.2.3 Updating the Population Register 136
7.2.4 Registers and time 137
7.2.5 Variables and time 140
7.3 The Population Register in new register countries 140
7.3.1 Different systems of identity numbers 141
7.3.2 Problems in countries without a central Population Register 142
7.3.3 How to improve coverage of the Population Register 143
7.3.4 Inventory of sources - addresses and time references 146
7.4 Methods to measure and improve quality 148
7.4.1 Three kinds of surveys should be combined 148
7.4.2 A new register-based system for statistics on persons 150
7.5 Conclusions 151
7.6 Challenges in old register countries 152
Chapter 8 The Population Register - Estimation Methods 155
8.1 Estimation in sample surveys and register surveys 156
8.1.1 Estimation methods for register surveys that use weights 157
8.1.2 Calibration of weights in register surveys 157
8.2 Calibration of weights - the Swedish LFS 161
8.2.1 Use of auxiliary information in the LFS 161
8.2.2 Nonresponse bias in the LFS 162
8.3 Calibration - where do people live? 163
8.4 Methods to handle overcoverage 167
Chapter 9 Defining Register Populations - Coverage Errors 171
9.1 Defining a register's object set 172
9.1.1 Defining a population 172
9.1.2 Can you alter data from the National Tax Agency? 176
9.1.3 Defining a population - the Farm Register 176
9.1.4 Defining a population - integrated registers 178
9.2 Defining a calendar year population 179
9.2.1 Defining a population - frame or register population? 180
9.2.2 Sampling paradigm versus register paradigm 184
Chapter 10 Building the System - The Business Register 185
10.1 The Business Register and the National Accounts 185
10.2 The base register for economic statistics 187
10.3 The scope of the register and choice of object types 188
10.3.1 The register with legal units and local units 189
10.3.2 The register with enterprise units and kind of activity units 191
10.4 Inventory of sources 195
10.5 Creating and maintaining the Business Register 198
Chapter 11 The Business Register - Estimation Methods 201
11.1 Multi-valued variables 202
11.2 Estimation methods 205
11.2.1 Occupation in the Activity and Occupation Registers 206
11.2.2 Industrial classification in the Business Register 210
11.2.3 Estimates from different register versions 213
11.3 Application of the method 214
11.3.1 Change of industry and time series quality 215
11.3.2 Transformation of weights 217
11.4 A decentralised or centralised statistical system? 218
11.4.1 The Calendar Year Register and the National Accounts 219
11.4.2 Choosing the best source for the National Accounts 220
11.5 Conclusions 224
Chapter 12 Censuses, Sample Surveys and Register Surveys -Conclusions 227
12.1 Attitudes towards the register-based census 227
12.2 The new national statistical system 231
12.2.1 The system of base registers 232
12.2.2 Activity registers and longitudinal registers 234
12.3 Survey design 237
12.3.1 Sample survey design 237
12.3.2 Register survey design 238
12.3.3 Creating register variables 241
12.4 Survey quality 245
12.4.1 Quality of registers and register surveys 246
12.4.2 The integration process - integration errors 247
12.4.3 Frame errors 247
12.5 Organising the new production system 248
12.5.1 Enterprise architecture and the register system 248
12.5.2 The register system and data warehousing 249
12.5.3 Missing values - a system-based approach 252
12.6 Final remarks 254
12.6.1 The Statistical Population Register 254
12.6.2 The system of base registers 255
References 257
Index 261
Preface xix
Chapter 1 Censuses, Sample Surveys and Register Surveys 1
1.1 The national statistical system 2
1.2 The traditional census-based system 3
1.3 New sources: Administrative registers and Big data 5
1.4 Basic concepts and terms 7
1.4.1 What is a register? 7
1.4.2 Databases, records and observations 8
1.4.3 What is a register survey? 10
1.4.4 A register survey: The Income and Taxation Register 12
1.5 New demands and opportunities require new methods 14
1.5.1 A new paradigm is necessary 14
1.5.2 New statistical methods 15
1.5.3 The basic principles of register-based statistics 18
1.6 Preconditions for register-based statistics 19
1.6.1 Reliable administrative systems 20
1.6.2 Legal base and public approval 21
1.6.3 Political support to strengthen the statistical system 23
Chapter 2 The Transition to a Register-based Production System 25
2.1 First obstacle: How to gain access to microdata? 26
2.2 Protection of privacy and confidentiality 26
2.3 Second obstacle: How to take care of dirty data? 29
2.4 The new production process 30
2.4.1 Contacts with administrative authorities 31
2.4.2 Metadata have a new role 31
2.4.3 Anonymisation of identity numbers 32
2.4.4 Editing of a single administrative register 33
2.4.5 Organising the work with administrative registers 36
2.5 Third obstacle: The national registration system 37
2.5.1 Legislation governs access to data 38
2.5.2 Too many registers, but no good registers - what to do? 38
2.5.3 Legislation rules obligations to report and what to report 40
2.6 Why has the census been so important? 41
2.7 Creating the register system 42
2.7.1 Where do you live? 43
2.7.2 Where do you work? 45
2.7.3 With whom do you live? 46
2.7.4 A centralised or decentralised national system? 47
2.8 Register surveys and estimation methods 48
2.9 A traditional census or a register-based census? 49
Chapter 3 The Nature of Administrative Data 51
3.1 Comparing questionnaire and register data 51
3.1.1 A questionnaire to persons compared with register data 51
3.1.2 An enterprise questionnaire compared with register data 54
3.2 Enterprise registers for combined use 56
3.2.1 Corrections in accounting data 57
3.2.2 Missing values in accounting data 58
3.2.3 Administrative and statistical information systems 59
3.3 Measurement errors in questionnaire and register data 60
3.3.1 Measurement errors 61
3.3.2 Taxation errors 62
Chapter 4 Building the System - Record Linkage 65
4.1 Record linkage 65
4.2 Record linkage in the Nordic countries 66
4.3 Deterministic record linkage 68
4.4 Creating variables by adjoining and aggregation 70
4.5 Probabilistic record linkage 73
4.6 Four causes of matching errors 79
4.7 The statistical system and record linkage 82
Chapter 5 Building the System - Quality Assessment 85
5.1 Four quality concepts 85
5.2 Making an inventory of potential sources 87
5.3 How can a source be used? 87
5.4 Quality assessment in a register-based production system 90
5.4.1 Analysing metadata 91
5.4.2 Analysis and data editing of the source 92
5.4.3 Comparing a source with the base register 92
5.4.4 Comparing a source with surveys with similar variables 93
5.5 Output data quality and quality of estimates 94
5.5.1 Analysing quality with a test census 94
5.5.2 Analysing quality with samples from the new register 95
5.5.3 Analysing quality with area samples 96
5.5.4 Measuring quality of basic register variables with the LFS 98
5.6 A coordinated system of registers 98
5.6.1 Are the base registers a coordinated system? 98
5.6.2 Quality indicators at the system level 99
5.7 Using the quality indicators 101
Chapter 6 Building the System - Editing Register Data 107
6.1 Editing in register surveys 108
6.2 Editing of a single administrative register 109
6.3 Consistency editing 110
6.3.1 Consistency editing - is the population correct? 111
6.3.2 Consistency editing - are the units correct? 118
6.3.3 Consistency editing - are the variables correct? 120
6.4 Case studies - editing register data 121
6.4.1 Editing work within the Income and Taxation Register 121
6.4.2 Editing work within the Income Statement Register 123
6.4.3 What more can be learned from these examples? 124
6.5 Editing, quality assessment and survey design 125
6.5.1 Survey design in a register-based production system 125
6.5.2 Survey design - management problems 127
6.5.3 Total survey error in a register-based system 128
Chapter 7 Building the System - The Population Register 129
7.1 Inventory of sources 131
7.1.1 Time references 131
7.1.2 Activities or 'signs of life' 131
7.2 The Population Register based on full information 133
7.2.1 Object types - Changing and unchanging registers 133
7.2.2 Variables with different functions in the system 134
7.2.3 Updating the Population Register 136
7.2.4 Registers and time 137
7.2.5 Variables and time 140
7.3 The Population Register in new register countries 140
7.3.1 Different systems of identity numbers 141
7.3.2 Problems in countries without a central Population Register 142
7.3.3 How to improve coverage of the Population Register 143
7.3.4 Inventory of sources - addresses and time references 146
7.4 Methods to measure and improve quality 148
7.4.1 Three kinds of surveys should be combined 148
7.4.2 A new register-based system for statistics on persons 150
7.5 Conclusions 151
7.6 Challenges in old register countries 152
Chapter 8 The Population Register - Estimation Methods 155
8.1 Estimation in sample surveys and register surveys 156
8.1.1 Estimation methods for register surveys that use weights 157
8.1.2 Calibration of weights in register surveys 157
8.2 Calibration of weights - the Swedish LFS 161
8.2.1 Use of auxiliary information in the LFS 161
8.2.2 Nonresponse bias in the LFS 162
8.3 Calibration - where do people live? 163
8.4 Methods to handle overcoverage 167
Chapter 9 Defining Register Populations - Coverage Errors 171
9.1 Defining a register's object set 172
9.1.1 Defining a population 172
9.1.2 Can you alter data from the National Tax Agency? 176
9.1.3 Defining a population - the Farm Register 176
9.1.4 Defining a population - integrated registers 178
9.2 Defining a calendar year population 179
9.2.1 Defining a population - frame or register population? 180
9.2.2 Sampling paradigm versus register paradigm 184
Chapter 10 Building the System - The Business Register 185
10.1 The Business Register and the National Accounts 185
10.2 The base register for economic statistics 187
10.3 The scope of the register and choice of object types 188
10.3.1 The register with legal units and local units 189
10.3.2 The register with enterprise units and kind of activity units 191
10.4 Inventory of sources 195
10.5 Creating and maintaining the Business Register 198
Chapter 11 The Business Register - Estimation Methods 201
11.1 Multi-valued variables 202
11.2 Estimation methods 205
11.2.1 Occupation in the Activity and Occupation Registers 206
11.2.2 Industrial classification in the Business Register 210
11.2.3 Estimates from different register versions 213
11.3 Application of the method 214
11.3.1 Change of industry and time series quality 215
11.3.2 Transformation of weights 217
11.4 A decentralised or centralised statistical system? 218
11.4.1 The Calendar Year Register and the National Accounts 219
11.4.2 Choosing the best source for the National Accounts 220
11.5 Conclusions 224
Chapter 12 Censuses, Sample Surveys and Register Surveys -Conclusions 227
12.1 Attitudes towards the register-based census 227
12.2 The new national statistical system 231
12.2.1 The system of base registers 232
12.2.2 Activity registers and longitudinal registers 234
12.3 Survey design 237
12.3.1 Sample survey design 237
12.3.2 Register survey design 238
12.3.3 Creating register variables 241
12.4 Survey quality 245
12.4.1 Quality of registers and register surveys 246
12.4.2 The integration process - integration errors 247
12.4.3 Frame errors 247
12.5 Organising the new production system 248
12.5.1 Enterprise architecture and the register system 248
12.5.2 The register system and data warehousing 249
12.5.3 Missing values - a system-based approach 252
12.6 Final remarks 254
12.6.1 The Statistical Population Register 254
12.6.2 The system of base registers 255
References 257
Index 261
Anders Wallgren and Britt Wallgren worked for many years in teaching and research as senior lecturers in statistics at Stockholm, Linköping and Örebro universities. They have also worked as consultants for several Swedish manufacturing industries. In connection with these consulting assignments, they began working with corporate administrative registers and developed statistical methods for business intelligence.
After their academic career, they came to Statistics Sweden and worked for about 20 years at the Department of Research and Development. They worked mainly with survey methodology for the statistical registers which form the basis for the production of official statistics in Sweden.
Since 2011 they have provided courses and consulting services on behalf of the Inter-American Development Bank in several Latin American countries and in the Caribbean.
After their academic career, they came to Statistics Sweden and worked for about 20 years at the Department of Research and Development. They worked mainly with survey methodology for the statistical registers which form the basis for the production of official statistics in Sweden.
Since 2011 they have provided courses and consulting services on behalf of the Inter-American Development Bank in several Latin American countries and in the Caribbean.
