This paper considers the degree of progress made by the field of dynamic microsimulation over the past five decades. It highlights the expanding breadth of the field, both in terms of the number of countries and in terms of the broadening policy area. It also outlines concerns in relation to lack of emphasis historically in relation to the transmission of codified and in the sometimes proprietary ownership model. Moving forward, an improved focus on the codification and peer review of methodologies used in the field is suggested. In terms of tacit knowledge transmission, the organisation of more regional meetings of the International Microsimulation Association is encouraged. In terms of future areas for model development, the opportunities are to incorporate behaviour to a greater extent, utilise more data in the big data revolution, to increase the field of microsimulation into new policy areas, and to share code and work in networks. Finally, the role of peer review is highlighted.
While microsimulation modelling helps us to understand economic and social processes and related policy at the micro-level, dynamic microsimulation modelling helps us to disentangle processes and policies with a temporal dimension.1 The sub-field goes back to the earliest research in microsimulation (Orcutt, 1960; Orcutt, Greenberger, Kobel, & Rivlin, 1961). In this paper we consider progress made by the area and consider ways to make greater progress.
In our view, the core purpose of microsimulation models is to understand and manage complexity. A model is a mechanism of abstracting from reality that helps us understand complexity better. In the context of policy design and evaluation, complexity of microsimulations can take the form of
population structure of the population,
behavioural response to the policy, and
These levels of complexity themselves interact with each other, as shown in Figure 1, resulting in a degree of complexity that is difficult to disentangle without recourse to a model.
Adding the time dimension makes the other dimensions all the more complex. For one, it opens a discussion on whether the data in the starting dataset need to be imputed. If not, then the model needs to take these missing values into account. Furthermore, even more compared to cell-based models, there is not much reason to assume that the results of reduced-form behavioural equations will yield intertemporal stable results, and if they do not, then a lot of tweaking is required. Furthermore, the dynamic element makes micro-macro iterations considerably harder, at least in the types of models that are “full dynamic”, that is other than models that apply static ageing techniques. Finally, for modelling purposes, one needs not only to become credible cross-sectional results but also longitudinal results. For this, both the incidence and prevalence levels, as well as durations need to be considered, while at the same time preventing some impossible transitions.
Furthermore, one needs to be cautious about the ever-increasing complexity of models. Enhanced complexity means that models are more costly, time consuming, harder to interpret, and thus result in a loss of the aims of modelling which is to simplify reality so as to gain insight. The challenge therefore is to have just enough complexity to produce a model that is useful.
There have been a number of survey articles written (including Dekkers & Van den Bosch, 2016; Li & O’Donoghue, 2013; Li, O’Donoghue, & Dekkers, 2014; O’Donoghue, 2001) which have described the progress of the field and reviewed the modelling choices made by different dynamic microsimulation models. While the breadth of the models in terms of countries has expanded, it may at first glance be considered that the field has not developed as much as, say, related Agent Based Modelling fields, which started later, but appears to have progressed more (Delli Gatti, Fagiolo, Gallegati, Richiardi, & Russo, 2018; Hamill & Gilbert, 2015).
Even though there is enough to be said in defence of our field (it is comparably small, less academic/more applied by nature, and microsimulation models are more costly in terms of development and maintenance than the average agent-based model (ABM), it is still highly relevant to assess why this might be and how the pace of these developments might be increased.
In this paper we review recent developments since 2013, looking at 100 or so articles (see Appendix) with the highest citations for dynamic microsimulation in Google Scholar. We consider also some of the challenges to the development of dynamic microsimulation and look at strategies to improve the impact of the field into the future.
Like any research field, progress is based upon building the achievements of others; that is Newton’s “if I have seen further it is by standing on the shoulders of giants”. In our own research in the field, both authors have learnt significantly from interacting with many people. However much of this learning in relation to microsimulation has been by word of mouth, via these interactions or via documentation in conference proceedings or books —many of which are now out of print (such as the excellent Orcutt, Merz, & Quinke, 1986). Without personal interactions with the leading figures in the field, it would have been challenging for us to develop the models we have. Based on our experience, we therefore would conclude that the field of dynamic microsimulation has relied on an oral tradition. It is a challenge for the field therefore to find more effective ways of transferring knowledge and improve impact.
In this article we will argue that progress has been limited due to the over reliance historically on tacit knowledge transmission and on the intellectual property model. However, we argue that the field is very useful and has improved in these areas and is growing and developing rapidly at present.
The structure of this paper is as follows: Section 2 will consider the progress made by the field over the past five decades. Section 3 discusses ways in which the pace of change and the impact of the field is improving and might improve further. Section 4 concludes.
In considering the progress made by the field, we will focus on the area of dynamic microsimulation modelling, while interpreting “dynamic” as the direct modelling of a time element. The Handbook of Microsimulation (O’Donoghue, 2014) contains a more substantial review of other microsimulation model types.
Consider first a dynamic microsimulation model with the following characteristics:
A demographic module, modelling leaving home, births, deaths partnership formation and dissolution, disability, education and broad location.
A labour market module containing participation, hours, unemployment and labour income.
A Tax-Transfer and Wealth module containing capital income and the main tax and transfer instruments.
A marriage matching module.
A simple macro-economic model and feedback loops linked with the microsimulation model via alignment.
Monte Carlo simulation of alternative random numbers.
By any standard, this is quite a broad range of attributes in a model. However, this is a description of the DYNAMSIM I dynamic microsimulation model (Orcutt, Wertheimer, & Caldwell, 1976) built at the Urban Institute in in the 1960s–1970s. On the face of it, therefore, there would seem to be relatively little progress made in field over the 40 years since this model was built.
However, a more comprehensive discussion of achievements should be based on past ambitions and whether these were met. In his opening address to the Ottawa conference of the International Microsimulation Association in 2007, Wolfson (2009) described the situation of the field 50 years after Guy Orcutt’s seminal paper (1957). Furthermore, Wolfson describes three ways in which Orcutt’s original vision should be adapted, and presents six “reflections on the coming decade for microsimulation”. As we are a decade later already, it is about time to look back at the Wolfson paper and see whether the ambitions have been met or whether current developments are in line with them.
One of the fields in which Wolfson argues that the original vision of Orcutt should be adapted is in the trade-off between “a single omnibus all-encompassing model” and more targeted purpose- built models (Wolfson, 2009). Building on the work by, among others, the DYNACAN team, one might argue that quite some work has been done in that area. As Dekkers et al. (2010a), Dekkers, Desmet, Rézmovits, Sundberg, & Tóth (2015) and Dekkers, Conti, Desmet, Sundberg, & Van den Bosch (2018) show, various dynamic microsimulation models are now capable to work in parallel with macro models. Furthermore, on a more technical level, many tools and techniques that previously were model-bound are now available through open-source toolkits such as Modgen, JAMSIM, JAS-mine and LIAM2. See De Menten, Dekkers, Bryon, Liégeois, & O’Donoghue (2014) for a discussion.
Further, Wolfson (2009, p. 27) makes the case that “a reviewed vision for socio-economic microsimulation should also include building bridges between […] applied models and […] agent-based modelers”. One of the problems, he noted, was that “the empirical grounding of many of the agent-based models is still a ways off’. This is no longer the case today. ABMs are becoming more empirically-driven, and increasing numbers of ABM papers are being presented at International Microsimulation Association (IMA) congresses. The 2018 Asia-Pacific regional conference of IMA is a case in point; it was organised jointly with the 10th International Workshop on Agent-based Approach in Economic and Social Complex Systems. This collaborative effort between IMA and the Pan-Asian Association for Agent-Based Approach in Social Systems Sciences (PAAA) will hopefully add to the increasing overlap between the two fields. An application which appears especially promising is the use of ABM-approaches to underpin the behavioural elements of dynamic microsimulation models (Richiardi, 2013; Van de Ven, 2017).
Next, we turn to the reflections that Wolfson developed in 2007 “on the coming decade for microsimulation” (Wolfson, 2009, p. 28). The first of these key points is the development of health microsimulation. This was a young but growing field at that time, and it is an important and still growing field for microsimulation today. For example, health microsimulation is now a standard session subject at the microsimulation (MSM) conferences, such as in Luxembourg (2015), Budapest (2016), and Turin (2017). However, when it comes to dynamic microsimulation, the number of specific models remains limited and it is “more common to see health modules ‘bolted to’ dynamic models primarily developed to project demographic change” (Schofield, Carter, & Edwards, 2014, p. 422). One might wonder whether this is such a major problem in that it would mean that a general multi-purpose dynamic model could be used for research in various fields, such as retirement, health and education. This then of course requires that the various teams have access to this base model. This is a problem this paper will get back to later.
The next area that Wolfson calls “promising” is that of social indicators. In particular does he refer here to the system of System of Social and Demographic Statistics (SDSS). Although this discussion is now largely past us, the discussion on what indicators dynamic microsimulation models can provide is still as relevant as it was before. Dynamic life-expectancy tables, gender pension gaps, redistributive consequences of selective mating, impact of pension system characteristics and labour market decisions, developments in educational attainment levels, health indicators; the list goes on. A recent development that might be rooted in the discussion on the SSDS in the late 1970s is that of the accrued-to-date-liabilities (ADL) in Europe. These are “the pension entitlements accrued by current workers (including deferred pension entitlements) and the remaining pension entitlements of existing pensioners” (Eurostat/European Central Bank, 2011, p. 16, Box 1). This is a concept of pension obligations that “provide an estimate of the cost of a hypothetical termination of a pension scheme without reneging on accrued entitlements” (op. cit Box 2, p. 18). Thus, pension rights accrued in future years are not taken into account. ADLs are being simulated by using dynamic microsimulation models in several countries, including France (Blanchet, Le Minez, & Marino, 2015), Estonia (Praxis, 2016) and Belgium (Brys, 2017).
Wolfson then proceeds to discuss the use of microsimulation models in a multi-country context. Obviously he here refers to EUROMOD as the example of this promising area (Sutherland, 2014, p. 79 and p. 99). In dynamic modelling, this trend has been nearly absent, at least until recently.
Now that the contours of a network envisaged by Dekkers & Zaidi (2011) becomes visible, albeit outside academics (Dekkers & Van den Bosch, 2016), the first internationally comparative studies using dynamic microsimulation appear (Dekkers et al., 2010a, 2015, 2018). At an obviously slower rate than EUROMOD, one might hope that this trend will continue and include non-European countries in the future as well.
Next, a particularly interesting strategic direction that Wolfson saw in 2007 was for microsimulation to “become firmly implanted in national statistical agencies” (Wolfson, 2009, p. 29). In the field of dynamic microsimulation, this is certainly the case in the US (where CBOLT – Congressional Budget Office Long-Term Model (Congressional Budget Office, 2018) is a good example), Norway and many countries of the European Union. Dekkers & Van den Bosch (2016) present an overview of 17 dynamic models used by ministries and public organisations in EU member states. Nearly half of them are not mentioned in Li and O’Donoghue (2013), and they differ from the models grouped in the latter on at least one important dimension: data. This will be discussed later.
Finally, the last important issue that Wolfson mentions, is scientific status. Back in 2007, he wrote “microsimulation still has not achieved the kind of scientific status it deserves” (Wolfson, 2009, p. 11). This is the main subject of the rest of this paper.
Our question is why has this happened? What are the possible reasons for this perceived lack of development and status? One possible reason lies in knowledge transfer in our field. There are two main types of knowledge transfer:
Codified Knowledge, where specific knowledge is written down.
Tacit Knowledge, where more abstract information may be more difficult to transmit.
As outlined above, much of our knowledge transfer in the field has been transmitted verbally between people on teams, at conferences and through networks.
For much of the period since the foundation of the field, where we have codified knowledge it has been mainly through the following forms:
Documentation – aiming to facilitate other team members utilising the models.
Published – mainly via books and conference presentations which may have been non-peer reviewed, had limited coverage, often went out of print.
Documents that may have only been available to those who attended an event and were rarely included in usual citation indices and searchable databases.
Where papers were published in peer reviewed formats, they were typically in journals where the focus was on the application rather than the methodology.
Reviewing recent papers in the field, there has been a large increase in the share of dynamic microsimulation papers published in peer reviewed journals at 67%, 9% in books, with only 16% in Mimeo’s and 7% in conference proceedings (Table 1). In O’Donoghue (2001), the proportion would have been the opposite. This is a very welcome change and has a facilitated an expansion in the field.
In addition to an increased focus in peer reviewed articles, the nature of application for dynamic microsimulation models has changed very significantly since the previous (op. cit.) review (Table 2). Reflecting the broadening of microsimulation as a field, we see that a much greater variety with health and demographic modelling papers having the highest shares of recent articles. In 2000, “Pensions” was the main focus, now in third place.2 “Spatial” and “Environmental” papers are becoming increasingly important, taking advantage of improved computing power and available data.
A significant proportion of the methods used in the field are not formally codified, meaning that new models have had to reinvent the wheel and re-develop existing methods over and over again. Where methods were formally codified, often they were codified in non-peer reviewed technical notes or discussion papers and thus lack the quality assurance that peer review can help to achieve. Another issue is that publication in a research centre technical paper or note has the high risk associated with the ending of funding, retirement of staff or end of life of a model. This thus places a sustainability risk for the field in respect to the core methodological foundations of the field. The story of DYNACAN and its working papers bears testimony to this.
A classic example is the methodology in relation to alignment used in dynamic microsimulation models. It is a calibration mechanism used to align simulated totals to external control totals and has been used since the 1970s. It is thus a core methodology within the field. However, there is relatively little documentation or guidance as to how to undertake alignment. Where it exists it is published in non-peer reviewed technical papers (Bækgaard, 2002) as team specific internal documentation (Johnson, 2001; Morrison, 2006), conference papers (Chénard, 2000a; Kelly & Percival, 2009) or in relatively hard to find volumes based on conferences (Chénard, 2000b; Neufeld, 2000). It should be noted that all these references date from the 2000s. As said, furthermore, most are not peer reviewed and often are hard to find, also because the teams that produced them have since then dissolved. One of the first peer reviewed journal articles that aims to assess the performance of a part of the methodology was only published in 2014 (Li & O’Donoghue, 2014). Idem for of the alignment of a continuous variable, that was used for “wage thrift” (Dekkers et al., 2010b). These are techniques so common in dynamic microsimulation that most reports do not even mention it, while it has an important impact on the simulation results. Is it any wonder, the methodology has received serious criticism (Winder, 2000)?
It is arguable therefore that one cannot trust the development of a method until it has been road tested. One way to do this is through a rigorous peer review process. Reviewing recent papers, less than a quarter of the papers had a primarily methodological focus. There is thus a need for a literature to be developed to document, test and provide rigorous quality assurance for the alignment of the many other variables that are aligned in the literature. This paragraph cites an issue in relation to one specific methodology, one could extend this to many other methods used within the field of microsimulation.
A relatively new way of exchanging knowledge is via, literally, code. The more that research teams use the same development environment (R, JAMSIM, LIAM2, ModGen, FEM, JAS-mine), the more the exchange of code is facilitated and the more the technologies are shared, the more they are tested. Take, for example, the recent transfer of the model MIDAS from Belgium to other countries. The receiving partner gets more than a bunch of code, obviously; she gets years of knowledge and of development-by-error. The same transfer of knowledge takes place within the FEM-project (Atella et al., 2017; OECD, 2017). Finally, some “how-to” papers (Blanchet, 2014; Dekkers, 2015; Dekkers, Tarantchenko, & De Menten, 2017) often remain unpublished because they do not fit in the classical framework of a scientific publication. Yet they also may be valuable in the exchange of practical knowledge and experience.
Another potential reason for a lack of progress is the ownership model that was often used in the field. There are essentially two choices, proprietary and open-source models of ownership.
In terms of the proprietary ownership model, it can be characterised by
Code or coding consultancy has been sold to potential clients; an example of this are the Czech model NEMO, the Polish ZUS model and the Romanian SimProVision 3. These models were developed by Deloitte in Prophet based on a generic template.
National rivalries in access to models and data.
An open-source focused ownership model can be characterised by
Collective gains for everyone by joint development, with private gains via citation and scientific reputation.
Quality control and appreciation is achieved by peer-review. Note that this can also be achieved by the sharing of code. For example, LIAM2 has benefited a great deal from the scrutiny of other users and developers, including Alexis Eidelman and Mahdi Ben Jelloul, both at the Paris School of Economics.3
An emphasis on the public good nature of research, particularly publicly funded research.
The open-source model is one more in line with public research and is the basis of advancement in most fields. Proprietary models are more associated with intellectual property based commercial research. However intellectual property based ownership and dissemination models only makes sense when an economic return can be gained and incentives private research and development. Within the field of microsimulation modelling is characterised by relatively low demand for these tools by clients with the capacity to pay for them. It is thus a business model that will stymie intellectual development resulting in relatively low sharing of knowledge, code and data.
The problem that prevents “direct” open-source models from happening is that each modeller considers the cost of development borne, and weighs it against the benefit of putting it online. The latter would be a gain in terms of reputation whereas the cost could be substantial, especially if the code is made available to teams in the same country as the developer. However, in our experience, sharing code becomes considerably more straightforward if the models have the same foundation.
In terms of quantifying progress, Hoschka (1986) outlined a number of constraints and issues associated with the development of microsimulation models:
Behaviour: many of the behavioural hypotheses in micro-simulation models are of insufficient theoretical and/or empirical basis.
Access to Data: quality and accessibility of the data required by micro models often are restricted severely.
Model Development: the development of micro-models frequently needs too much time and its costs are accordingly high.
Computer Hardware: running micro models usually requires a lot of computer time
Validation: the prediction quality of micro-models has not yet been systematically evaluated and validated.
Complexity: large microsimulation models are so complex that they are difficult to comprehend and control.
Table 3 presents our opinion in relation to the improvements that have been made. In terms of behaviour, there has been significant improvements in econometrics, particularly in relation to labour supply (Blundell & Macurdy, 1999) and the retirement decision (Stock & Wise, 1990). There however is arguably less endogenous behaviour within the field of microsimulation than in the parallel fields of agent based modelling and social simulation (Davidsson, 2002). Furthermore, the simulation of structural labour supply has mostly be limited to comparative statics (i.e. models without a time element). Dynamic models where endogenous behaviour is at the core are rare, with CBOLT (Harris, Sabelhaus, & Sevilla-Sanz, 2005) and SIDD (Simulator of Individual Dynamic Decisions) (Van de Ven, 2016) as notable exceptions.
Relative to 1986, the year that the Hoschka paper was published, there is vastly improved access to micro-data, mainly as administrative data. Many countries have invested in the digital government by which a citizen should deliver information only once. The infrastructure on administrative data and the merging of various administrative datasets has been the result. The benefits of administrative data are obvious for our field. Holman et al. (2008, in Harron et al. 2017) mention large sample sizes, detailed data on hard-to-reach populations, for generating results with a high degree of external validity, and application for policy making. Additional advantages of administrative data include that the definition of the variables is grounded in legal concepts and do not depend on self-assessment by the respondents. Increasingly this data is becoming accessible to the scientific community (OPRE, 2016) although differences remain (Connelly, Playford, Gayle, & Dibben, 2016). Of course administrative data is by definition secondary data, and some aspects therefore are not covered. Traditional examples of this are educational attainment levels and training, and the composition of the social household beyond the fiscal unit. In some countries and years, this information is found by combining administrative data with Census data or Labour Force Surveys, but in other cases this information is lacking. Also, information on frontier workers and on informal money transfers is for obvious reasons often only partially available. Furthermore, administrative data is of course not collected in one go, but rather consists of linked datasets gathered for different and specific purposes. Not all fields of life are covered by this data in the same level of detail, and missing data and linkage issues may remain a problem (Harron et al., 2017). Administrative data has the potential to change social sciences, and especially our particular field of interest. It especially makes the above discussion on the sharing of code all the more relevant, because of the possibilities it offers for reproducible research (Connelly et al., 2016; Playford, Gayle, Connelly, & Gray, 2016).
However, up to today there remain legal restrictions that make it difficult for candidates outside academics to get access to administrative data. This might explain why in the European member states many dynamic microsimulation models that use administrative data can be found in the public sector (Dekkers & Van den Bosch, 2016). Furthermore, the recent EU General Data Protection Regulation (GDPR Portal, 2018) might have an important impact on the accessibility of individual data for research purposes. Finally, where administrative data is increasingly available with the above caveats, availability of survey data, especially panel data, is in Europe poorer now than in the 1990s with the demise of the European Community Household Panel.
In terms of model development, the last decade has seen a move away from the proprietary model, with greater shared access to models available for free via open-source and other sharing arrangements such as LIAM2 (De Menten et al., 2014), JAS-mine (Richiardi & Richardson, 2017, 2016) and MODGEN (Spielauer, 2006) to name a few. Computer hardware has improved hugely, so that no longer do microsimulation models need to be associated with the most significant computer hardware infrastructure as in the case of Orcutt’s earlier models, where it is possible to run most models on laptop computers.
However, the use of these models is relatively limited in the recent review. Perhaps given the greater variety of applications, model teams have not used the frameworks developed for often Pensions and Social Protection analysis. It would be useful for the field to provide a citable location for the documentation of algorithms, even if they do not use one of open-source models. Access to these algorithms would improve the diffusion of innovation within the field.
The field has also broadened with the extension of the methodology to new policy areas cited in Table 2, such as health policy (Schofield et al., 2014) rural policy (O’Donoghue, Hynes, Morrissey, Ballas, & Clarke, 2013; Van Leeuwen, 2010) and environmental policy (Hynes & O’Donoghue, 2014) and beyond OECD countries (Essama-Nssah, 2014).
As implied in the discussion on alignment above, validation remains a struggle. In our experience, model development is 10% construction and 90% validation reflecting the complexity of microsimulation models, which remains a constant battle.
In summary, one can argue that quite a good deal of progress has been made, but that knowledge transfer and proprietary structures have slowed the speed of progress. There still remain substantial methodological gaps to be filled and properly understood to provide confidence in the suite of methodologies that are currently being utilised in the field.
In this section we look to the future and consider potential directions for the field to develop and the challenges it faces. To set the scene, let us remind ourselves what dynamic microsimulation models are. They are models where the unit of analysis is at the micro level, such as households, individuals, firms or farms that simulate policy, social or economic change in an inter-temporal setting. They are therefore powerful tools for running scenarios to understand the impact of economic, social or policy processes on the distributional characteristics of the population over time.
What dynamic microsimulation models are not are forecasting models. Furthermore, dynamic microsimulation is a comparably slow process of development and validation. All this may have contributed to the perceived failure of models in the 1980s and earlier, where, anecdotally, there had been an expectation of being able to forecast rather than undertake scenario analysis.
In determining, where the field should go, let us consider what users want. In our experience, working with a wide variety of users, those who use models and their results want more:
increasing spread and use
The skill set of a microsimulation modeller is quite demanding and those who develop microsimulation models typically acquire personal skills such as:
detailed policy or sectoral knowledge
team work skills
good communication skills
As these are highly marketable and transferable skills, there is the potential for significant personal returns to investment in microsimulation modelling at the individual level.
Microsimulation modelling, and dynamic microsimulation modelling in particular, as a field is quite broad. Table 4 classifies the papers presented at the IMA World Congresses in Turin and Canberra by type of institution. The pattern is similar in both. About half come from universities, with about 20% coming from applied research institutes. The share from government was higher in Canberra with about one third than in Turin at about 20%, with Universities increasing their share to over 50%. There was a small private sector presence in both. This breadth reflects the applied policy focused nature of the field and sees interactions between both academics and policy analysts.
This policy focused nature and the mix between academics and (public sector) research is a discerning characteristic of our field, and this is one equilibrium that we should seek to maintain and improve. However, this requires a proactivity. Dekkers and Van den Bosch (2016) show that too much work on dynamic microsimulation is done in ministries or public sector institutions. A part of this remains “under the radar” and does not appear in conference presentations, and certainly not in peer reviewed scientific publications. For these organisations, working in international scientific networks, participating in conferences and workshop, and producing scientific output is not in their nature. An important task for the IMA should be to bridge the gap between these institutions and the international scientific community.
In terms of defining where the field should go next, we consider what different models stakeholders want. We first focus on what do funders of research and analysis want. These may include research funding bodies, commissioners of research and analysis by policy or industry stakeholders.
The primary requirement is usefulness and cost-effectiveness.
A secondary requirement is quality assurance; confidence that the research produced is of good quality.
Their next requirement is that the research achieves impact; and that decision makers will use the outputs of our models and analysis to affect change and make decisions.
In recent years, the pursuit of research metrics such as the number and impact factor has dominated, as for example in various research assessment exercises in different countries. Impact in this sense depends upon the journal impact factor which depends upon the citations by other researchers. These metrics do not typically contain information about end use. Increasingly the actual end use impact of publicly funded research is being questioned; what is the impact on society, policy making or in industry?
One might make the case that impact of dynamic microsimulation, albeit less visible than static microsimulation, especially through EUROMOD, is still considerable. First of all, it still exists in academia, even with the comparably low standing. But more importantly, a considerable proportion of models is being used outside academics, where the impacts on policy making need to be more direct.
Dynamic microsimulation modelling is an analytical field that takes inputs from other social science fields and can be applied to undertake ex-ante analysis of policy reform and to evaluate existing policy. Also, much of the research and model development is commissioned in order to have policy impact. However due to the nature of many of the publications in the field in conference proceedings and volumes, and the comparable important proportion of work being done outside academics, dynamic microsimulation modelling has not used traditional metrics to demonstrate quality as much as in other fields and as a result research impact factors may be lower.
We now focus on what do microsimulation modellers want. We can simply classify modellers into “developers” and “users”.
to learn from techniques, applications and best practices developed by others;
methodological development; add to knowledge and applying to new questions and analytical areas;
networking; opportunities for peer comment and learning;
validation of status and expertise through quality assurance mechanisms;
many people who utilise microsimulation modelling who do not engage with the IMA to keep abreast of all relevant developments.
Users of dynamic microsimulation model want
to know what is the most appropriate method, model or modelling technique use when confronting a particular question;
to be aware of a network that can be called upon for assessments or advice on strategic decisions in model investments;
to know how to undertake an analysis for their country that is similar to a question that has already been analysed in the literature;
quality assurance of analysis done;
validation of status and expertise through quality assurance mechanisms;
training for new users.
In terms of where to go next with regard to areas of research within dynamic microsimulation, Table 5 outlines some personal views. In terms of behaviour, the lack of behaviour based upon rigorous economic and social science theory is still a frequent criticism and where there is behaviour, there could be an improved focus on causality even if much of existing econometric literature is difficult to extrapolate into a full population policy setting. Ways to go forward include the full-integrated modelling of behaviour in the dynamic life-cycle model, for example, as in the model SIDD (Van de Ven, 2016, 2017). This, however, makes the model very complex and inflexible. Furthermore, combinations with alignment are to our knowledge not possible. An alternative could be to include a “traditional” one-period labour supply model in the dynamic microsimulation model (such as Dekkers, Decoster, & Capéau, 2014). The problem with this approach is that the decision is made again each period and independently from decisions in previous periods, or expectations on the future. A third and promising approach is the combination of dynamic microsimulation and ABM (Richiardi, 2013). In terms of access to data, the challenge remains in how to channel the huge datasets that are becoming available in this era of big data and the potential that cloud computing has in analysing and microsimulating with this data. What will be the consequences of the ever-tightening privacy rules?
As outlined above, there are still many unanswered questions in relation to what are the best methodological choices to make in the use and development of microsimulation models. Despite Pudney and Sutherland’s (1994) work on the use of confidence intervals in microsimulation models, their use is still low, with some exceptions (Creedy, Kalb, & Kew, 2007; Pudney, Hancock, & Sutherland, 2006; Richardson, Pacelli, Poggi, & Richiardi, 2018). Also, the pitfalls of using confidence intervals in combination with alignment remain to be explored. Finally, methodological approaches to explaining the simulation results remained needed in order to counter the “black box criticism” (Dekkers, 2014).
In terms of new policy applications, there are still many potential extensions of the field in other welfare policy areas such as education policy and focusing on other units of analysis such as a hospital within health microsimulation, or other collective households such as homes for the elderly. The opportunities in large, administrative, datasets also open up possibilities for multi-regional microsimulation, or to model urban and rural regions and their specificities in terms of income, health and labour market characteristics. There are also opportunities to explore big global questions like impact of climate change and greater market risk. However complexity remains a constant trade-off. We are not the first to argue caution in relation to complexity, while at the same time encouraging greater complexity in areas of analysis.
Finally, progress achieved in the development of shared models or modelling tools should continue. The first contours of networks are arising with MIDAS/LIAM2 and, separately, FEM. Working and developing together in such a network will in all phases increase productivity through the sharing of experience and the identification of best practices, allowing for a faster implementation of new techniques and a more thorough testing of them. Furthermore, these networks allow for internationally comparative analyses (for example, as presented in the latest European “Pension Adequacy Report” (European Commission (EC), 2018, p. 121), or the OECD’s “Preventing Ageing Unequally” report (OECD, 2017, p. 48, Box 1.2), which receives much more visibility than country-specific works.
An inherent risk is that these networks would themselves remain separate and that between- network exchange of techniques and best practices would remain limited. Preventing this is where the IMA would have an important role to play. This role will be discussed in more detail later in this article.
In order to maintain and enhance development we need to continue and improve the codification of our methodologies and peer review them to provide quality assurance. In the case of model development, this should be done not just by paper description but by detailing algorithms and by evaluating the model performance against objectives and alternatives. This will assist us in preventing the “reinvention of the wheel”.
There is a balance and differential focus between applied policy research and methodological research. However both rely on each other. The challenge is how to create this symbiosis. This symbiosis and the general development of the field requires
scientific knowledge transfer
training and education
In order to ensure quality, research and analysis at whatever level requires peer review. This process helps to provide quality as it requires the research or analysis to meet the standards of peers. The usefulness of the research to peers, its significance and status after publication then depends upon the number of citations of work. However within this field, we often ignore the mechanics of this process either by publishing papers as conference papers without peer review and when citing research literature in our own papers, citing working papers. Doing this both lessens the quality assurance process, even if the quality is high and reduces the capacity of performance metrics to attribute significance to the work undertaken. By using this approach, we collectively as a field reduce our visible impact to the wider world, which in turn has implications for research funding prioritisation and the attractiveness of the field to new researchers.
While we often think of employing this process simply for new research articles, there is an equal need for rigorous peer review and quality assurance for applied analytical work, say developing new models and analyses using existing methodologies. This also includes new technologies that are being introduced in open-source simulation packages. The rigorous use of these new tools by various (teams of) model developers is the best way to quickly find any problems, and assess the usefulness.
The success of the International Journal of Microsimulation (IJM) is a critical piece of infrastructure to facilitate this process. The impact factor of the Journal is important for researchers in terms of types of journals to target. The relatively new journal is now in in the top third of Repec rankings with an Impact Factor of 2.1 and is ranked ahead of many competitor journals for those who work in the area. Further, increasing the impact factor of our journal is a specific goal for our field. We want to have papers that are of relevance to our members and stakeholders. However in only publishing and citing working papers instead of peer reviewed articles, we undermine the real impact factor associated with research in the field. We should try to cite IJM articles rather than working papers. We should also think about submitting our working and technical papers for peer review, rather than seeing these non-peer reviewed papers as the end in itself. Given the very fast turnaround time of the journal publishing first in the IJM is viable opportunity rather than publishing as a working paper without compromising the capacity to get citable work into the public domain quickly.
However important for the status and perceived impact of articles, the pursuit of the impact factor should not be the sole driver of the journal’s strategy. Its role in providing quality assurance to new models and analyses in the sector is important for wider quality assurance in the sector. Indeed, submitting our working and technical papers for peer review, as we previously called for, might not necessarily add much to the impact factor of our journal, since these technical papers will only be cited by other model developers and not by those from the outside that use and interpret the results of our work. However, if we were to focus solely on improving the Impact Factor, we would probably exclude these less novel papers. The loss for our field would be important, because it is necessary that these new techniques or applications are disseminated and tested.
While we focus on improving the codification of knowledge in the sector, it should not be at the expense of the transmission of tacit knowledge. The network building gained through membership and participation in association activities is important. They provide opportunities to meet and discuss.
We should however continuously revisit our offerings and activities in this space. Until recently, the World Congress of the association was our main networking event. It has proven to be a successful conference with good attendance in different continents. However as it moves around the world, it may be difficult for everyone to engage due to resource constraints of individual and particularly junior researchers and PhD students. The association therefore decided to establish regional meetings. The first of these were organised in Europe (Hungary, 2016) the US (Washington, 2016) and Japan (Narita, 2018). They have proven to be successful, and further the possibilities for the association to support developments in our field. Finally, we also need to engage more with growing parts of the world in terms of microsimulation in China and south-east Asia, Latin America, the Middle East and Africa.
We may also need to think of alternative vehicles to facilitate knowledge transfer via, for example, the provision of training courses. These courses have begun to develop, by for example the LIAM2 or FEM, consortium or courses, also in ModGen and JAS-mine, in specific universities. It may be timely to consider more systematic training offerings.
In this paper, we have outlined some reasons for what might be argued as relatively slow progress within the field of dynamic microsimulation. Concerns in relation to lack of emphasis were identified in relation to the transmission of codified and in the sometimes proprietary ownership model. The knowledge transfer mechanism has had a sub-optimal focus on verbal and non-peer reviewed methods of knowledge transmission, with many of the key developments not codified and subjected to peer review. We highlight the expanding breadth of the field, both in terms of the number of countries and in terms of the broadening policy areas.
Moving forward, we urge an improved focus on the codification and peer review of methodologies used in the field. Publication in this journal, which has steadily improved its impact factor, is critical in underpinning the quality of analysis and methodological development in the field. In terms of tacit knowledge transmission, we encourage the organisation of more regional meetings of the association, especially in parts of the world where microsimulation is in an infant stage.
There has anecdotally been a perception of failure historically to achieve objectives of some earlier dynamic microsimulation models. However, this perceived failure have more to do with expectations being too high in relation to the speed and costs of development, as well as the capacity to “forecast” as opposed to project or undertake scenario analysis. There clearly is a need for a general discussion on what these models can, and cannot, do, and how the costs can be reduced. This latter discussion should include the identification of best practices, the exchange of know-how and the further development of networks based on open-source modelling technologies.
In terms of future areas for model development, the opportunities to incorporate behaviour to a greater extent, utilise more data in the big data revolution via the improved computing architecture in cloud computing, to extend the field of microsimulation into new policy areas such as education and big global challenges like climate change, and to improve our knowledge of the degree of certainty in relation to simulation estimates.
Note that we here define dynamic microsimulation models as those models that directly incorporate a notion of time, i.e. where individual characteristics change over time. This definition is by itself not undisputed, because the original definition (Harding, 1996, p. 4) included behavioural models used for comparative-static analysis, while excluding static-ageing models. See Dekkers (2016) for a discussion of this point.
Probably this ranking is too low because of the models that are mentioned in Dekkers and Van den Bosch (2016) that are not included in previous overviews, and that are all on pensions.
Hoschka Headings were identified at the top of Section 2.4 as constraints and issues associated with the development of microsimulation models.
Papers reviewed in preparation of recent literature review
Introducing the OECD-Schaeffer-CEIS Future Elderly ModelPaper presented at the 6th World Congress of the International Microsimulation Association.
Micro-Macro Linkage and the Alignment of Transition Processes. Technical (Paper No. 25)NATSEM, University of Canberra.
La microsimulation dynamique: principes généraux et illustrations en langage R. Document de travail Insee/DMCSI, No. M2014/01La microsimulation dynamique: principes généraux et illustrations en langage R. Document de travail Insee/DMCSI, No. M2014/01.
Labor supply: A review of alternative approachesIn: O Ashenfelter, D Card, editors. Handbook of Labor Economics. Elsevier. pp. 1559–1695.
Accrued-to-date pension entitlements in Belgium. (Working Paper No. 6-17)Brussels: Federal Planning Bureau.
Microsimulation in the New MillenniumValidation of longitudinal dynamic microsimulation models: experience with CORSIM and DYNACAN, Microsimulation in the New Millennium, Cambridge University Press.
Earnings in DYNACAN: distribution alignment methodologyPaper presented at the 6th Nordic Workshop on Microsimulation.
Microsimulation Modelling for Policy Analysis: Challenges and InnovationsIndividual alignment and group processing: an application to migration processes in DYNACAN D, Microsimulation Modelling for Policy Analysis: Challenges and Innovations, Cambridge University Press.
An Overview of CBOLT: The Congressional Budget Office Long-Term ModelReport of the Congressional Budget Office.
The role of administrative data in the big data revolution in social science researchSocial Science Research 59:1–12.
Confidence intervals for policy reforms in behavioural tax microsimulation modellingBulletin of Economic Research 59:37–65.
Agent Based Social Simulation: A Computer Science ViewJournal of Artificial Societies and Social Simulation, 5, 1.
LIAM2: a New Open Source Development Tool for Discrete-Time Dynamic Microsimulation ModelsJournal of Artificial Societies and Social Simulation 17:9.
What are the driving forces behind trends in inequality among pensioners? Validating MIDAS Belgium using a stylized modelIn: G Dekkers, M Keegan, C O’Donoghue, editors. New Pathways in Microsimulation. Ashgate Publishing Limited. pp. 287–304.
On the modelling of immigration and emigration using LIAM2, FPB Note LIAM2-11155, Federal Planning Bureau, BrusselsOn the modelling of immigration and emigration using LIAM2, FPB Note LIAM2-11155, Federal Planning Bureau, Brussels.
The Flip Side of the Coin: The Consequences of the European Budgetary Projections on the Adequacy of Social Security PensionsEuropean Journal of Social Security 12:94–120.
The Flip Side of the Coin: The Consequences of the European Budgetary Projections on the Adequacy of Social Security PensionsEuropean Journal of Social Security 12:94–120.
Life-Cycle Income Analysis ModellingWhat are the consequences of the European AWG-projections on the adequacy of pensions? An application of the dynamic micro simulation model MIDAS for Belgium, Germany and Italy, Life-Cycle Income Analysis Modelling, Lambert Academic Publishing.
What are the consequences of the AWG 2018 projections and hypotheses on pension adequacy? Simulations for three EU member statesReport of the Federal Planning Bureau, Brussels, (forthcoming).
Integrating a random utility random opportunity labour supply model in MIDAS Belgium: presentation of on-going workPaper presented at the European Meeting of the International Microsimulation Association, Maastricht School of Governance, UNU-MERIT.
On using dynamic microsimulation models to assess the consequences of the AWG projections and hypotheses on pension adequacy: Simulation results for Belgium, Sweden and HungaryMimeo Federal Planning Bureau, Brussels and Central Administration of National Pension Insurance (ONYF), Budapest.
Using Links in LIAM2. FPB Note LIAM2-11672Brussels: Federal Planning Bureau.
Applications of microsimulation modellingProspective microsimulation of pensions in European Member States, Applications of microsimulation modelling.
The European Network for Dynamic Microsimulation (EURODYM) – A vision and the state of affairsInternational Journal of Microsimulation 4:100–105.
Agent-based Models in Economics: A ToolkitAgent-based Models in Economics: A Toolkit, Cambridge, University Press.
Handbook of Microsimulation. Contributions to Economic Analysis 263135–166, Distributional Change During Development, Handbook of Microsimulation. Contributions to Economic Analysis 263.
The Pension Adequacy Report 2018: current and future income adequacy in old age in the EU. Volume I. Publications Office of the European UnionThe Pension Adequacy Report 2018: current and future income adequacy in old age in the EU. Volume I. Publications Office of the European Union.
Technical Compilation Guide for Pension Data in National Accounts. Eurostat Methodologies & Working papersEurostat.
https://www.eugdpr.orgEU General Data Protection Regulation (GDPR) portal.
Agent-Based Modelling in EconomicsJohn Wiley & Sons.
Behavioural Effects of Social Security Reform in a Dynamic Microsimulation with Life-cycle Agents (Working Paper Series 2005-6). Congressional Budget OfficeBehavioural Effects of Social Security Reform in a Dynamic Microsimulation with Life-cycle Agents (Working Paper Series 2005-6). Congressional Budget Office.
Challenges in administrative data linkage for researchBig Data & Society, 4, 2, http://doi.org/10.1177/2053951717745678.
A decade of data linkage in Western Australia: strategic design, applications and benefits of the WA data linkage systemAustralian Health Review: A Publication of the Australian Hospital Association 32:766–777.
Microanalytic Simulation Models to Support Social and Financial PolicyRequisite research on methods and tools for microanalytic simulation models, Microanalytic Simulation Models to Support Social and Financial Policy, North-Holland.
Environmental ModelsIn: C O’Donoghue, editors. Handbook of Microsimulation. Contributions to Economic Analysis, 263. Emerald Group Publishing Limited. pp. 449–478.
Nonlinear Alignment by Sorting. (Working Paper CORSIM)Nonlinear Alignment by Sorting. (Working Paper CORSIM).
Longitudinal benchmarking and alignment of a dynamic microsimulation modelPaper presented at the 2nd General Conference of the International Microsimulation Association.
A survey of dynamic microsimulation models: uses, model structure and methodologyInternational Journal of Microsimulation 6:3–55.
Evaluating Binary Alignment Methods in Microsimulation ModelsJournal of Artificial Societies and Social Simulation 17:15.
Dynamic ModelsIn: C O’Donoghue, editors. Handbook of Microsimulation Modelling, 293. Emerald Publishing Ltd. pp. 305–343.
Make it so: event alignment in dynamic microsimulationOttawa: DYNACAN Team.
Mirosimulation in Government Policy and ForecastingAlignment and Variance Reduction in DYNACAN, Mirosimulation in Government Policy and Forecasting, North-Holland.
Dynamic Microsimulation: A Methodological SurveyBrazilian Electronic Journal of Economics 4:77.
Handbook of Microsimulation ModellingEmerald Group Publishing Limited.
Spatial Microsimulation for Rural Policy AnalysisSpringer-Verlag.
Preventing Ageing UnequallyOECD Publishing Paris.
Using administrative data in social policy research (OPRE Report No. 2016-62)Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, Washington, DC.
A New Type of Socio-Economic SystemReview of Economics and Statistics. Reprinted with Permission in the International Journal of Microsimulation 1:3–9.
Simulation of Economic SystemsThe American Economic Review 50:894–907.
Microanalysis of socioeconomic systems; a simulation studyHarper and Brothers.
Policy exploration through microanalytic simulationPolicy exploration through microanalytic simulation, Urban Institute.
Microanalytic simulation models to support social and financial policyMicroanalytic simulation models to support social and financial policy, Elsevier Science.
Administrative social science data: The challenge of reproducible researchBig Data & Society 3:1–13.
http://www.praxis.ee/en/works/accrued-to-date-pension-entitlements/Accrued to Date Pension Entitlements.
Simulating the Reform of Means-tested Benefits with Endogenous Take-up and Claim CostsOxford Bulletin of Economics and Statistics 68:135–166.
How reliable are microsimulation results?: An analysis of the role of sampling error in a U.K. tax-benefit modelJournal of Public Economics 53:327–365.
Female labour force projections using microsimulation for six EU countriesInternational Journal of Microsimulation, forthcoming.
Artificial Economics and Self Organisation. Agent-Based Approaches to Economics and Social SystemsThe missing link: AB models and dynamic microsimulation, Artificial Economics and Self Organisation. Agent-Based Approaches to Economics and Social Systems, Lecture Notes in Economics and Mathematical Systems, Vol. 669, Springer.
JAS-mine: A new platform for microsimulation and agent-based modellingInternational Journal of 'Microsimulation 10:106–134.
Agent-based Computational Demography and Microsimulation using JAS-mineIn: A Grow, J Bavel, editors. Agent-Based Modelling in Population Studies. Springer. pp. 75–112.
Health ModelsIn: C O’Donoghue, editors. Handbook of Microsimulation Modelling (Contributions to Economic Analysis, Volume 293). Emerald Group Publishing Limited. pp. 421–447.
The “LifeCourse” Model, A Competing Risk Cohort Microsimulation Model: Source Code And Basic Concepts Of The Generic Mirosimulation Programming Language Modgen (MPIDR Working Paper)Max Plank Institute for Demographic Research.
Pensions, the Option Value of Work, and RetirementEconometrica 58:1151–1180.
Multi-Country MicrosimulationIn: C O’Donoghue, editors. Handbook of Microsimulation Modelling. Emerald Group Publishing Limited. pp. 77–106.
The Simulator of Individual Dynamic Decisions (SIDD)In: G Dekkers, J Mészáros, editors. Applications of microsimulation modelling. Társadalombiztosítási Könyvtár. pp. 128–152.
Exploring the Importance of Incentive Responses for Policy ProjectionsInternational Journal of Microsimulation 10:134–164.
Urban-Rural Interactions: Towns as Focus Points in Rural Development115–135, Microsimulation of Rural Households, Urban-Rural Interactions: Towns as Focus Points in Rural Development, Springer.
Modelling within a thermodynamic framework: a footnote to Sanders (1999)Cybergeo: European Journal of Geography.
Preface – Orcutt’s vision 50 years onIn: A Zaidi, A Harding, P Williamson, editors. New Frontiers in Microsimulation Modelling. Farnham: Ashgate. pp. 21–30.
A Bayesian approach to labour market modeling in dynamic microsimulation (No. 201301)Danish Rational Economic Agents Model, DREAM.
Microsimulation for Clinical Decision-Making in Individual Patients With Established Coronary Artery DiseaseCirculation Journal 77:717–724.
Health benefits and cost effectiveness of endoscopic and nonendoscopic cytosponge screening for Barrett’s esophagusGastroenterology 144:62–73.
Reforging the wedding ring: Exploring a semi-artificial model of population for the United Kingdom with Gaussian process emulatorsDemographic Research 29:729.
An Investigation of the Sensitivity of a Dynamic Microsimulation Model of Urban Neighbourhood DynamicsIn: G Dekkers, M Keegan, C O’Donoghue, editors. New Pathways in Microsimulation. Farnham: Ashgate. pp. 306–319.
Simulating the need for health-and elderly care in Sweden-A model description of Sesim-LEVIn: G Dekkers, M Keegan, C O’Donoghue, editors. New Pathways in Microsimulation. Farnham: Ashgate. pp. 41–60.
T-DYMM: the treasury dynamic microsimulation model of the Italian pension system
Immigration and Diversity in 2036: New Projection Results for Canada using DemosimPaper Presented at the World Congress of the International Microsimulation Association.
LifeLossMOD: A microsimulation model of the economic impacts of premature mortality in AustraliaInternational Journal of Microsimulation 7:33–52.
Probabilistic household forecasts based on register data-the case of Denmark and FinlandDemographic research 28:1263–1302.
Sub regional estimates of morbidities in the English elderly populationHealth & Place 27:176–185.
Reforming old age security: Effects and alternatives. CIRANO Working Papers 2015s-11CIRANO.
Modelling the distribution of returns on higher education: A microsimulation approachEconomic Modelling 38:328–340.
The impact of demographic change on policy indicators and reforms. FLEMOSI Working PaperThe impact of demographic change on policy indicators and reforms. FLEMOSI Working Paper.
Liam2: a new open-source development tool for discrete-time dynamic microsimulation modelsJournal of Artificial Societies and Social Simulation 17:9.
The simulation properties of microsimulation models with static and dynamic ageing-a brief guide into choosing one type of model over the otherInternational Journal of Microsimulation 8:97–109.
On using dynamic microsimulation models to assess the consequences of the AWG projections and hypotheses on pension adequacy: Simulation results for Belgium, Sweden and Hungary
Importance of the order of the modules in TransMobPaper Presented at the World Congress of the International Microsimulation Association.
Development of a microsimulation of melanoma mortality for evaluating the effectiveness of population-based skin cancer screeningMedical Decision Making 35:243–254.
Information Diffusion in Multi-Agent CommunitiesPaper Presented at the World Congress of the International Microsimulation Association.
Microsimulation analysis of financing options for long-term services and supportsWashington, DC: Urban Institute.
Financing long-term services and supports: Options reflect trade-offs for older Americans and federal spendingHealth Affairs 34:2181–2191.
Synthetic population dynamics: A model of household demographyJournal of Artificial Societies and Social Simulation 16:8.
Simulating the health status of the Luxembourgish elderly in the longer runPaper Presented at the World Congress of the International Microsimulation Association.
Testing the statistical significance of microsimulation results: A pleaInternational Journal of Microsimulation 6:50–77.
Equity versus efficiency in public pension schemes: Compensating for joint work-family decisionsPaper Presented at the World Congress of the International Microsimulation Association.
Modeling household formation and housing demand in Denmark using the dynamic microsimulation model SMILEDREAM Conference Paper, 4th General Conference of the International Microsimulation Association.
Leveraging demographic and epidemiological methods to improve mortality projections in a population-based health microsimulation modePaper Presented at the World Cogress of the International Microsimulation Association.
The Population Health Model (POHEM): an overview of rationale, methods and applicationsPopulation health metrics 13:24.
Modeling income dynamics for public policy design: An application to income contingent student loansEconomics of Education Review 37:273–285.
The Effect of Changes in Nuptiality Behavior after the 1980s on the Poverty Rate for the Elderly in Japan-Analysis Using a Dynamic Microsimulation ModelThe Review of Socionetwork Strategies 8:1–18.
Microsimulation of the Impoverishment of Elderly Women in JapanPaper Presented at the World Congress of the International Microsimulation Association.
The uncertainty of public health microsimulation model predictionsPaper Presented at the World Congress of the International Microsimulation Association.
Changes in the income distribution of the Dutch elderly between 1989 and 2020: a dynamic microsimulationReview of Income and Wealth 59:460–485.
Two Pension Microsimulation Models for SloveniaPaper Presented at the World Congress of the International Microsimulation Association.
Stroke and active life expectancy in the United States, 1999–2009Disability and Health Journal 7:472–477.
Occupation, education and social inequalities: a case study linking survey data sources to an urban microsimulation analysisIn: F Pagliara, M de Bok, D Simmonds, A Wilson, editors. Employment Location in Cities and Regions: Models and Applications. Berlin, Heidelberg: Springer. pp. 203–222.
The role of microsimulation in the development of public policyIn: M Janssen, M Wimmer, A Deljoo, editors. Policy Practice and Digital Science: Integrating Complex Systems, Social Simulation and Public Administration in Policy Research. Cham: Springer. pp. 305–320.
Determinants and disparities: a simulation approach to the case of child health careSocial Science & Medicine 128:202–211.
Learning and search strategies in the context of hidden-action problemsPaper Presented at the World Congress of the International Microsimulation Association.
New Pathways in Microsimulation203–216, Combining EUROMOD and LIAM tools for the development of dynamic cross-sectional microsimulation models: a sneak preview, In, O’Donoghue, C, New Pathways in Microsimulation, Farnham, Ashgate Publishing Limited.
A spatial microsimulation approach for the analysis of commuter patterns: from individual to regional levelsJournal of Transport Geography 34:282–296.
Immigration, Diversity, Human Capital and the Future Labor Force of Developed Countries: the European ModelPaper Presented at the World Congress of the International Microsimulation Association.
Downscaling heterogeneous household outcomes in dynamic CGE models for energy-economic analysisEnergy Economics 65:87–97.
Estimation of rates of return on social protection: ex ante microsimulation of social transfers in CambodiaJournal of Development Effectiveness 8:67–86.
Modelling private wealth accumulation and spend-down in the Italian microsimulation model CAPP_DYN: A lifecycle approachInternational Journal of Microsimulation 6:76–122.
Assessing the Incentives for Voluntary SavingsPaper Presented at the World Congress of the International Microsimulation Association.
A New Normal?: How Might Savers and Governments React to Lower Returns?Paper Presented at the World Congress of the International Microsimulation Association.
Combining agent functional types, capitals and services to model land use dynamicsEnvironmental Modelling & Software 59:187–201.
Development of a population-based microsimulation model of physical activity in CanadaHealth Reports 24:11.
Generating a dynamic synthetic population-using an age-structured two-sex model for household dynamicsPloS one 9:e94761.
Handbook of microsimulation modellingEmerald Group Publishing.
Nowcasting in microsimulation models: a methodological surveyJournal of Artificial Societies and Social Simulation 17:12.
Inequality, Redistributive Policies and Multiplier Dynamics in an Agent-Based Model with Credit RationingItalian Economic Journal 3:367–387.
Eurace Open: an agent-based multi-country modelPaper Presented at the World Congress of the International Microsimulation Association.
Conditional inference trees in dynamic microsimulation-modelling transition probabilities in the SMILE modelConference paper for the 4th General Conference of the International Microsimulation Association.
Artificial Economics and Self Organization. Agent-Based Approaches to Economics and Social Systems. Springer, Lecture Notes in Economics and Mathematical Systems, vol. 6693–15, The missing link: AB models and dynamic microsimulation, Artificial Economics and Self Organization. Agent-Based Approaches to Economics and Social Systems. Springer, Lecture Notes in Economics and Mathematical Systems, vol. 669, Berlin.