Tag Archives: social simulation

Postdoc simulation analysis (spoiler alert: job insecurity is bad)

Once again I’ve been working with the academic job security simulation again.  Yesterday I’d finished altering the research funding model so that our poor agents no longer lived in a world of government largesse where population increases are always matched by an increase in funding to keep grant acceptance rates at 30%.

After tweaking things a lot last night and earlier today, I found that a funding level set at an initial 30% with a 2% increase per timestep led to research output levels very close to the previous version of the model.  The proportion of grants funded slowly drops over the course of 100 timesteps, heading from that starting 30% down to about 17% at the end of an average run.

I also added a simple retirement mechanism to this version: after 40 semesters, agents start to think about retirement and have a fixed chance (20% at the moment) of leaving the sector forever.  The result of this is a significant rise in the return-on-investment measure as the senior academics start to leave the sector; seems we had a lot of senior academics coasting along without producing much in the way of research!  Compared to the previous version, the older academics produce significantly less research-wise — I’m presuming this is because the rich-get-richer aspect of the increasingly competitive funding environment leads to a larger proportion of failed applicants deciding to bow out of the rat-race altogether.

Having taken a brief look at all this I decided to test the feedback given to me at Alife XV.  In the initial simulation, promotions had a huge positive impact on research output regardless of whether they were made entirely at random or based on research quality.  Several people at the conference suggested that this may no longer be the case if I implemented a more constrained funding system.

So, I ran the simulation 800 times across a range of parameter values with limited funding and retirement mechanisms both turned on.  I then used my old pal, the software called Gaussian Emulation Machine for Sensitivity Analysis (GEM-SA), to crunch the numbers and come up with a statistical model of the agent-based model, and then ran that 41,000 times.  The final output of interest is the total research produced across the agent population at the end of the simulation.  The analysis looks like this:

gemsa1-cropped

Turns out my colleagues were onto something, which I expected (and hoped for, because otherwise that might mean the simulation might have some problems).  In this version of the sim altering the chances of promotion for postdocs does little on its own, accounting for only 0.11% of the output variance.  This factor interacts with the level of stress induced by impending redundancy, however, and this interaction accounts for 11.03% of the output variance.

The largest effect here is driven by Mentoring levels — the amount of research boost given to newly-promoted postdocs.  Second-largest is the stress caused by looming redundancies.  This is a significantly different result from the previous version of the simulation — I’ll run a parameter sweep of promotion levels later as well, to get the complete picture.

For the sake of completeness, here’s the graph of the main effects produced by GEM-SA:

gemsa2-cropped2

Tomorrow I’m hoping to do a similar analysis, but this time leaving Mentoring at a lower, constant level and varying a slightly different set of parameters.  My poor laptop needs a break for a little while, it’s pumping out crazy amounts of heat after all this number-crunching.

My other, larger task is to come up with a way to measure the overall human cost of this funding/career structure.  I think I can make a good case at this point that job insecurity is not great for research output in the simulation, given that across many thousands of runs I’ve yet to find a single one in which insecure employment produces more research for the money than permanent academic jobs.  I’d to be able to compare scenarios in terms of human cost as well, so perhaps taking a look at total redundancies after 100 semesters as the final output for some analyses might give me some ideas.

That aside I think I’ve made a decent start on an extension of the conference paper.  Thanks to all those who came to the talk in Mexico and gave me some useful feedback!

 

Advertisements
Tagged , , , ,

Returning to the postdoc simulation

I’ve been doing some tests on my postdoc simulation today. As suggested by colleagues at Alife XV I’ve implemented a funding system in which the total available funding increases at a lower rate than the population, leading to increased competition.

Total research output under these conditions does increase pretty significantly — however, return-on-investment remains negative, meaning we still would get more for our money by hiring half as many permanent researchers instead of postdocs. Postdocs are still the group producing the lion’s share of the actual scientific work, while permanent academics nearly all of their research time to grant-writing.
 
The return-on-investment is less negative than under the previous funding condition, but bear in mind the simulation currently doesn’t account for redundancy payments or training costs for new postdocs.  In these runs results were showing a return of -2.5 papers per unit of funding invested as compared to a postdoc-free scenario; in the unlimited-funding condition with the same settings, the figure averaged -3.6/unit.  Redundancies were higher in this condition, about 150 more each run than in the unlimited funding condition.  This could change significantly, however, depending on the final formula I use for year-on-year research budget increases, given that postdocs’ fates are directly tied to how much research money is available.
 
The big question is whether under this condition we still see no improvement in research output or return-on-investment when candidates for promotion to permanency are selected by quality rather than randomly. At this early stage there’s little difference — I’ve only done a few runs, but non-random promotions have not demonstrated a significant difference from random ones in either total output or ROI.  We’ll see if that changes when I do a larger sequence of runs.

My next test after that will be to try this version of the simulation with much longer runs to see if things stabilise at all, or whether the uncertainty introduced by the high-turnover postdoc population continues to drown out any attempts at rewarding high achievers with more grants.  We’re already looking at 50-year runs here, though, so if after two centures of terrible job security we see hugely better cost efficiencies I’m still not sure that’s a massive win for the postdoc side of things.  But I suppose that rests on whether you care about the human costs or not.

There’s a lot of tweaking to be done so these are very early days, but it’s an interesting first result.
Tagged , , , ,

Modelling Research Careers

Thanks to the stimulating discussions that came about during and after the recent Simulating the Social Processes of Science workshop, I’ve been making a start on a research proposal which combines two of my major interests: simulating institutions and social interactions; and inequalities in research careers.

At the moment, my colleagues and I are putting together an executive summary of our idea, which is still taking shape. We hope to develop computer simulations of the current career structure of the academic community, focusing on the recent explosion of insecure short-term contracts for postdoctoral researchers. Here’s a sneak preview of the executive summary:

Motivations

The academic community in the UK has become an increasingly casualised workforce in recent years; some 74% of researchers are on fixed-term contracts. Insecure employment can have significant impact on individual researchers, such as increased stress levels or reduced productivity due to the need to spend significant time searching for further work. However, the systemic impact of this trend on academic institutions and on the broader research community has yet to be investigated in any significant way.

As a result of the prevalence of fixed-term contracts, academic institutions face numerous challenges: a much-increased rate of staff turnover; regular and frequent loss of specialized skill-sets; high costs of training new staff; and the inability to retain skilled young researchers with high potential. More broadly, the academic community as a whole may face a loss of overall productivity as increasing numbers of young researchers lose research time to complications of the career structure, and the consequent lack of sustained, long-term research efforts due to the short-term focus necessitated by fixed-term work. This research programme will examine the impact of the career structure of academia on research productivity using innovative modelling frameworks.

Aside from the obvious self-interest at play here, in that I’m currently stuck in this situation myself, what I find most compelling about this idea is what we may learn about the structural problems of academia. As the use of fixed-term contracts has been increasing, we’ve seen a number of fundamental shifts in the ways that universities operate. We see a much greater emphasis on attracting international students, competing for international recognition, and an ever-expanding management structure which puts academics under constant, ceaseless scrutiny. Understanding the effects of these changes will be a major part of this proposal, and I hope that the insights we gain from this work might help us develop alternative approaches to conducting research — approaches that might help academics regain their autonomy and job security.

The next major step in fleshing out this proposal is to develop our theoretical framework more.  Focusing on the impact of research career structures on research outputs will help us to make the case for this work to potential funders, who will certainly have an interest in discovering how our current structures might be made more productive.  But at the same time, looking more deeply at how these structures have evolved and what institutional changes in universities have facilitated these problems will be more enticing to other working academics who might be interested in collaborating with us.

So, much work remains to be done.  Comments and ideas are always welcome.

Tagged , ,

4th World Congress on Social Simulation

I’m currently attending the 4th World Congress on Social Simulation, which is being held in Taipei, Taiwan at National Chengchi University.  I gave a presentation today entitled ‘Semi-Artificial Models of Populations: Connecting Demography with Agent-Based Modelling’.  I enjoyed giving the talk, particularly the encouraging and valuable feedback I received from colleagues from Russia, Japan and America.

I’ve uploaded my slides — bear in mind they were written in somewhat of a rush!

As for Taiwan, so far it’s been fantastic.  The streets are lively and clean, public transport is fast, cheap and reliable, the food is great, and people have been very polite and helpful.  Taipei 101 was particularly impressive; the building design is striking and the views are spectacular.  I’m looking forward to seeing more sights during the rest of my week here in Taiwan!

Tagged , , ,