Pledge to Open Science - 2024 review
In 2019, as I was approaching the end of my doctorate, I decided to commit to open science practices. Why? Can’t say it any better than Erin McKiernan, who inspired me to do this.
If I am going to “make it” in science, it has to be on terms I can live with.
More than four years have passed. It is time to look back and assess how truthful I have been to my own commitment. And maybe make a couple of adjustments.
Spoiler alert: I’m writing this primarily for myself. So, unless you are somehow particularly interested in what’s happening in my life, you should close this tab and move on with your day.
Still reading? Let’s go then!
Step 1: Assessment
I will not edit, review, or work for closed or hybrid access journals or conferences.
This is an easy one: The pledge gave me an ethically acceptable to say “no” to reviewing requests, which, as we all know, consume a lot of time. I genuinely regretted saying “no” once when I was invited to the TPC of RTSS, a top-notch conference I really like. I made one exception and agreed to review for a small workshop that a friend of mine was chairing, which wasn’t open access but one operational research. That was another of my weak points! While drafting this post, I also reviewed a paper for CACM (a hybrid journal) because that paper perfectly matched my current research. So, two exceptions.
There is probably a cost to saying “no” regarding establishment/reputation as an early career researcher: being part of such PCs is a very effective way to get known by senior community members. But since I anyway changed fields after my doctorate, this did not matter much to me; I wouldn’t get invited since no one knew me (yet).
So it was pretty easy and relatively painless. I did break the pledge a bit, though. Could do better.
I will publish only in open access journals or conferences; at least when I am the lead author.
That’s the trickiest point; let’s have it right now.
Since my pledge, I have published only one paper in a hybrid conference (for which we paid the open access fee). However, in all honesty, I have submitted several other papers to hybrid conferences. We just got rejected. So, pledge broken?
Well… yes. But I don’t feel bad about it.
Objectively, there are not many gold open-access options in my field. USENIX conferences are great, but getting accepted is tough. Besides, there is too much randomness in the reviewing process to sensibly ask the Ph.D. students I work with to wait just until the next NSDI deadline, and hope we get in this time… For better or worse, students need the papers to graduate, and it feels misplaced to let my self-imposed rules add more constraints for them. So, for the students I advised, I always put the open-access option on the table if there was one, but I let it go if a hybrid venue was preferred.
For the work I led, I could be tighter on my commitment. This record is still clean, but again, in all honesty, I plan to submit to hybrid venues soon. Why? In short: Picking my battles. I really believe in “real” open access and that it is worth fighting for. I also know it is not the most pressing issue on our hands right now.
Over the past two years, I got into sustainability research. I think advancing science in that area is much more pressing than open-access activism. I chose to maximize the chance of my work contributing to those advances, which means publishing (well, submitting…) in the most appropriate and reputable venues in my community. So, I chose not to make open access a strict requirement, only a (strong) preference.
So, publishing only in open-access venues? Broken.
To balance my karma, I pushed extra hard on some other points in my pledge. But let’s focus on one point at a time.
I will post pre-prints, when possible.
Easy to do, and I think I’ve done it whenever relevant. Check.
I will promote open review principles.
I’ve done this whenever I had the opportunity. Concretely, I took part in two projects related to open review.
The first project is the co-creation of JSys, a scholarly-led journal founded upon many progressive open science practices. These include open reviews and–of course–a real open-access publishing model (known as diamond open access in the publishing jargon), among many others. JSys is now in its fourth year of activity; submission numbers are low, but the quality of published papers is high. I am pretty proud of what we achieved, to be honest, especially given the limited time that we can afford to dedicate to it… That’s always the challenge with scholarly-led stuff. Scholars are too busy. But anyway.
The second “project” was essentially participating in a call for proposals to refresh the conference organization in my community. I made several, including the adoption of an open review model. Interestingly (but not surprisingly), this was one of the most debated proposals, with strong opinions for and against it. I hope we can convince the community to try it out and experiment, but nothing concrete on the horizon… We’ll see.
Objectively, I’ve done my fair share of promoting open review. Check.
I will use open-source software, when possible; if not, I will favor license-free software.
I’ve definitely done that, but it’s pretty effortless in my field, as most software tools are open-source. Check.
- I will share code and other artifacts, when possible.
This one is not effortless, but I’ve done it too. One concrete challenge is defining the limit of what artifacts are useful/reasonable to share. One can always do “more.”
I also think it’s hard for academics to assemble nice artifacts because we don’t do it often. If one follows best practices in software development and data management from the beginning of a project, then sharing artifacts later becomes (relatively) easy. The problem is that students often don’t know those best practices, and supervisors don’t follow the technical details of projects closely enough to catch issues early–and they may also not know those best practices either. At least, that’s the perception I have in my corner of academia.
Generally, I think the artifact situation is improving in our field. Many conferences encourage artifact sharing in one way or another, artifact evaluation committees are becoming more and more common, pseudo-standards emerge. There is still lots of room for improvement, but there is progress. One can also share artifacts at submission time, even if the reviewing is double-blind. Some nice tools make this easy enough, but that will be for another post.
Artifact sharing, check.
I will share raw and curated data.
This is somewhat redundant to the previous point. So far, I’ve done it too. It might start getting trickier for me as my current research uses proprietary data that I won’t be able to publish in its raw form. Let’s see how I manage this in the future.
For now, data sharing, check.
I will promote the DORA principles
I think the DORA principles are extremely important. Many universities, including mine, have already signed DORA; which is great! The real challenge now lies in getting the principles applied by the evaluation committees. That’s a lot trickier, but there was little I could do about this since I haven’t been in any such committees yet. So, wait and see.
Check.
I will use CRediT and publish a contributor’s record with all my research outputs.
I have done this with most of my papers. Over time, I’ve built some simple tools to make gathering data for and compiling a CRediT report easier. I published some parts on GitHub. Here are examples of a stand-alone report or one included directly in a paper (appendix, p.17).
Objectively, I think the main value of pushing for CRediT is to make people more aware of the diversity of possible contributions to a research project. I don’t think it can fix or improve the hiring decision process (sorry…). But hey, the cost is also minimal if we’ve got the tools to do it, and more transparency is always nice. I’ll keep pushing for CRediT.
Check.
I will encourage professional societies to support open access.
This overlaps almost 100% with the previous point about open review. I won’t repeat it. Open access is essential; I’ll keep pushing for it whenever possible.
Check.
I will communicate about my research to the general public.
I have done quite a lot of science communication during my doctorate. I enjoyed it, but sadly, I no longer have much time for it. I do think it is an essential part of my work as a researcher, even more so now that I work on sustainability.
One thing I did was speak about my work on a podcast and at a network operator’s conference, but I would not count either of these as the “general public.”
So, science communication: Could do better.
I will speak out about my choices.
Easy. Check.
I will regularly and honestly assess my fulfillment to this pledge.
It’s the first time I’ve done this self-assessment, so I can’t say anything about the regularity yet. It’s been honest, though. Okay, check.
So, let’s sum things up.
[Table]
Step 2. Adjustment
I still believe strongly in all the points on the list. Some are somewhat redundant, but that’s fine. However, there are two things I want to add.
- I will promote the adoption of ORCID.
- I will limit my professional long-distance flying to one trip per year.
In practice, I have been doing both of these for a while already. I think it’s just time I make it official.
Step 3. Advocacy
I made this pledge, and I just reviewed it. Most things I did pretty okay at, and I most notably failed on what would obviously be the most challenging thing: restricting my publishing options.
Now, what? What’s the takeaway? Does such a pledge make any sense?
It does to me. I think taking that pledge influenced my actions and nudged them to be more aligned with my principles. Of course, one may argue that such formalism is not necessary. It isn’t. But it helps. It helps me, at least.
It helps because it’s very easy to let our principles take the back seat when making decisions. We think, “It’s okay for this time,” or “Oh, but that is such a great opportunity!” I know, I’ve done it too (see above). But it is harder to find excuses when we pledged before knowing which decision we would have to make. I committed to this; if I make this particular decision now, I’ll break my commitment. It’s as simple as that.
This does not mean I see my pledge as a dogma that I must follow no matter what. As I wrote above, I did break it a few times. I think that’s okay as long as I remain honest with myself. If exceptions become the rule, I should update my pledge. Ultimately, it’s only about being “on terms I can live with.” My pledge, my rules.
What about yours?