Add tips on project planning #284
Conversation
|
In order to make this chapter complete, you can follow, or at least reference the (excellent) Turing way chapter on project design : https://the-turing-way.netlify.app/project-design/project-design.html |
|
I think we have two different things here. The chapter in the Turing Way seems to be about facilitating an open source project, and it lists some things that are needed to make it so that random people on the internet can contribute to the project if they feel for doing that. Then there's a bit of requirements engineering, and something about sensitive data. Altogether, it's a pretty random collection of things that may or may not be useful, depending on the scale of your project. Barbara's text is more about how to organise your work in the context of an NLeSC project, which puts it in scope for the Guide rather than the Turing Way. It's about how to figure out what the goal of the project is, and how to then work towards that goal. And while the project protocol can tell you which meetings to organise and documents to produce, and the PM how much time you have left, and your line manager how to spend your personal development budget, as a Lead RSE you're expected to know how to run the project yourself. Most of our engineers however come here as trained scientists with a bit of programming experience. They've never done a collaborative project before in which someone else is the scientist and they're the RSE. They don't know how we work, and the Guide is here to teach them. So far, it mostly contains technical things, but how to organise your project is at least as, and probably more important. So I'm happy to see someone pick this up and contribute a chapter, I think it will be very helpful to have. |
|
@nielsdrost, weren't you working on something similar for the project management protocol? Does it make sense to join forces with this effort? |
| ### Know where you are | ||
|
|
||
| A lot of decisions you will make along the line depend on the exact kind of project you are working on. | ||
| Knowing the project and the place you are starting from is therefore an important initial step. |
There was a problem hiding this comment.
Maybe something here about reading up on the domain? Papers, books, etc. To be able to talk to the partners on an equal level.
| Make a list, as comprehensive as possible, of all the different steps you need to take and elements you want to complete as part of this project. | ||
| The resolution of this list should not be too high; at this point, think about the project as a whole, and don’t lose yourself in details like individual issues or work items. | ||
|
|
||
| Also, do not make this list alone: work together with the Lead Applicant, they are the designer of the project, after all. |
There was a problem hiding this comment.
I would phrase this differently. We design the project together (based on their proposal, of course, but sometimes it can radically differ after we dove into it). If they want someone to "just" execute their plan, they should hire a software developer.
There was a problem hiding this comment.
I think of this as the scientists deciding on the research question they want to ask (I'm happy to help if they want me to, but it's their call in the end), scientists and RSEs deciding together on methodology, and RSEs being responsible for how the code is developed.
We do now have the Software Sustainability call in which we actually design the project together in the sense that we co-author the proposal, but that's only a small number of projects.
There was a problem hiding this comment.
Indeed, the research question is the partners' main input. Answering the question is what you do together. That can even go further than just deciding on methodology together. RSEs are researchers. They can write the paper, they can present the results at conferences. Sure, the results will be mostly digital methods, but that goes for much of the work of "regular" researchers as well. The point I want to make (here and above as well) is that we are equal partners. We shouldn't constrict ourselves, but rather assert ourselves. We are demand-driven and the demand is a question and some proposed solutions (often sensible ones, of course). After that, all bets are off ;) In practice, this comes down to many of the great tips given in this PR, but just keeping this in mind I think may help in subtle word choices that either set us free to be co-researchers or tie us down into a subordinate role. Most of the time, all this is probably not an issue, but still there are many cases where colleagues struggle with their role in project teams, and we should make sure we remind ourselves what the intent is :)
|
|
||
| Some other things to keep in mind: | ||
|
|
||
| - Don’t forget to include time for learning new essential skills. |
There was a problem hiding this comment.
Excellent point! And: don't be afraid to be open about this. And if partners don't like it: too bad. It's part of how we work :)
| - Don’t forget to include time for learning new essential skills. | ||
| - Depending on how good you are at estimating time, it may be wise to double, triple, or even quintuple your initial time estimates to come to a more realistic number. | ||
| - The question is not “how long does this take”, but “how long does this take you”. Look at comparable things you have done in the past, learn from it, and apply the lesson to your current time budget. | ||
| - Operations uses 10-15% of hours off your budget as overhead, so you have less time than is listed on the proposal. |
There was a problem hiding this comment.
Management, actually. The operations part is a different 15%, which is not part of the hour budget as you see it in Exact/Ganttic.
|
I really like this text :) I can think of some additional tips and tricks (see also my review comments), and imho it can use a bit more focus on the research aspects as well. I could make some suggestions about that, if desired, or maybe someone else wants to :) But I first would like to hear from @nielsdrost first as well, or some other PM, who I think are working on this too, so that I don't reinvent their wheels :P |
|
|
||
| Perhaps you are working from an existing tool. | ||
| Is there a community? Are there existing users? Are basic maintenance and user support questions likely to distract you from your project tasks? Or are they in fact an important element of your project? | ||
|
|
There was a problem hiding this comment.
Perhaps using bullet points can increase the readability of these questions. What do you think?
There was a problem hiding this comment.
I tried, but it didn't flow well. I think it's because these are not questions that are on the same level, they feed into each other.
| Making sure the module is complete before you move on to the next means that you do not run out of hours with a beautiful software tool that nobody knows how to use. | ||
|
|
||
| Working in completed steps also ensures that your software does as least some things 100%, instead of meeting _all_ requirements for 60%. | ||
|
|
There was a problem hiding this comment.
I really like the "Work in iterations" advice. It has an additional benefit: having a complete module at the end of one iteration allows us to safely forget about many of the fine details. This will become particularly handy at the beginning of the next iteration, when we'll get to speed quickly.
There was a problem hiding this comment.
Agreed!
I ended up not adding this point to the document; it is more personal and process-related, but the main advice stands :)
|
Before spending more time on this pull request, it would be good to clarify with everyone involved if we want guidance on running projects in the guide: I received an email discussion from @tbkkr @nielsdrost @jiskattema @RvanNieuwpoort and @jmaassen where it was requested that this chapter be removed. |
|
Sorry everyone for having this linger around so long! There've been some offline discussions on if/where we need to share this kind of information. For sharing experiences etc. within the escience center , we now also have our intranet. I can put this there? |
|
Well, if the NLeSC Guide isn't for telling our engineers how to do their job, what is it for? All the generic stuff has already been contributed to the Turing Way (excellent idea), so if there's no NLeSC-specific stuff allowed in the Guide then what's left? Also, if the Guide doesn't tell our engineers how we run projects, then what does? As it is, every new employee needs to reinvent the wheel themselves. Now we finally have someone making an attempt at fixing this, and now the management tells us we're not allowed? The intranet runs on Sharepoint, which is unsuitable for editing a large document like the Guide collaboratively, at least last time I tried to use it. Maybe we should just make the Guide a private repository, for internal use, that documents our process (we'll invent one) and experience for new people to learn from? |
Most things have indeed been copied to The Turing Way, but the language guides do not fit in The Turing Way, as that is language-agnostic. However, they do seem useful to a larger audience. For example: I regularly point researchers who want to get started with Python to our Python chapter. Maybe we could keep just those, but share them under a different name than the guide? Perhaps together with the training materials. |
|
Thanks all for the feedback and the discussion. In the question of where to put this, I can explain why I chose the guide specifically: this is (at least to my knowledge) an NLeSC-specific resource, but more importantly: as @LourensVeen mentioned, it is collaboratively edited. For me as an RSE it is not possible to simply add something to the intranet. The guide, by contrast, not only provides that option, but it remains a living document: anyone with additional tips and ideas (or who disagrees with mine!) can chime in, and add this knowledge. This already happened in @egpbos' and @PabRod's reviews above; I am very interested to hear from more colleagues. If there is a way to achieve this in an intranet document, I would be very happy to know about it. I think it's essential to have this option in a resource that depends on experience and knowledge building, rather than top-down instructions. |
|
Disclaimer: the following is not a management directive, just brainstorming ;) I personally think it's very valuable to have different perspectives on what "doing projects" means available for new (and existing!) colleagues and externals to learn from. Personal perspectives are probably easier to write (just do a braindump) than something generic where you have to take everyone's ideas (and feelings ;) ) into account. At the same time it would be richer in details that perhaps get shaved off when you try to make a generic guide. The reason I think different perspectives are valuable is that different projects have different needs and best practices, depending on the people involved, the goals, logistical constraints (not enough people for goal A, for instance, so you go for goal B or A.2 or something), etcetera, etcetera. All projects and all engineers have something unique to teach us. However, I'm wondering whether the Guide is the best format for this. As mentioned, compiling all the disparate experiences into something generically "true" would be a huge effort and the question is whether it will still be digestible separated from the many story-contexts that generated them. Wouldn't it be better to gather stories like this on the Blog? We can link to those from the Guide. I think we already have a couple of blogposts that would fall into this "lessons learned" category, so we can already make a start. What do you think @bvreede (et al.)? |
|
My two cents:
|
I don't think so either, so far it nicely takes into account different perspectives. I'm just thinking about how it may evolve and especially about maintainability. But perhaps this concern is solved easily in the same way as we did before with language chapters: @bvreede do you want to take responsibility as chapter owner to make sure this chapter stays up to date, complete and issues, PRs etc are handled quickly? Perhaps also do a little advertising inside the Center to get broad input, since this is such a broad topic. If so, and the content discussions are handled, I would personally be fine with merging this, how about you @bouweandela? Do note that in the (near) future we will probably decide on a Guide format 2.0, and we may start killing some darlings. Of course, this discussion and the many good points raised will be taken into account. Just so you have an idea about the discussion: one of the points is that the Guide gets too outdated. We seem to lack the resources and/or cannot make it a priority to maintain the Guide to the point that all its content is actually useful. Newer engineers have on occasion reported that they read parts and that while they appreciated the concept it really was not up to best standards at [current time]. So the question is how to handle it such that what we put on there stays fresh. If anyone has good ideas about this, please let me know! Maybe in a separate thread or just via mail, DM or in person. |
|
I think that there needs to be a private place for at least some of it. Yes, we can write (and should, and should have) a nice manual on how we do collaborative science, and that will be useful for the wider world, or at least for prospective applicants to get an idea of what they're in for. Maybe they'll even read it, and maybe that will reduce some friction once we start a project. And maybe some other eScience organisation somewhere can use it for inspiration when they're making their own. However, practice isn't theory, and there's practical knowledge that is better to keep internal. Things like how we dealt with this case where we really disagreed with the scientists about how to do something, or how we resolved an issue with academic quality. I would want to be able to write about these things somewhat candidly. No need to name names, but the situation needs to be described in enough detail that it's useful for future colleagues in a similar situation. A collection of anecdotes sounds like a really nice format, I agree that trying to integrate it will just make it abstract and vague and not-so-useful. |
|
@egpbos I am happy to keep this particular chapter up to date. (I'm also all for the killing of darlings, and there needs to be an infusion of energy into this here guide imho, so Guide 2.0 has my full support; I have some vague thoughts but will get back to you.) I have proposed a lightning talk (scheduled on Oct 12th), which could also serve as advertisement for this chapter. It would be nice to collect more thoughts on this topic. Perhaps the proof of the pudding is in the eating; let's see how it evolves in the near future, and we can always extract it from the guide in the form of a more solidified document on Intranet, or a (collection of) blog post(s). I'll take this opportunity to process a few review comments as well. More are most certainly welcome, and I'll await a final decision about merging this! |
|
I agree with @jiskattema and others that this would perhaps be better as an internal resource as part of the Intranet. As PMs we need to add more information on project management there anyway. Perhaps @jiskattema can make a start with the Intranet pages and together with @bvreede have a look if the contents of this chapter would fit? |
|
Although it pains me, with all the energy that everybody and especially @bvreede put in, I am going to close this PR. Feel free to blame and hate me for not having clearly set the stage for making new contributions. I plan on fixing that soon. The main message will be: please first discuss (make an issue) before writing something new and big like this. In case it happens anyway, like in this case, I would strongly encourage people to not let the work go to waste, but rather to publish it elsewhere; I would say our blog in this case is a pretty good place, since the advice in this chapter is highly reusable for any RSE. Also, that gets you personally more exposure and maybe interesting feedback from the broader RSE community out there than when this would become "only" an internal document. Up to you, of course! |
|
Oh, sorry, I forgot to add why :) For me, the first main reason is actually the "integrism" argument as @PabRod calls it. I think project experiences are hugely valuable, but since they are so context-dependent, I think they are better presented as such: individual experiences. The danger I see in presenting this (or any other text) as "the way" of doing projects is that people may think they are doing something wrong if they are in a different type of project; and indeed there are quite different types of projects! I believe we should as an organization make sure that people reflect on how their projects are going face to face, rather than via abstract text. Projects will sometimes be bumpy, that's inevitable. But so long as we keep talking about our work, we have a chance at recognizing when things get too bumpy and solving it. The second main reason is that we have the project management protocol. Making that document more day-to-day useful for RSEs, by adding (or linking to) tips like in this PR, sounds like a great idea to me. And, in fact, I think the PMs have been working on that (preventing duplication of efforts is another reason for discussing in an issue before writing); at least, I have discussed something like this at some point with either @nielsdrost or @bakhshir. |
|
Thanks @egpbos! Just to clarify (although I know that no defense is necessary), this was never written as an intended chapter for the guide, rather; these are my own notes from picking the brains of several brilliant colleagues, as I myself tried to get advice on good planning practices. No work on my end has gone to waste, in other words. I appreciate all the discussion and work that went into dealing with the PR though! I hope it was useful in some way, too. |
The Planning chapter is currently empty. I've been collecting practical advice (thanks @fnattino @vanlankveldthijs @jiskattema @LourensVeen !) on planning an eScience Center project, and thought this may make a good first edition of this chapter in the guide.
Closes #1 🙃