Suggestions for research and research-related activities
Here, I list ressources that I hope to be helpful for research and research-related activities.
Some of it might be obvious, but I prefer to err on the side of mentioning more rather than less.
Navigating Research Results
DBLP is a database of all CS publications (and some maths publications).
E.g., this is
Sabine Oechsner's DBLP entry
and this is .
Geoffroy Couteau's DBLP entry.
In addition to calculating infamous citation indices
(For a discussion of content-oblivious quality measures, see
Oded Goldreich's comments), Google Scholar actually also has useful functionalities. E.g., when you search for the PDF of a paper, you can type the title into
Google scholar, and Google scholar will search through the relevant location, including, e.g., authors webpages. Moreover, Google Scholar is tremendously useful
when you want to explore a scientific area, that you know little to nothing about. Proceed as follows: (1) Find an arbitrary paper in the area, read the introduction and
find one of the earliest papers in the area. (2) Type that title into google scholar. (3) Click on "cited by..." and you obtain essentially a list of all papers in that area.
You can then search for specific paper in that areas by only searching within the papers that cite that original paper.
Talks and more information:
You can often find talks about papers via the Google Video search (i.e.,
type the title of the paper). Moreover, many authors have additional
information (such as slides etc.) on their personal webpages.
Essentially all crypto papers are published here, around 1000 per year. People who use twitter or newsfeeds often integrate IACR ePrint into them and then get weekly updates (it's about 3 paper titles per day
which seems okay for people who process 50 twitter messages per day (I personally don't, but many other people find this useful...)): Link.
Introduction to Mathematical Reasoning
is a fantastic friendly introduction to mathematical reasoning by Michael Hutchings.
Program Committee Algorithm
Decision-making in program committees is an important activity of our scientific community. Here is how I prioritize my time as a PC member: Choosing the right preferences is tremendously important and saves a lot of annoyance later, so it is worth to spend time on preferences. Secondly, the discussion phase usually requires additional last-minute reviews, studying additional material, reflecting results etc.. This work cannot be delegated. Thus, I reserve ample time for the discussion phase.
The only phase where work can be delegated is the review phase. Being aware of the tremendous workload during the discussion phase, I rigorously try to find subreviewers for all papers in my stack. Some reviewers fail to submit a review or are late or too superficial etc., so some reviewing work is always left. Moreover, I often have a discussion over Skype with the subreviewers to better understand their view. This way, I am in a good position to discuss the article.
How to choose preferences and subreviewers? As subreviewers, I try to find reviewers who would like to read an article anyway, i.e., this way, they can do a service to the community with very little additional cost to them or to the community. I take this into consideration when bidding on papers. For the papers that I mark as my preferred papers, I usually already have a candidate reviewer in mind. Given the work put into choosing reviewers at the preference phase, if n is the expected numbers of assignments, I usually mark only n papers as strongly preferred. I then choose some additional 2n papers that get a better score than "I don't want to review this.". Also for these papers, I ensure that I will not suffer if I have to review them.
Once the assignments are made, I immediately send out review requests to the subreviewers and try to give them at least 5 weeks for the review. The earlier the request is sent out, the easier it is for the reviewer to plan and the more likely the reviewer is going to accept the review. Sending out review requests can be emotionally uncomfortable because one might feel that one asks others for a favor. This feeling changes, significantly, when asking someone for a review on a paper that they actually want to read and actually care about. It makes a difference whether someone accepts a review from a "I should and I can" or "I want" state of mind. Thinking about reviewers who actually want makes the committee work a lot of fun.
In the end of the review phase, once the reviews are submitted, there are usually some clear-cut cases of papers that will be included and papers that will not be included. For the papers in the grey zone, I now invest additional effort, discuss with "my" reviewer and the other PC members, read parts of the paper myself etc.. As I don't need to take care of the clear-cut cases and as I already have input from the subreviews that indicate what to focus on, this reviewing work is significantly less. Eventually, this process leads to a relatively informed decision of whether or not to include the paper. Of course, committee work involves a lot of speculation and binary decisions, but we do try to make a program that is most useful to the scientific community.
The discussion itself: It is often uncomfortable to express opinions that are hard to articulate or where the reasons leading to the opinion are hard to articulate. Still, I try my best to express an opinion whenever I have one (or express that I have no opinion if I have none), because on the average, having an opinion means I have some sort of information, even if that information is hard to exactly pin down. I treat my opinion as something different than the result of the social process. I.e., I might express an opinion and say that I withdraw it if a specific other person (who I deem more qualified in this area) thinks differently etc.. I hope that this way, I can contribute my opinion as an input to the social process while my opinion is properly weighed by its confidence and does not dominate the social process.
A few guidelines that I use are the following: (1) Being late on preferences delays the entire process/carries the risk of getting bad assignments. So, being late on submitting preferences should not happen. I mark it in my calendar as soon as the date is announced. (2) Quality beats punctuality. Having a high-quality late review is much more useful for the reviewing process than a punctual low-quality review. So, if a reviewer requests more time than given by the PC chairs for the review time, I usually agree to it if it is a good reviewer. A good reviewer is someone who is an expert in the field and cares about the paper enough to be likely to write something informed about the submitted article. In case there are hard deadlines (especially if there are rebuttals), it's good to check with the chair what to do if a reviewer requests more time.
Being on a PC and taking it seriously is a lot of work and comes on top of everthting else one does. Thus, underestimating the work or not prioritizing the search for reviewers in time can lead to work overload experiences, major delays (rather than minoe delays) and lack of time during the discussion phase. In the discussion phase, a PC member cannot be replaced, so being short on time in that phase is a substantial problem for the process (whereas, e.g., delegating all reviews or being a few days(!) late on a few(!) reviews usually is usually compatible with the average-case functioning of the process.).
Return to Chris Brzuska's Homepage
Last update: 2019-04-03