Chi2014 reviews summary

AC

• The current submission provides a mix of [three contributions] but does not develop any of them in enough depth.
• The [interactions section is] not developed enough:
• What are the dimensions of this taxonomy?
• What is the rationale for this classification?
• For the taxonomy to be useful, a table or figure needs to be provided where all previous work is laid out.
• Ideally, the taxonomy should be presented as a design space with meaningful and independent dimensions.
• Eliciting ideas from end users is not enough: a taxonomy or design space should be the product of extensive analytical reasoning by the authors themselves.
• Do the contributions lie in the novelty of the ideas proposed?
• If yes, then they need to be clearly contrasted with previous techniques.
• The interesting (but challenging) problem of how to actually support surface capture is not addressed (and the possible technical solutions discussed p.9 are not overly realistic).
• Does the contribution lie in the evaluations?
• The solution does not seem to scale well to more than 2 applications
• The image capture experiment yields interesting results, but does little to explain why surface capture is faster than camera capture.
• The report on the participatory design session is not especially useful.
• It is hard to understand why surface capture outperformed camera capture, and as a result, the study "falls short of providing true insight that could inform the design of other techniques" (R1)
• According to R4, the "authors should have questioned their results" and provided insights that could inform designers of such applications.

Reviewer 1

• This is a potentially interesting topic and I think the paper has some ideas to contribute, but it also is not totally clear to me after reading exactly what is and is not new here, as compared to previous related papers.
• That there weren't clearly or markedly new ideas that arose from [user centered design] didn't help too much -- many of the basic ideas proposed felt familiar from previous related works.
• [What is] the rationale around why the authors expect transparent displays to have significant advantages over other approaches.
• On this latter approach a key missing reference in my view is the PACER work of Liao et al, which uses the camera on a (non-transparent) mobile to do some of the same kinds of things.
• The Overlay interactions made sense, but at the same time it was hard to see how these were necessarily better than sketching on top of content on a regular stylus-enabled tablet computer.
• I found the dual display input interactions the most interesting, but still there are related ideas there from LucidTouch and back-of-device interactions that have been explored in several other papers already.
• [DualTouch] would have been a good reference on the basic idea of and recognition of flipping the device over as a mode switch.
• I also liked the tap & flip idea, which seems fairly new (though again, probably the Tilt+Touch-in-Motion paper should be cited as a related technique).
• The Chen paper on dual-display techniques is another key one on the idea of flipping over as a motion-gesture and information navigation technique.
• It would also be nice to see some of the Anoto-paper-based active reading type of interactions referenced here and discussed a little more either in the related work or in contrast to the Overlay, Extraction, Annotation, and/or Area Trigger techniques.
• This seems to be borne out by the small user study later in the paper, but again there even though an absolute difference in means was found, it was never explained where that performance difference came from -- why one technique outperformed the other.
• I didn’t think the first paragraph of the Software section added anything of note (and the figure could go with that) and, while some brief mention of the capture-based registration might be in order, it seems a standard enough approach that this whole section (and the evaluation of it) could largely be omitted, because I don’t see anything particularly new or interesting here. This would buy at least half a page to spend more discussion and motivation around the user studies, which I think would be helpful
• “Home” and “MultiTask Bar” conditions are kind of standard on mobiles, these actually differ across different devices so you should spell out exactly how they look and act in your study.
• There is no rationale or discussion at all as to why the observed performance differences occurred.
• It was not clear to me why the Tap’N Flip technique should be faster for app switching than the other techniques, for example, and likewise for the capture task I couldn’t understand why the transparent device should have been twice as fast. Clearly some steps are being simplified or a better match to the user’s mental model of the task is being achieved, but in the discussion of results this is not broken down or spelled out as to why the authors think these differences occurred.
• I think the discussion could have left out the color mixing issue. You already raised this issue in related work, and as far as I can tell you don’t actually have any new technique implemented here yet.
• I would suggest a minor reorganization of your Discussion to first raise and amplify the key questions and answers raised by your technical explorations and study, and they saving the future-looking subsections till the end.
• There may be enough here for the paper to squeak in, particularly if the authors can improve the related work and also spell out more clearly WHY the techniques seemed to offer performance improvements over the other techniques in the study in a convincing manner. I suggest the rebuttal should focus on that.

Reviewer 2

• The paper does deliver value by showing that these can be done on transparent mobile devices.
• The potential value in this type of paper is that it can quickly bring the community up to speed and inspire broader use of the involved technology.
• Ultimately the usefulness and impact of this type of paper depends on how novel and inspired the techniques are.
• AR-style annotations: The interaction could also be done--and have been done--by devices with a scanner, i.e., scan first, then annotate. Might it be more interesting to explore interactions with animated contents, which would then merit the continued presence of the object? Rotoscoping a video, maybe, by placing the device over a video source?
• Dual-sided. The authors use flipping merely as a mode switch. This has been proposed by many papers in the past. I would have preferred to also see true dual-sided use, i.e., use of both sides at the same time, as in Ishii's clearboard.
• Also, it was not quite clear to me where the added value of snapping under the screen was compared to other devices featuring a scanner.
• In summary, I feel that none of the techniques are "novel novel", but they have certainly never been shown on this type of device.
• I would enjoy seeing a quick statement in the abstract (and the rebuttal) on the 1-2 techniques the authors think of as most innovative and what sets them apart from the related work.

Reviewer 4

• The paper overall did not point out why transparent tables are advantageous to a standard iPad.
• The study is very detailed, but why does it only evaluate a subset of the proposed techniques?
• The implementation section takes up a lot of space, but discusses things that are well-understood and explained in the related work already.
• The discussion seems somewhat devoid of an actual and actionable take-home message. If I were an interaction designer for transparent displays, how would this affect my work?