Feedback on 3 Elicitation

[daphne-barretto]

3.1

• "AL has successfully enhance various real-world systems." should be "enhanced"
• "Here, collecting data often incurs significant financial and time costs because physical robot arm worns out over time." should be "arms wear out"
• "Typically, in robotic, robots learn by observing human demonstrations. " should be "robotics"
• "There are several method for quantify model uncertainty." should be "There are several methods for quantifying model uncertainty."
• "Exact posterior computation can become computationally prohibitive, especially for complex likelihood function, and approximated Bayesian computation is proposed to address this." should be "Exact posterior computation can become computationally prohibitive, especially for complex likelihood functions, and approximated Bayesian computation is proposed to address this."
• This text is unclear: "Exact posterior computation can become computationally prohibitive, especially for complex likelihood function, and approximated Bayesian computation is proposed to address this. For example, ensemble methods involve training multiple models and combining their predictions to provide an estimate of uncertainty. Ensemble methods are relatively easy to implement, but they are noisy and still somewhat expensive. Conformal prediction methods also provide a framework for estimating uncertainty by offering a measure of confidence in predictions based on the conformity of a given instance with the training data." The text For example seems to imply that ensemble methods are an example of approximated Bayesian computation, but the reader should not be assumed to be able to decipher if this is true or not. I recommend clarifying the relationship here.

3.2

• Title "3.2 Estimating the Value of Additional Data with Acquisition Function" could be "3.2 Estimating the Value of Additional Data with Acquisition Functions"
• Common acquisition functions being in a paragraph form and other acquisition functions being in bullet point form draw more attention to other acquisition functions over the common ones. However, I am interpreting the common ones to be more important. I would recommend making the common ones in bullet points instead and just putting others in paragraph form, or making it all consistent between the two groups.
• Related, it's interesting that the 3 common acquisition functions go into detail with their own paragraphs and code, and so does Variance reduction but not the other two "other" acquisition functions error reduction and model change. I found myself looking for more information on those two, since the other 4 functions mentioned had more details in the chapter.
• The select x_1 notices are printed away from the grouping of calculations they actually represent. Same thing for query-by-committee code.

- "The code below demonstrate that uncertainty sampling methods yield the same conclusion of selecting x1." When is this true? What considerations are there? Same questions for query by committee. - "Query-by-Committee ([Beluch et al. 2018](https://mlhp.stanford.edu/src/chap4.html#ref-AL_committee)) is selects samples for labeling based on the level of disagreement among members of a committee." Remove "is" to just be "Query-by-Committee ([Beluch et al. 2018](https://mlhp.stanford.edu/src/chap4.html#ref-AL_committee)) selects samples for labeling based on the level of disagreement among members of a committee." - What is Shannon information? This is not explained in the text. It should be mentioned which part of the equation represents this. - Recommend making a new paragraph starting with "To evaluate the performance of variance reduction strategy, Cohn, Ghahramani, and Jordan ([1996](https://mlhp.stanford.edu/src/chap4.html#ref-AL_variance)) studies the Arm2D problem."

3.3

• "An alternative source of domain knowledge could be users themselves" It would be good to clarify what the original / non-alternative source of domain knowledge was, as it appears to me that the previous section was still eliciting preferences from users.

3.4

• There isn't a Case Study 1 in the chapter, just 2 and 3.
• The start of this section reads as if it's the start of an entire chapter. It would be good to have a delineation from Active Learning to Metric Elicitation, and clearly explain how the two concepts relate to each other for understanding the content of this chapter.
• It is not very clearly stated why performance metric elicitation is a case study of active learning
• "Finally, the setup for metric elicitation is identical to the one examined in the previous chapter. " Does chapter mean the X in 3.X? I was interpreting this entire 3 Elicitation section as one chapter, and the Introduction on the Book Structure aligns with this.

3.5

• "For example we can see how a query between two items can split the plain into two halves" should use "plane"
• "In the following subsections, we describe the process of estimating compatibility and active elicitation in more detal." Mistype of "detail".
• The last paragraph of the summary mentions topics not actually in this chapter: "Additionally, the chapter examines the integration of foundation models into robotics, highlighting the transformative innovations of R3M and Voltron. R3M’s pre-training on diverse human activities dramatically improves robotic manipulation with minimal supervision. Meanwhile, Voltron builds on these capabilities by incorporating language-driven representation learning for remarkably adaptable and nuanced robotic task performance. These models represent significant leaps in robotics while opening new frontiers for future research and applications."

[daphne-barretto]

For 3.5, I highly recommend incorporating information on work related to Shuran Song's diffusion policy and related work: https://diffusion-policy.cs.columbia.edu/. It discusses how to learn from incompatible demonstrations beyond just trying to only elicit compatible ones, as the textbook currently mentions.