From 95ac18b6e058580f5f4266f802da703ca3d9570d Mon Sep 17 00:00:00 2001 From: Elijah Melton Date: Mon, 6 Jan 2025 18:00:03 -0800 Subject: [PATCH] sync changes: - M site/index.html - A site/systems-research/hints-for-computer-system-design.html - A site/systems-research/how-to-read-a-paper.html --- site/index.html | 42 ++-- .../hints-for-computer-system-design.html | 205 ++++++++++++++++ .../systems-research/how-to-read-a-paper.html | 219 ++++++++++++++++++ .../paper-review-template.html | 205 ++++++++++++++++ site/systems-research/strong-inference.html | 218 +++++++++++++++++ .../hints-for-computer-system-design.md | 36 +++ systems-research/how-to-read-a-paper.md | 44 ++++ systems-research/paper-review-template.md | 36 +++ systems-research/strong-inference.md | 37 +++ 9 files changed, 1021 insertions(+), 21 deletions(-) create mode 100644 site/systems-research/hints-for-computer-system-design.html create mode 100644 site/systems-research/how-to-read-a-paper.html create mode 100644 site/systems-research/paper-review-template.html create mode 100644 site/systems-research/strong-inference.html create mode 100644 systems-research/hints-for-computer-system-design.md create mode 100644 systems-research/how-to-read-a-paper.md create mode 100644 systems-research/paper-review-template.md create mode 100644 systems-research/strong-inference.md diff --git a/site/index.html b/site/index.html index 17a3e53..8832d36 100644 --- a/site/index.html +++ b/site/index.html @@ -185,7 +185,7 @@

My Notes

- 147 + 151 Notes
@@ -201,6 +201,26 @@

My Notes

Recent Notes

diff --git a/site/systems-research/hints-for-computer-system-design.html b/site/systems-research/hints-for-computer-system-design.html new file mode 100644 index 0000000..f10069b --- /dev/null +++ b/site/systems-research/hints-for-computer-system-design.html @@ -0,0 +1,205 @@ + + + + + + Hints For Computer System Design + + + + + +
+
+ Home » Hints For Computer System Design +
+

Hints For Computer System Design

+
+ Last modified: 2025-01-06 + +
+
+

source

+
Paper Title
+
+

What is the Problem?

+

Summary

+

Key Insights

+

-

+

Notable Design Details/Strengths

+

-

+

Limitations/Weaknesses

+

-

+

Summary of Key Results

+

-

+

Open Questions

+

-

+
+ +
+ + \ No newline at end of file diff --git a/site/systems-research/how-to-read-a-paper.html b/site/systems-research/how-to-read-a-paper.html new file mode 100644 index 0000000..a710b3e --- /dev/null +++ b/site/systems-research/how-to-read-a-paper.html @@ -0,0 +1,219 @@ + + + + + + How To Read A Paper + + + + + +
+
+ Home » How To Read A Paper +
+

How To Read A Paper

+
+ Last modified: 2025-01-06 + +
+
+

source

+
How to Read a Paper
+
+

What is the Problem?

+

Researchers spend 100s of hours reading papers every year, so they ought to know how to do it effectively. Keshav argues that reading a paper is a skill that can be learned, and that it is a skill that is not taught in school.

+

Summary

+

Keshav recommends a three-pass approach to reading a paper:

+

First Pass

+

Gives you a general idea of what the paper is about. Read the title, abstract, introduction, and conclusion. Only read section and sub-section headers. Also glance over the references and note which you've read. This should only take 5-10 minutes.

+

After the first pass, you should be able to answer the following questions:

+
    +
  • Category: What type of paper is this? (e.g. measurement, analysis)
    • +
  • Context: Which other papers is it related to? What is the theoretical background?
    • +
  • Correctness: Do the assumptions appear to be valid?
    • +
  • Contributions: What are the paper's main contributions?
    • +
  • Clarity: Is the paper well written?
    • +
+

Second Pass

+

Grasp the content of the paper, but not necessarily the details. Read the whole thing, but ignore things like proofs. It helps to also annotate/take notes during this pass.

+

Pay special attention to diagrams, and also mark relevant unread references. This should take about an hour.

+

After this pass, you should be able to summarize the paper in a few sentences to a peer. If you still don't understand, you can do one of three things:

+
    +
  • Set the paper aside and hope
    • +
  • Come back to the paper later after looking up references you didn't understand
    • +
  • Continue to the third pass anyways 😢
    • +
+

Third Pass

+

Understand the paper in depth. You should virtually re-implement the paper, following all reasoning and challenging every assumption. Also try to think about how you would have presented the information differently. You should note potential follow-up work during this pass. This should take up to 5 hours, and at least 1 hour for a well-written paper and well-read reader.

+

After this pass, you should be able to reconstruct the paper's structure from memory. You should also be able to critique the paper and pinpoint implicit assumptions, limitations, and potential improvements.

+
+ +
+ + \ No newline at end of file diff --git a/site/systems-research/paper-review-template.html b/site/systems-research/paper-review-template.html new file mode 100644 index 0000000..c300e70 --- /dev/null +++ b/site/systems-research/paper-review-template.html @@ -0,0 +1,205 @@ + + + + + + Paper Review Template + + + + + +
+
+ Home » Paper Review Template +
+

Paper Review Template

+
+ Last modified: 2025-01-06 + +
+
+

source

+
Paper Title
+
+

What is the Problem?

+

Summary

+

Key Insights

+

-

+

Notable Design Details/Strengths

+

-

+

Limitations/Weaknesses

+

-

+

Summary of Key Results

+

-

+

Open Questions

+

-

+
+ +
+ + \ No newline at end of file diff --git a/site/systems-research/strong-inference.html b/site/systems-research/strong-inference.html new file mode 100644 index 0000000..0da75e5 --- /dev/null +++ b/site/systems-research/strong-inference.html @@ -0,0 +1,218 @@ + + + + + + Strong Inference + + + + + +
+
+ Home » Strong Inference +
+

Strong Inference

+
+ Last modified: 2025-01-06 + +
+
+

source

+

Strong Inference

+
Certain systematic methods of scientific thinking may produce much more rapid progress than others.
+
+

What is the Problem?

+

The process of doing research has become less standardized in some fields, especially when compared to the more structured approach of fields like molecular bio and high-energy physics.

+

When a formal scientific method isn't adhered to, especially when formulating hypotheses, the process can become less efficient and less effective, leading to fewer discoveries and slower progress over time.

+

Summary

+

John R. Platt critiques the application of the scientific method in modern research, arguing that the process has become less structured and less effective in some fields. He proposes a more structured approach to hypothesis formulation, which he calls "strong inference".

+

The paper goes on quite a bit about historical examples of the principles of strong inference working in practice, as well as a detailed breakdown of how to systematically apply the method to research.

+
    +
  1. Devise alternative hypotheses - Generate multiple competing hypotheses that could explain the phenomenon being studied.
    2. +
  2. Devise a crucial experiment - Design an experiment that can unambiguously distinguish between the competing hypotheses, or at the very least, eliminate some or all of them.
    3. +
  3. Carry out the experiment - Conduct the experiment and analyze the results.
    4. +
+

Strong inference is essentially to carry out this process at every vertex of the logical tree of inquiry, and to do so in a systematic and structured way. He suggests keeping a notebook explicitly for this, and to pay particular attention to the process of hypothesis generation.

+

Key Insights

+
    +
  • Have multiple competing hypotheses, and come up with the most efficient way to eliminate them via experimentation.
    • +
  • Systematically, explicitly, and regularly follow this process, from hypothesis generation to experimentation to analysis.
    • +
  • Ask yourself the two questions: "How would we know this hypothesis is wrong?" and "What hypothesis does this experiment disprove?"
    • +
+

Notable Design Details/Strengths

+
    +
  • Deviations from strong inference only really manifest themselves as useless delays in the research process. Many scientists do a lot of busywork for no reason, which could be avoided had they spent more time formulating hypotheses.
    • +
  • Strong inference is strongly dependent on the actual induction being done when formulating hypotheses. This needs to be logically sound.
    • +
  • It is a system that works if done correctly, as it's essentially the minimum amount of work needed to make a discovery without just getting lucky.
    • +
+
+ +
+ + \ No newline at end of file diff --git a/systems-research/hints-for-computer-system-design.md b/systems-research/hints-for-computer-system-design.md new file mode 100644 index 0000000..80d67bb --- /dev/null +++ b/systems-research/hints-for-computer-system-design.md @@ -0,0 +1,36 @@ +# [source](https://example.com) + +###### Paper Title + +--- + +### What is the Problem? + + +### Summary + + +### Key Insights + +- +- + +### Notable Design Details/Strengths + +- +- + +### Limitations/Weaknesses + +- +- + +### Summary of Key Results + +- +- + +### Open Questions + +- +- diff --git a/systems-research/how-to-read-a-paper.md b/systems-research/how-to-read-a-paper.md new file mode 100644 index 0000000..773d244 --- /dev/null +++ b/systems-research/how-to-read-a-paper.md @@ -0,0 +1,44 @@ +# [source](https://web.stanford.edu/class/cs114/reading-keshav.pdf) + +###### How to Read a Paper + +--- + +### What is the Problem? + +Researchers spend 100s of hours reading papers every year, so they ought to know how to do it effectively. Keshav argues that reading a paper is a skill that can be learned, and that it is a skill that is not taught in school. + +### Summary + +Keshav recommends a three-pass approach to reading a paper: + +#### First Pass + +Gives you a general idea of what the paper is about. Read the title, abstract, introduction, and conclusion. Only read section and sub-section headers. Also glance over the references and note which you've read. This should only take 5-10 minutes. + +After the first pass, you should be able to answer the following questions: + +- *Category*: What type of paper is this? (e.g. measurement, analysis) +- *Context*: Which other papers is it related to? What is the theoretical background? +- *Correctness*: Do the assumptions appear to be valid? +- *Contributions*: What are the paper's main contributions? +- *Clarity*: Is the paper well written? + +#### Second Pass + +Grasp the content of the paper, but not necessarily the details. Read the whole thing, but ignore things like proofs. It helps to also annotate/take notes during this pass. + +Pay special attention to diagrams, and also mark relevant unread references. This should take about an hour. + +After this pass, you should be able to summarize the paper in a few sentences to a peer. If you still don't understand, you can do one of three things: + +- Set the paper aside and hope +- Come back to the paper later after looking up references you didn't understand +- Continue to the third pass anyways 😢 + +#### Third Pass + +Understand the paper in depth. You should *virtually re-implement* the paper, following all reasoning and challenging every assumption. Also try to think about how you would have presented the information differently. You should note potential follow-up work during this pass. This should take up to 5 hours, and at least 1 hour for a well-written paper and well-read reader. + +After this pass, you should be able to reconstruct the paper's structure from memory. You should also be able to critique the paper and pinpoint implicit assumptions, limitations, and potential improvements. + diff --git a/systems-research/paper-review-template.md b/systems-research/paper-review-template.md new file mode 100644 index 0000000..80d67bb --- /dev/null +++ b/systems-research/paper-review-template.md @@ -0,0 +1,36 @@ +# [source](https://example.com) + +###### Paper Title + +--- + +### What is the Problem? + + +### Summary + + +### Key Insights + +- +- + +### Notable Design Details/Strengths + +- +- + +### Limitations/Weaknesses + +- +- + +### Summary of Key Results + +- +- + +### Open Questions + +- +- diff --git a/systems-research/strong-inference.md b/systems-research/strong-inference.md new file mode 100644 index 0000000..b6566cf --- /dev/null +++ b/systems-research/strong-inference.md @@ -0,0 +1,37 @@ +# [source](https://pages.cs.wisc.edu/~markhill/science64_strong_inference.pdf) + +#### Strong Inference +##### Certain systematic methods of scientific thinking may produce much more rapid progress than others. + +--- + +### What is the Problem? + +The process of doing research has become less standardized in some fields, especially when compared to the more structured approach of fields like molecular bio and high-energy physics. + +When a formal scientific method isn't adhered to, especially when formulating hypotheses, the process can become less efficient and less effective, leading to fewer discoveries and slower progress over time. + +### Summary + +John R. Platt critiques the application of the scientific method in modern research, arguing that the process has become less structured and less effective in some fields. He proposes a more structured approach to hypothesis formulation, which he calls "strong inference". + +The paper goes on quite a bit about historical examples of the principles of strong inference working in practice, as well as a detailed breakdown of how to systematically apply the method to research. + +1. **Devise alternative hypotheses** - Generate multiple competing hypotheses that could explain the phenomenon being studied. +2. **Devise a crucial experiment** - Design an experiment that can unambiguously distinguish between the competing hypotheses, or at the very least, eliminate some or all of them. +3. **Carry out the experiment** - Conduct the experiment and analyze the results. + +Strong inference is essentially to carry out this process at every vertex of the logical tree of inquiry, and to do so in a systematic and structured way. He suggests keeping a notebook explicitly for this, and to pay particular attention to the process of hypothesis generation. + +### Key Insights + +- Have multiple **competing hypotheses**, and come up with the most efficient way to eliminate them via experimentation. +- **Systematically**, explicitly, and *regularly* follow this process, from hypothesis generation to experimentation to analysis. +- Ask yourself the two questions: "How would we know this hypothesis is wrong?" and "What hypothesis does this experiment disprove?" + +### Notable Design Details/Strengths + +- Deviations from strong inference only really manifest themselves as useless delays in the research process. Many scientists do a lot of busywork for no reason, which could be avoided had they spent more time formulating hypotheses. +- Strong inference is strongly dependent on the actual induction being done when formulating hypotheses. This **needs** to be logically sound. +- It is a system that **works** if done correctly, as it's essentially the minimum amount of work needed to make a discovery without just getting lucky. +