q.Rmd

# Q Methodology {#q}

<!-- TODO question here is: how do we measure deliberation? -->

<!-- remember, this is going to be exploratory -->
<!-- only part of it tightly hypothetico-deductive -->

<!-- TODO so you're putting the same arguments *into* the items which where the treatments to test whether the treatment wored; isn't that circular?!? Maybe refer back to the yoga study to argue that:
- no, it's not (because pragmatism?, because patterns?)
- well, so what, it's remedial, public Sociology kinda thing -->


<!-- %„Die Öffentlichkeit lässt sich am ehesten als ein Netzwerk für Kommunikation von Inhalten und Stellungnahmen, also von Meinungen beschreiben“ (Habermas 1992:
%436).
%„vermachtete Öffentlichkeit“, in der sich etwa finanzstarke Lobbygruppen wiederfinden würden).
this is very similar to concourse -->

		<!-- %Mutz (2008) is all about how it is important to make deliberative democracy falsifiable. -->

<!-- Thomson
    %Also:
    %``Some researchers have assumed that they can dispose of deliberative theory by showing that political discussion often does not produce the benefits that theorists are presumed to claim for it.
    %They extract from isolated passages in various theoretical writings a simplified statement about one or more benefits of deliberative democracy, compress it into a testable hypothesis, find or (more often) artificially create a site in which people talk about politics, and conclude that deliberation does not produce the benefits the theory promised and may even \ldots'' (tbc) -->

Q methodology and deliberative democracy share epistemological and normative roots in American pragmatism: that meaning is intersubjective, that communication is human action and that we can -- and should -- reach some mutual understanding on the objective and moral world, even if contingent and preliminary.
This study illustrates and explores how this common legacy makes Q methodology and and deliberative democracy are a good fit for political psychology.
On the one hand, deliberation posits the regulative ideal under which individuals can freely constitute and express their political subjectivity: a mutual and egalitarian give-and-take of reasons.
On the other hand, Q may serve to measure the quality of deliberation, occupying a sweet spot between quantitatively constrained interpretations for researchers and qualitative leeway for citizens' viewpoints: increased consistency or structuration of Q sorts might be falsifiable meta-standards without substantive (and circular) prejudice as to which one viewpoint would be "deliberative".
The study concludes by suggesting further extensions and applications of Q methodology and deliberative democracy.

Table: Factorial Concourse Sampling Scheme

| Values / Axiology | Beliefs / Ontology | Preferences / Taxes |
|:----------------------------------------------:|:-------------------------------------:|:-------------------------:|
|  | *General* |  |
| Efficiency | Individuals vs Groups | Corporate Tax |
| "Actions are good if consequences are good". | "People are motivated by incentives". | "Unfettered markets". |
|  | *Specific* |  |
| What makes a tax desirable? | What makes a tax doable? | Which tax do we want? |
| "A good tax gets at the yield to capital". | "A doable tax is a withholding tax." | "Corporate Income Tax". |


<!-- I don'T think I do the kind of prismatic etc. balanced block design as per \cite[30][]{Brown1980} because I don't have ex-ante theories of what the pattern of understandings will be; I just hypothesize that they will change and straighten up.
As a result, an ANOVA is probably not appropriate (ibid.)

Or maybe I can do this, kinda like \cite[35]{Brown1980} on (Robert?) Reich:

also compare 60 in \cite[35]{Brown1980} that might be it

somewhat oddly, thompson refers to beliefs as ``biases'', as though inbiased facts were a possibility even though operant subjectivity can, by definition not be, well, objective and in operant subjectivity on economic matters in particular, is unlikely to arrive at such uncontentious statements.

summary graph on page 69 is very important.

note in a later conclusion that this should be deveoped a) into a broader standard for deliberative effects and b) into a broader study concerning moral development, cognition and thinking about the economy.

Notice that a lot of the ecologically occuring statements on tax are empty, false or non-sensical and thus should \emph{not} be in the sample.

Shalom Shwartz wrote a q-diss!
	Schwartz, S. H. 1978. Reflections on a Q dissertation and its opposition. Operant
Subjectivity. 1: 78-84. -->

<!-- The respectful give and take of deliberative citizen participation is expected to change political views, though in which way, and how to measure it, is less clear.  -->
<!-- On the one hand, conventional survey research offers rigorous and advanced analytics, but is marred by deductive operationalizations and often falls short of a convincing measure of communicative rationality.  -->
<!-- Qualitative approaches, on the other hand, leave much to the researcher's discretion, and risk conflating a measure of deliberative quality with the researcher's substantive theory of justice or reason. -->

<!-- As a mixed method, Q methodology is better suited for the study of deliberative democracy. -->
<!-- It offers a holistic, and operantly defined, yet quantitatively rigorous view on political subjectivity. -->

<!-- Dryzek and Niemeyer have shown that Q methodology can be a productive *mixed* method to study deliberative democracy: It offers a holistic, and operantly defined, yet quantitatively rigorous view on political subjectivity. -->

<!-- however, q tradittionally provides no clear way to study the *treatment effects* of some participatory intervention -->


<!-- # Habermas Ach Europa
so clearly, we want only a notion of a plausible *possibility* of rational solutions, we don't want to look just at rational solutions (first vs second order).
However, you can't just defer this to process; that would be insufficient, and turn full circle. A deliberation is good if it makes rational solutions plausible, and solutions are plausible if they result from good deliberation. Discuss the different operationalizations about this, including DQI.
So what we need, is some kind of standard that is not procedural, but also not substantive, but in-between, kinda like  Rawls. But we don't have that ideal position (but we might approximate it?)

The question now is: can Q-method, or intersubjective rationality and meta-consensus *be* this thing, and if so, why.

Q Methodology, maybe via its quantum theoretic reading (it's about viewpoints, and there exist no objective statements (like zero energy atom states) outside of it -- that's not soo far from discourse theory.
	on this see especially Wendy S. Rogers in operating subjectivity; she argues it is and/or can be used as a form of discourse analysis.

Somehow the stuff that happens during deliberation is actually quite close to some of the stuff that q methodologists rights and that stephenson assumed; it's in expressing and exchanging views vis-a-vis subjects. Both approaches negate that there's stuff "in the braiN" that just need be extracted. -->


<!-- One fitting conceptional framework might be: Habermas source of reasons from the lifeworld, devoid of power. 
This, fittingly, is also one formulation of the ideal scientist.

## goldstein keoane 1993

> "We do not seek to explain the sources of these ideas; we focus on their effects" (7)
Notice: my view isn't really about ideas as explaining anything; ideas *are+ the stuff. -->

<!-- ### On Reflectivism: (similar to constructivism, symbolic interactionism)

> "beliefs are a central element of all research because (...) analysis turns on how 'knowledgable practices constitute subjects'. Reflectivists 'share a cognitive, intersubjective conception of process in which identities and interests are endogenous to interaction, rather than a rationalist-behavioral one in which they are exogenous.'" (5)

> "This explains how preferences are formed and how identities are shaped"

**Bingo! This is the exact place of pluralism vs. deliberative fora on tax. This is the kind of process I'm looking at.**

> "Reflectivist students of the impact of ideas on policy often argue that interests cannot be conceptualized apart from ideas that constitute them, and that it is therefore futile to try to distinguish interests and ideas analytically, as we seek to do. Adopting such a view, however, would foreclose the possibility of evaluating the hypothesis that ideas are hooks against our argument that they often exert a major impact on policy." (26)

### Three Types of Beliefs

1. World Views
  > "conceptions of possibility" (8)
  > "the connections between world views and shifts in material power and interests are complex and need investigation" (8
2. Principled Beliefs
	right from from, just from unjust (9)
	> "mediate between world views and particular policy conclusions; they translate fundamental doctrines into guidance for contemporary human action." (9)
3. Causal Beliefs
	> "... about cause-effect relationships which derive authority from the shared consensus of recognized elites, whether they be village elders or scientists at elite institutions" (10)


### What is Q-Methodology?



### Why Q-Methodology?

- Works with *small* n (of people), but needs *large* n (of statements)
- Blaug 1997 107: is one of the few empirical methods available for the systematic study of intersubjectivity that "has been informed by discursive and domination-free notions of opinion formation"
- Watts & Stenner 2012: K157: "reveals the key viewpoints extant among a group of participants and allows those viewpoints to be understood holistically and to a high level of qualitative detail."
- Watts & Stenner 2012: K501: "allows us to interpret the emergent factors, and hence to understand the nature of shared viewpoints we have discovered to a very high level of qualitative detail"


### How Q-Methodology?

- People will receive sets of statements (40-90) that they rank order to fit under a normal distribution.
- The statements will be a *structured sample* (equivalent to "purposive sample"), drawn from my prior work.


### Analysis

<!-- Onsampling Q statements from non-natural occurences, from some email from the list

> I agree with Lucy Parry on both counts:  (a) Q statements are not variables, but elements in a sample, drawn from the universe of subjective communicability; and (b) statements ought to be altered as little as possible from their original form.  In a doctoral dissertation from some years ago (Newman, 2005), statements about physician-assisted suicide and euthanasia were drawn from two academic articles that claimed comprehensiveness from Kant onward, but the language was in academese and so some words were replaced with equivalents that were more accessible to members of the general public who, along with experts, would be taking the Q sort—e.g., by converting “The patient’s values should form the basis for the regimen to treatment” to “The patient’s values should form the basis for medical treatment.”
> Mine is VERY similar to this design:


But to respond to Jonathan Bishop….  He says that "the selection of variables [by which he means statements] must be based on theory,” but the statements are in no way dependent upon the theory in the same way that scale items are dependent upon the concepts that they are assumed to represent.  In the above-mentioned study of doctor-assisted suicide, for instance, the authors who were summarizing the debate (Fins & Bacchetta, 1994, 1995) claimed that it boiled down to three principles, viz.:

PRINCIPLES
(a) deontology  (b) clinical pragmatism  (c) consequentialism

VALENCE
(d) pro  (e) con

All statements that they reported fit into one of these three categories, and to expand diversity in the Q sample, statements both for and against doctor-assisted suicide were included.  Eventually, n=6 statements from each of the (2)(3)=6 categories above were selected, for a Q sample of N=36.  The role of the above factorial design ends with the selection of statements, however, and the results are not analyzed via variance analysis, which preserves the categories; rather, by way of factor analysis, which preserves the subjectivity (Brown, 1999; Stephenson, 1993/1994).  In Q, it is rarely the case that there are hypotheses to test, in keeping with the hypothetico-deductive framework; more commonly, there are structures to be found and understood, in line with an abductory framework.e



Notice that maybe the condition of instruction really did change during my study or it should in future: people can give their viewpoint from their own view, or from a rawlsian or Sth especially if the conference proceeds and take it as real civic duty.
Consider for example that Paul stenner had people draw scenarios of an experience of jealousy to get people in the headspace.
Civicon might be something similar.

Problem is: is that then an artifact or a result?

Diss: a little bit of data is a very dangerous thing

> -->

<!-- It is perhaps unsurprising that intellectual humility, and a concomitant public discourse ethic are hard to operationalize, and harder to promote (as per Question 2 of the RFP). 
Intellectual humility realism would seem to defy direct measurement or overt promotion by definition: inner virtue cannot be incentivized, but is easily faked.
A plural understanding of intellectual humility appears more hospitable to empirical research and intervention, yet, when pressed into such service, each *single* concept – say, "openness to new ideas" – easily crumbles into a formalistic shell, somehow *less* than its share of the sum of all polithetic sibling concepts.
Aside from attractive, but counterfactual ideal speech situations [e.g. @Rawls-1971], communicative action is similarly slippery in the field.
Taking a *substantive* perspective, researchers and activists risk becoming judges of just *what* is the "better argument" deserving of a "forceless force" [@Habermas-1984].
Merely *procedural*, often *negative* operationalizations – say, absence of coercion, or number reasons provided – may be necessary or correlated, but they fall far short of the telos of human speech: reaching understanding.

It may be especially valuable to learn to imitate (and therefore understand) those principled commitments behind *ideological viewpoints*, on which reaching *metaconsensus* – agreeing what to disagree on – might be our best, humblest hope [@Niemeyer2007].
I use ideology here not as a pejorative, but simply as a shorthand for complexly interrelated sets of *ontological beliefs*, *axiological values* and *policy preferences* (ibid.).
In the domain of taxation and the economy, for example, ontological beliefs might concern the nature of human motivation (homo economicus, or homo reciprocans?), axiological values might prescribe a consequentialist or deontological ethic, and policy preferences might differ on the appropriate base (income or consumption?) for taxation (an expansive list is developed in my dissertation).
Because ideologically *a priori*, these viewpoints are all *non-falsifiable*, when pared down to their core convictions.
This is obvious for values and preferences, but also holds for beliefs, when man-made institutions are concerned, as is the case in policy and politics. 
Within uncertain biological limits, these institutions shape the very human nature which they were designed (or evolved?) to augment [e.g. @Dawkins1976].
For example, whether and how "humans react to incentives" may, to some degree, depend on the institutions under which said human grew up, including tax and other economic institutions, thus possibly turning "homo economicus" from falsifiable assertion to political choice.
This emancipatory claim – that we are, or could be, the masters of our own nature – is, of course, itself contested.
People disagree to what extend we can, or should, transcend our ancestral baggage by institutional intervention.
Disagreement on ontological beliefs, in other words, recurs to a second-level axiological question of whether, among other things, to *let evolve* or *design*, contesting the boundary between values and beliefs.

Thus teasing apart ideological viewpoints is not merely constructionist casuistry: it is, rather, an act of intellectual humility, while remaining deeply committed.
It is humble, because by stating the normative – not claiming the positive –  you embrace your fallibility and context, and speak in terms that reasonable people *can* disagree on.
It is civil, because by laboring to pare down another's viewpoint to such deeply-felt convictions, you honor – not merely tolerate – the other, you strive to see things *her* way, from the *ideological inside*.
It is principled – not relativistic –, because by *achieving such true disagreement*, you clarify your own viewpoint, but do so in terms comprehensible to those who disagree.

An ITT is a fitting paradigm to measure such metaconsensus, but much depends on the implementation details.
Existing, sparsely documented designs use open-ended communication prone to idiosyncrasies and inaccessible to quantitative rigor or [survey questions](http://econturingtest.com), which again turn on *operational* definitions of, say, a "libertarian" viewpoint, sure to be controversial.
Q methodological approaches provide a more organic, because in turn *operant* definition of subjectivity as pure behavior [@Brown1980].
In Q, participants rank-order statements on some spontaneously meaningful dimension, such as truthfulness (belief) or desirability (value).
Using items as *cases* and participants as *variables*, shared patterns are then extracted via principal principal analysis [@Stephenson1935].
Resulting rotated component scores can then be interpreted as ideal-typical rank orders of empirically *shared* viewpoints.
Crucially, Q method yields holistic *patterns* of item ranks, where, for example, the rank of one item can be interpreted *ipsatively* in light of the position significant other items.
Q methodology can also be productively expanded to cover belief, value and preference dimensions as well as categorical judgments.

Moreover, these 17 ordinary citizens refined their viewpoints, and clarified their disagreement after they participated in the week-long [CiviCon Citizen Conference on Taxation and the Economy](http://www.civicon.de) I hosted as part of my field research in Germany.
Participants also noted that the Q-sorting activity *itself* helped them to clarify their own, and opposing viewpoints.

The debate on taxation and the economy is marred by division like few others, and in dire need of intellectual humility.
As I show in my dissertation, underneath contradictory positive claims, and undergirding the rich scientific literature, lie contrasting viewpoints rooted in deep ontological and axiological commitments.
An ITT-Q can be developed and tested using this domain, but should be extensible to others.


![Illustration of a Paper-based Q-sorting Process](img/Qsorter.jpg) -->

<!-- q methodology is "zwischen den stuehlen", and for that reason alone seems appropriate for this research project (jk).
It's a gallisches dorf, and as happens to be, they are right about some things, they have real enemies, and they are misunderstood by most.
So the voices become more strained, more insular and isolated.
(A prominent Q methodologists recently characterized their annual meetings as a "freak show", ) "humane and mathematical" (Brown 1980: 263) -->

<!-- explain how the sorting proceeds from a zero state, distending from zero - this is where subjectivity becomes operant -->

<!-- incidentally, talk and thought as action (Bentley 1908 177 in BRown 1980 331), is oddly foreshadowing ideas of perfomativity -->

<!-- Notice in my prologue that the first research I did was WVS; exactly the kind of research that Q is opposed to, oddly.
Also, I know do tax, exactly the kind of research that I was previously opposed to. -->

<!-- In a word, q method is "subjective" but not "arbitrary" Brown 1980: 257 -->

<!-- Saturation sampling is mentioned citing Glaser and Strauss 1967: 61-62 in Brown 1980 -->

<!-- comment more broadly on the inadequacy of survey research on welfare states;
- both because WS is a non-issue,
- because items on WS are bs
- because survey research makes people passive, instrumental rationality, it's not about the context
- notice how more often than not, the data availability of surveys determines the research (including that there are no widespread surveys on preferences towards the economy) – so we really have to look into the politics of survey design, especially the BIG surveys, the wield great power. -->


<!-- Notices from video
- Concourse is similar to comm. practice (Habermas)
- Concourse is old concept from cicero, stephen brown video 1
- similaratiy: It's PRAGMATIC, about SOLVING PROBLEMS (loosely defined)
- Video 2: any single statement concourse be the launch pad for a new entire study, so you kind of start at an arbitrary stage
- random numbering is important to Steven Brown video 2 -->


<!-- > In placing emphasis on the centrality of self-reference, as outlined in more detail elsewhere (Brown and Taylor, 1972, 1973), we run the risk of being lumped with those phenomenologists and existentialists who tend to reify the self, denuding it of all meaning in an effort to save mankind from mathe- maticians. (Conversely, in suggesting the use of factor analysis, we run the opposite risk of being accused of having made the world unsafe for humanists.) It is therefore necessary to rescue the self and subjectivity from any meta- physical smoke screen behind which others may seek to conceal it.
Fundamentally, a person's subjectivity is merely his own point of view. It is neither a trait nor a variable, nor is it fruitful to regard it as a tributary emanating from some subterranean `stream of consciousness." It is pure behavior of the kind we encounter during the normal course of the day, as when a person prefaces his remarks with "As far as I'm concerned ... ," or "In my opinion
... ," or whatnot.
- brown 46

brown 48 speaks in fn of converse non attitudes


broadl q vs r
> In R, columns are singled centered around the postulate of individual differences for objectively scorable traits; the elements of the sample (persons) do not interact. In Q, rows are single centered around the assumption of intraindividual differences in significance; the elements of the sample (statements) interact in the course of Q sorting. In R, traits are variables; in Q, persons are variables. Owing primarily to the subjectivity involved in Q technique, the results from Q method cannot be reduced to those obtained in R, each being subsumed by a different data set. Claims to the contrary notwithstanding (e.g., Burt, 1972), Q and R are not merely reciprocal ways of examining the same matrix of data.
> brown 55


Q method, is not, in fact a non-positivist research project per se.
Stephenson, in the foreword to Brown 1980 boasts:

> modern science has prospered by eliminating whims and arbitrary subjectivities from its fact-finding missions into the world "outside."
> Q methodology follows the same prescriptions for what we consider "inside" us, matters of mind, consciousness, wishes and emotions, and it does so in terms of theories, universals, and laws, precisely as for modern physics.
> Stephenson in Brown (x)


In way, Q method *does* venture to test the claim of radical constructivism, that we each live in our own world, incomprehensible to others (cite Verena's story on the guy who can't read or write), it does so by exposing us to the combinatorial explosion of the Q sort and if, from that combinatorical explosion, some common factors emerge, the simplest explanation would in fact be, that people *do*, after all, understand language in some broadly congruent ways.

The difference between Q methodology and survey research can hardly be overstated.
As Brown contrasts, the R methodological or survey research view of "subjectivity"  has been to define these *operationally* --- not *operantly* --- that is, they become measurable only by the researchers hypothesizing:

> Procedurally, components x, Y, and z are declared to be properties of trait A (say, anomy); statements X, Y, and Z are constructed and subjected to tests of reliability and validity; and the scale is administered to respondents.
> - Brown

One may argue, that all empirical science proceeds like that, with say, an operational (even arbitrary) definition of what a meter unit of length is. -->

<!-- really look at the separation of operational from operant, see brown 1980 page 3; on this page is precisely the kind of constraint and deductive hypothesizing that I cannot afford; because that would not be deliberative, and I don't have that kind of hypotheses anyway. -->


<!-- Notice a bit problem: Q methodology does, in fact, enforce VnM consistency, hence in future, we might not want to use it for preferences, just beliefs and values.

Instead of sorting on two dimensions, it might be easier to just sort them twice --- and record the result, should have the same form -7 +7. Or is there a problem?

> "There never was a single matrix of scores to which *both* R and Q apply."
> --- Stephenson 1953: 15 as cited in Brown

Half of this is because the data need to be in the same measuring unit, and even standardizing masks the important mean differences (!) that is key.
On the other hand, data gathered from q data is bad for R analyses because the scores are, by definition, *not* independent.

Q method makes visible (as per Brown 21) not just how much people agree or disagree with a given item, but how important they find that item vis-a-vis other items.
That seems to be naturally the case in deciding values and beliefs on the economy, where not only the valence, but their relative importance matter a great deal, especially values vs beliefs.
) -->

<!-- Great part on Brown 28 on how misleading, logicocategorical R survey research can be; says more about the hypothesizing researcher, than the subject area.
Same may be the case in taxation, especially when you get to the meta items that problematize the relative legitimacy of explanations, something that - with disdain - Caplan called "preferences over beliefs", but that someone else might call "emancipation" -->

<!-- > "Operational definitions begin with concepts in search of behavior; operant definitions begin with behavior in search of concepts.
> The difference in temporal sequence is of the utmost importance to a behavioral science."
> Brown 28

Notice that this (the above) isn't just about deductive vs inductive; it's about the ontology and epistemology of concepts; what are they, and how could they be seen?
Explain the "behaviorism" in here. -->


<!-- Great quote on what happens during Q:

> Given that the sample of statements is structured (table 6), each Q sort can in itself be considered as a miniexperiment, the structure being a kind of thought maze through which the subject's attitude wanders, attaching itself to this idea, rejecting that one, ignoring others.
> Each Qsort is therefore subject to analysis of variance, which for this subject (81) is shown in table 7 along with the analysis of the Q sorts of two other subjects (87 and 812).
> brown 31


## Q Distribution

Notice that the difference between the two is because I need an uneven number of columns --- this may not matter much, except there being a significance to the 0, as per Brown:
> "all meaning distends from the middle" (brown 22)

The forced distribution was defined as follows: -->

```{r q-distribution}
q_distribution <- as.integer(c(  # set up distribution
  "-7" = 1,  # these names are crap, they don't really work, maybe rather make this a matrix if you really want names
  "-6" = 1,
  "-5" = 2,
  "-4" = 4,
  "-3" = 6,
  "-2" = 9,
  "-1" = 10,
  "0" = 11,
   "1" = 10,
   "2" = 9,
   "3" = 6,
   "4" = 4,
   "5" = 2,
   "6" = 1,
   "7" = 1
))
```

<!-- Brown talks about platykurvic etc. on 1980 200

This differs a little bit from what a standard normal distribution would be for the given parameters, of `r sum(q_distribution)` items and a maximum value of `7`. -->

```{r compare-automatic, include=FALSE, eval=FALSE}
make.distribution(nstat=78, max.bin=7) - q_distribution
# notice I am not doing that anyway
```

<!-- TODO MH: make visualization of bins -->


<!-- ## Q Concourse

Brown says it clearly: 28: it's easier to draw a definitive universe of people from which to sample, because there are registers, and they are also discrete.
Statements are not discrete, and there are possible infinities.
Loevinger is scited in brown on how it is impossible to delineate. -->
<!-- TODO MCH: refer to what matthias wrote about language and the possibility to express infinite things with finite elements, chomsky or something -->

<!-- > "more art than science"
> -brown 186


on synthesized statements via Q listserv from Steven Brown:
> In a doctoral dissertation from some years ago (Newman, 2005), statements about physician-assisted suicide and euthanasia were drawn from two academic articles that claimed comprehensiveness from Kant onward, but the language was in academese and so some words were replaced with equivalents that were more accessible to members of the general public who, along with experts, would be taking the Q sort—e.g., by converting “The patient’s values should form the basis for the regimen to treatment” to “The patient’s values should form the basis for medical treatment.”,

from the same email, clarifies quite well why I'm not getting out the same vars as I plugged in:
> But to respond to Jonathan Bishop….  He says that "the selection of variables [by which he means statements] must be based on theory,” but the statements are in no way dependent upon the theory in the same way that scale items are dependent upon the concepts that they are assumed to represent.  In the above-mentioned study of doctor-assisted suicide, for instance, the authors who were summarizing the debate (Fins & Bacchetta, 1994, 1995) claimed that it boiled down to three principles, viz.:
>
> PRINCIPLES
> (a) deontology  (b) clinical pragmatism  (c) consequentialism
>
> VALENCE
> (d) pro  (e) con
>
> All statements that they reported fit into one of these three categories, and to expand diversity in the Q sample, statements both for and against doctor-assisted suicide were included.  Eventually, n=6 statements from each of the (2)(3)=6 categories above were selected, for a Q sample of N=36.  The role of the above factorial design ends with the selection of statements, however, and the results are not analyzed via variance analysis, which preserves the categories; rather, by way of factor analysis, which preserves the subjectivity (Brown, 1999; Stephenson, 1993/1994).  In Q, it is rarely the case that there are hypotheses to test, in keeping with the hypothetico-deductive framework; more commonly, there are structures to be found and understood, in line with an abductory framework.
 -->

```{r q-concourse}
q_concourse <- import.q.concourse(  # import concourse
  q.concourse.dir = "keyneson/",
  languages = c("english", "german")
)
```

<!-- Heady ideas seem to have some history in Q, as for example the items on Brown 36


## Q Sampling

more on the factorial design, sources for sampling structured in brown 188

> "Structuring ensures balance, however, and so is recommended where practicable."
> --- Brown 38

My point exactly --- you just can't do it otherwise, if you where to randomly draw statements, very few deliberative state
ents would show up; it would be an echo chamber, greatly influenced by the *distribution* of viewpoints, a concern anathema to q methodology.
We don't care how many there are of some viewpoint.
This would also be counter to the deliberative sentiment; an argument has strength not because of its popularity, but because of inherent qualities.

Also, Brown (38) suggests that as a practical matter, most people won't be able to tell the difference between a structured and random sample of statements, which, I guess is good news.
He does not, however, provide evidence for that - and it's not obvious how that should be tested. (bit it might --- gh issue)

> "As should be evident, however, theoretical structure in no way obtrudes on a person's rendering of his viewpoint, any more than the theory of relativity influences the speed of light.
>  The factorial structure is not necessarily a hypothesis for testing, although it can be used as such; it merely provides a possible, as opposed to necessary, explanation of the resulting factors
>  --- Brown 39

now that might be a bit of an overstaement; surely garbage in will dictate garbage out, and a hypothetical, baad sample may greatly constrain the expression of soem viewpoint.
but admittedly, the factorials need not be expressed in factors, though overly "factorial" item formulations will make it hard to see what else might be at play in people's viewpoints.

Still, *this* is where the sausage is made in Q.
It's the dirty secret, and not a lot of people talk about it.
Factor extraction, by contrast, is out in the open, and may be subjected to alternative, but conceivable standards.

We need a carpet; it's just a way of looking at it, just a model, and it shouldn't be overstated --- especially because the dimensions (2?) and the geography of the room to be tiled are, by definitio
, unknown.
A q sample (and concourse) will always be more of *regulative* ideal --- hence the github account.

What we have here is a *factorial* design of statements (not of people!) -->

```{r q-sampling-structure}
q_sampling_structure <- read.csv(  # read in sampling structure
  file = "keyneson/sampling-structure.csv",
  fileEncoding = "UTF-8"  # should not be necessary, all data is ascii
)
```

<!-- generally nice quote: q methodology lets "subjects (sic!) speak for himself (Sic)" brown 45

This sampling structure is, in fact, very similar to what Brown does on page 30 --- except that the other axis is, arguably, farther away from the resultant factors than in brown#s case.
My are not actually that theory driven; they are history-of-ideas driven ("marxism", "liberalism") --- they need not imply that actual people had these ideologies today, but merely that these might be some co
sistent and widely divergent ways of thinking about taxation.
These are heuristics, not more.
This works only so-so- in my case, and it's more of a guideline than an actual balanced-block sample.
Maybe, just maybe, that's ok, as Brown ()

> "In our rush to construct and test models, however, we are apt to forget a basic principle, namely, that all models ought not be obtrusive.
> ..."
> --- Brown 30

Brown says the good thing is that through q, even from a limited number of statements, people can still give us there viewpoint.
This is, in fact, another example of the elements of language.
Garbage-in-garbage out, unfortunately still holds, if with qualifications. -->


```{r q-sampling}
q_set <- build.q.set(
  q.concourse = q_concourse,
  q.sample = q_sampling_structure$handle,
  q.distribution = q_distribution
)
names(dimnames(q_set)) <- c("items", "languages")
```

```{r make-cards, eval = FALSE}
library(qmethod)
make.cards(q.set = q_set[sort(row.names(q_set)), ], 
           study.language = "english", 
           output.pdf = TRUE, 
           show.handles = TRUE,
           file.name = "Output/cards-civicon_researcher",
           duplex.double = TRUE)
```


<!-- This is from lorenz project prop:
> Moreover, depolarization happens only on factual and much less on value-laden issues (Vinokur and Burnstein 1978).

Deliberation is not something you can be sure of that it happened; it is a regulative ideal – so that causes quite a lot of problems for studying it, especially if you're interested in its RESULTS or effects; it almost becomes circular.

5The experimental literature on deliberation is rapidly expanding. To a large extent, however, it is descriptive (see, for an overview, Karpowitz and Mendelberg 2011). -->


<!-- notice that actually, weighting and flagging are again, latent variable models thinking
Maybe the effectiveness on consistency and other q sort stuff depends on the complexity or modernity of the issue? Plausible say abortion prostitution or assisted suicide might not change as much in values beliefs preferences because they are so much less complex
 -->

<!-- ## Lego Analogy -->

<!-- the lego analogy also works for the information that this is about; relative, and operative --- though it is easy to overstate, because the kind of object your can build may in fact be more restricted than it is the case in legos.
though with only simple bricks (say, no transparent elements, no surface, no angled bricks for roofs, you end up with pretty brutalist architecture) -->

<!-- Brown 1980: 13 on the trick for transposed matrix; it's gotta be the same units.
Look at Brown 1980: 14 for the body parts analogy; putting it back together in R requires researchers' subjectivity. -->

<!-- garbage in garbage out is in Brown 1980 262 -->

<!-- #Niemeyer2007

487: "there is considerable uncertainty regarding the appropriate nature of desired deliberative outcomes".

500: Meta-consensus, "or agreement about the nature of the issue at hand, not necessarily on the actual outcome"

500: inter-subjective rationality, "when individuals who agree on preferences also concur on the relevant reasons, and vice versa for disagreement"
507: "any pair of deliberators with similar subjective positions -- in that they agree on values and beliefs -- ought also to agree on preferences"


    <!-- ## Methode Q-Sort

    - Q-Sort wurde in mehreren Versionen 8 mal durchgeführt, Änderungen eingearbeitet
    - Q Methode wurde im Doctoral Colloquium vorgestellt, Feedback von Christopher Cohrs (UB) eingeholt
    - Siehe Auszug DC Präsentation (anbei) -->


<!-- notice that as per Brown around 75, q methodology might also be used as a tool for deliberators to figure out exactly what kinds of policies they might agree upon!

Item ``taxation is the price we pay for a civilized society'' (o.W. holmes)


Main Effects		Levels
Consciousness 		a) Con I  	b) con II 	c) con III
Values  			d) personal 	e)lega-political 	f) socio-economic

Q-methodology and habermas! Must look into this!

___
McKeown

notice that factor loadings are actually *not* conventionally very important in Q; mostly factor scores. That'd be different for me; I'd wager the most general hypothesis that factor loadings would improve.

run interviews to gather the q concourse?

discuss the whole adapted vs. naturalistic controversy: in some instances, naturalistic items might work; but not for tax, because here some things are just *never* discussed, that's the very idea of this research.
(it is, in fact, questionable, whether the same problem might not also plague other research, as on love. Non-existent counterfactuals are always important – say, on love. You always have to be careful with hegomony in the language. Of course, the balanced-block design takes care of this already)
In my research, I'll make sure that items are readily comprehensible in lifeworld-terms, which, luckily *is* very possible for taxation, because it is about the oikos.
Of course, non-naturalistic, ``adapted'' items bring back the issue of external observation ,or rather, of things that the researcher has put in here. This problem cannot be entirely negated, but q methodology reduces it significantly.
For once, people can still sort items they do not understand or find odd in the middle, near 0, what Stephenson described as the quantum origin of the scale, with 0 information.
But also, no matter which other items were added to the q-sort, or on which other p-sample it would be replicated, the results of this q-sort would still be valid, because it's about operant behavior.

got to look at the dryzek political discourse q thing from 1993

apparently positive or negative can also be a line in the balanced block approach, as in \citep[23]{McKeown2013}

should I describe severeal conditions of instruction, like: what the actual tax system is like, what it should be like? (Maybe not, because that would be true/false)

In q and p sort, the criterion is saturation.

make people sort items on equity and effifiency? That might make some sense (even though my point is that both expand, which really won't work here.)

---

Niemeyer Diss
- check strong democracy barber

Look at dryzek and list 2001 on metaconsensus.

maybe in opposition to niemeyer, I think you can impose one normative theory – rawls, because it's meta.
just like intersubjective rationality.

Dryzek 1990 discourse theory – this seems important, it is what niemeyer is drawing on!

Intersubective rationality -- that people want the same preferences, because of the same values and beliefs –-- is kind of a strong assumption, because we don't strictly speaking know that such a consistency exists (except and in so far as it merely follows from vNM). But we can accept this as a regulative ideal; something to be strived for, not achieved. And if we never get any further to it, or can't, with that would wither the plausibility of deliberative democracy as an utopia, which, after all, depends on this.
Look into vNM to say that we have collective preferences might imply that we also have this kind of consistency.

get pelletier 1999 as example on q in deliberation

get dryzek 1993

discuss deliberative failure as in niemeyer diss6 69f

i should have some kind of stratified sample, too -- family, entrpreneur, worker etc. -->

<!-- my tax items will, obviously, still be real and part of concourse – they are just on the very periphery, which is exactly what I am looking at here.

niemeyer diss 158 compares q methodology to making a species survey of a small area in ecology; that's a good analogy why small numbers work. Also refer here to the brown example of calculating the species factor between arm, leg etc length.

remind people when they do the q-sort that they are not at all being tested; not the correctness, nor the consistency of their ranks (between the two points in time) is tested.

need to argue carefully why the actual policies (taxes) should be q-items, too
1) because they're difficult to understand, as opposed to the bloomfield options
2) because q measures what we'd need to know about them anyway.
3) they are many-dimensional, and their vary dimensionality (much like the other q-items) is of interest. They can't be graded on a n access to non-access dimension as bloomfield (214)

check again dryzek and list 2001 for preference structuration

---

\cite{Dryzek1993} is very important, completely a fan.

The most radical meta-standard that could be formulated would indeed be pure q methodology, with a focus on the loadings and numbers of factors, maybe the three-step of dryzek and niemeyer. That's a tall order, and I'm not sure it works, but that would be the ideal.

Maybe it is not an accident that Dryzek stressed so much the reconstructive merit of q (for democratic theory) and went on to be a formative voice for deliberative democracy.

\cite[52]{Dryzek1993} write: ``Toput it another way, our units of analysis, when it comes to gener- alization, are not individuals but discourses. And we soon get to a point where adding individuals to a study does not yield any new informationunless the extra individuals are truly different-another reason for stressing diversity in subject selection.''
In other words, q methodology works with small n, because it is concerned with the dimensionality, or structure of subjectivities, not the prevalence. Notably, q methodology is downward compatible; if you have a lot of people do it, you can become ever more confident in the relative factor *loadings* (which is what you'd be interested, then). Notably, this is not the case the other way around; if you administater an r survey with *lots* of items, it's still not same as q, because it lacks the same unit.

\cite{Niemeyer2013} is also about a local issue, and even shorter (the birdge thing)

also note that focusing only on the overall q loadings would be sort of against the theory; and conflict with judgemental rotation. Still that would be the general effect of which this might be a special case, if the list/dryzek piece has it right.

also notice that

---
notice that \cite{Brown-2006} does something quite dissimilar; the locals are right, always. I can't do that for tax.
But I should also point out that my research interest is quite circumspect.

--

notice that q methodology does sit a little uneasily with critical theory, including habermas, because it (initially) seems to permit *no* false consciousnesses.
This, of course, is because of the items in the q sort are not supposed to be factual questions, but one of preference (ideology being facts wrapped up in norms.) (uh-uh, this is bullshit)

my selections of q sorts is essentially statements that follow habermas' rules for communicative rationality; it would be non-sensical to ask the for the subejctivity of powered speech.
This might at first seem interesting (and in line with my research), but it violates the central tenet of q methodology, namely to take operant subjectivity serious and place it front and center.
the participant, in other words, is always right.
Also, ``testing'' for powered speech, would be hypothesis testing – which is not supposed to happen with q; I would, in effect, be interested in the movement of individual items.
Instead, I am looking at the dimensions.
Also, if there is such a thing as communicative rationality, it should reveal itself in neatly aligned factors, if not before, than after deliberation.
The same logic still holds; If, as predicted, the factor space clears up, something must have prevented it from clearing in the first place. That something is powered speech.

---

\cite{dryzek2005handle} This seems important, right down my alley for why I need q.

describes why in-depth, etc. are all bad; also how I would become a deliberator in an in-depth interview.

--
\cite{Durning1999} cite this guy; that's a pretty damning review at the beginning of positivism.

--
\cite{hutchinson2012} on complexity, or a post-positivist use and advocacy of q; that's not really what I'm doing, I think. Or maybe it is.

--
\cite{Niemeyer-2011-bookchap}

notice that I am doing something new; there are no q sorts on the economy, or on such highly structured/complex material, or with this research interest.
So I have to develop them myself, from the literature. That's hard.

Should I ask directly for metaconsensus? (no that would be hypothesis testing!) Instead, metaconsensus should emerge as factors.

Notice that single-peaked preference orderings (as in 19 in the above) are a special case that may or may not be observed here; it is difficult to find in terms of taxation, which is multi-dimensional (at least equity and efficiency).

My beef is not with a meta-substanstive standard, that is ok – and there *might* be such a thing as a universally acceptable, rational decision; but it's wrong to assume that in the first place. If might be a regulative ideal, and so should the test be designed. It's something that can be approached, or maybe not, or maybe not in our lifetime. (because what young et al are really worried about is power that persists, and clearly, that might still be the case, and that might be a good argument for deliberation to the extreme in our time (see archong fung on this))

Note how q fits well with my earlier admonition of piercean pragmaticism.

Also note how the standard to which items are held, is again, fittingly, a regulative ideal ``something that people can reasonably disagree with'' – and I might fail that standard.
Importantly, I can still test that.
Also note:
- q items can be double-barrelled etc.
- q items should not be true/false
- q items should not be phrased as hypothesis-testing
- q items should allow deliberators to say anything that could be said (cover the floor with carpet tiles)
- q items should be in terms of the lifeworld
- q items should not ``cheat'' or ``test'' the sorter.

Think again, hard, on why I can't have (too much) stuff from the newspaper in here.

Q methodology also lends itself because, except for the apriori theorized work by McCaffery et al, there isn't an awful lot of theory that would tell us exactly how people ``mis''construe tax.

Factors from q are ideal types of people who score things similarly, the factor SCORE is than the rating a person epitomizing this factor (not part of the sample) would give to that factor.
Factor loadings

Explain again factor loadings vs factor scores

Factor loadings are how much the variance of any particular person is explained by the factor.

Remember that the movement of *individual* items is not meaningful (that goes back to R methodology)

Argue why/if actual taxes should be part of the q sort, too.

Honestly, there is also the point of small n, which is great.

Q methodology (or rather niemeyer/dryzek) should also isolate, to some extent, the charge that I might simply talk people *into* any one tax.

make clear; ideally, this methodology is an *iterative* one, because there might be reasons or voices that the q-sort does not include, but that should be included in future.
So I will document whatever flows out of this, revise the q-sort afterwards.

Also make really clear at the outset that the q-sort is not that all-important; there are gradations.

Also notice that I have tested and expanded on this q-set initially with some items from quite extensive pre-tests, literature research, as well as online.

How are Intersubjective rationality and metaconsensus explicated in Q results?
Figure this out.

Explicate where this research is pushing the boundary:
1) highly policized but structured issue
2) radical opening of tax
3) new application of q method to the subjectivity on complex stuff (cite the article)
4) extension, application of metaconsensus and intersubjective rat. to a complex, technical issue with multidimensional preferences (schedule and base)

Somehow discuss franzis issue about between and within group variance!

Niemeyer 2014 shows exactly this; how it works as a regulative ideal, how there might be differences in how different formats achieve or fail by this goal.

Notice that there might be nothing wrong per se with universal, rational consensus.
After all, what are we do to, but rely on reasons; and reasons always imply (as Habermas wrote) sort of as their telos universal consensus, they are proto-forms of such universal consensus.
It should also be clear that any argument will at the end rest on this and there's no easy formal way around this; I can't math myself out of this.
Science, for example, should work on the basis of reasons alone, and we have a sense that this works, more or less (if only we could get rid of some of the power).
Still I should notive that intersubjective rationality (IR) and metaconsensus (MC) are kind of railings on which to hold; they help constrain interpretation and abstract away some of the problems, but not all.

Notice that absent such a meta-standard, and (the probable) absence of universal consensus (except where required by the format as in consensus conferences) we won't know which consensus is the rational one, if there are several competing ones, again falling back on a substantive struggle.
That's bad.
Also notice that as Niemeyer rights, with universal consensus (which can be measured) it's hard to tell whether the consensus is actually rational, consensus (or dissens), or any chance in preferences migh happen for a number of reasons, including may worrisome non-deliberative ones such as groupthink, polarization, mere exposure/priming or social influecne/desirability.

let people pick pseudonyms.

I am not sure I can even respond to the difference criticism of democracy; I disagree with that.
Point to the article on that, on policy sciences.

cite the pornography example: this is a very good example, and I should cite this to illustrate why it is so important to point out, and explain, missing counterfactuals such as the PCT (much like porn for women, by women).
Same for the q-sort.

make a 2-2 table to explain the choice of data gathering format; one is predefined hypothetico-deductive, the other is unconstrained, qualitative.
And then there is a third ... Q

---

\cite{Ockwell2008}

look at the \cite{Dryzek1993} matrix, maybe this works better than merely beliefs, preferences, values?

---

\cite{Pelletier1999} might be the first guy to actually use q to measure deliberative results.

\cite{price1968} uses q methodology concernign the financial behavior of families, but unfortuantely is quiet on the actual items.

---

I should notice some peculiarities about q, including (but can't do an all-on theory of inquiry, for that, see Brown and Stephenson)
- operant subjectivity
- concourse
praise the opportunity and oppeness and respect for participants this method brings; without being unconstrained.



look again at the bal


--- -->

<!-- %My q sort is NOT about the details of taxation, or their empirical questions, not about housing or pendlerpauschale or some such thing

%go for horizontal vs. vertical equity
%go for head tax! this is the most efficient tax!

%via Keep:
	%maybe do pairwise comparisons and rotate that (comparability?)
	%make sure that items are intelligent (not ressentiments), easy to understand and sort of present the best way you could phrase something
	%there's actually a beauty to this, even though so much of the original critical impetus has been lost (or has it)? people deliberating and exchaing some of these reasons IS the last end. And yes, there's probably still an inequality story, but this needed to be shown first.

%notice Fischergleichnis to explain value theory of labor! this might just work!

% notice that my values, beliefs are not the same that fishkin talks about!
	%and still, as fishkin (and someone else writes) knowledge gain is by no means unimportant; it's just that fishkin's criteria are necessary conditions, but probably not sufficient for good deliberation. Meta-consensus might be sufficient conditions; they are what Habermas suggested there *might* be.
--

idea:
Look into ontology
- individualism
- groups/classes
- relations etc
(CIT!)
value theory of labor -->