Subjective Experiments
Doing better than introspection
The above piece is called “Self-Deception” by the artist Mason. I find it a delightfully teasing name for the piece. To me it looks like what people seem to think introspection does, and they’re meaningfully wrong. And at the same time, it’s also an amazing depiction of a method for looking at subjectivity that does way better than introspection.
In today’s post I want to spell out that method.
I have a vision for how we might do subjective science. I won’t try here to explain in much detail what “subjective science” means or why I think it’s worthwhile. The short version is:
There’s this space we’re inclined to call our “subjective experience” or our “mind” or our “interiority”, and
science as it has been done so far has had a hard time really examining that space despite it being extremely important to matters of the heart and soul, but
I’m quite confident that this challenge has been a limitation of how science has been done so far and that we can get around those limited methods.
If you want the longer version, I wrote an article on the topic here:
That “better than introspection” bit that I see in Mason’s art is a piece of how to do subjective science, as I see it. I don’t claim to have worked out everything. Not by a long shot. But I think I’ve worked out enough that people can meaningfully grow as good scientists of their own subjectivities.
Part of what I have in mind here is, I’m putting together a practice group that’ll start with a few core methods like the one I’ll describe here, and sincerely develop subjective science. In part I want a social context that encourages and supports me in doing the practices that I think are excellent here, and I think the natural way to do it is for it to be that kind of support for everyone involved. I want to write this article in part as a reference for that group.
So I’m going to skip most of the motivation and try to get to the method as directly as I can.
Keep yourself out of your experiments
I do need to name a few pieces of logic for why the method is the way it is though. Otherwise a lot of the design seems weird and arbitrary and tempting to skip.
I actually spelled out the lion’s share of the logic in a fair bit of detail in my 2024 chapter on self-reference, particularly the part on how agents cannot self-refer. But that particular part can be a challenge to read and mostly requires the reader to have read the rest of Valentine’s Logos first. So for the sake of efficiency, I’ll just lay out some of the conclusions (without justification) so that you have context on why I’m structuring subjective experiments the way I am.
The key issue is that minds cannot globally self-refer. I don’t mean it’s hard for them. I mean they literally cannot do it.
But if instructed to try, they’ll try anyway, which creates a whole host of subjective illusions.
But you really do need your mind to work well in order to do good science. Scientific knowledge is made of good explanations that you’ve empirically tested aren’t wrong. Getting there requires thinking through explanations and their logic, and designing experiments, and conducting them, and noticing what the results imply. So if your thinking mind isn’t working well, you basically can’t do science.
The solution I’m proposing is to get self-reference out of how your mind examines your subjectivity.
Because of how identity works, on the inside that move should feel like you are examining “your subjectivity” as though it’s an object you’re looking at. But taken literally, that’s directly impossible. You cannot view your own subjectivity from the outside. That’s kind of the whole point!
So instead, you stay very honest about what your mind is actually doing when you try to get it to look at the inner space it’s embedded in. It’s not actually looking at itself. It’s instead creating a mental model of your subjectivity and examining that model. Then you, inside your own first-person experience, peer through your mind at that model.
There’s a trick I find helpful for keeping the mind straight here: when thinking through the scientific logic (i.e. coming up with good explanations and designing experiments), I imagine that I’m looking at someone who’s just like me whose subjective experience I happen to have something like telepathic access to. I’m the scientist examining this subject. I’m trying to understand them, not me.
I find that this move pretty reliably unscrambles my mind. It makes thinking through the scientific method much more precise and clear.
From having walked others through this reasoning in person many times, I’ve learned that it’s helpful to offer two notes of warning. They don’t fully prevent the errors, but the warnings seem to help people notice when they are making those errors and to learn not to with practice:
It’s very common to slip back into first person. (“Oh, I bet I’m doing this thing because of XYZ.”) That reintroduces attempted self-reference. It might seem pedantic at first, but you really do want to keep good discipline here: you are never examining yourself. You are only ever examining a subject who’s outside of you. (“Oh, I bet they are doing this thing because of XYZ.”) Sometimes it’s helpful to refer to them by a name you don’t go by, and/or to use a physical object you’re looking at as a representation of the subject.
Your proposed explanations should spell out why your subject gets the subjective experiences they do. Having them introspect is a proposed experiment. You might use introspection (as the subject) to generate hypotheses, but it’s important to think through why your subject would have the subjective experiences they do in response to that attempt at introspection. “The introspection is accurate” is a hypothesis. You do not check your hypotheses against introspection as the source of truth. Sometimes your theories will lay out exactly why introspective certainty that your hypothesis is false is in fact confirmation of your theory (as long as some other outcome could have disconfirmed your theory).
I’ll give examples of applying these ideas shortly.
Seek to understand problems, not to solve them
But before I give examples, I need to name one more bit of theory:
In general it’s often helpful to start with a problem. Something like “I keep having thus-and-such struggle in my romantic relationship” or “I feel bad a lot of the time” or “I want to flourish more but I’m not sure how.”
But if you try to directly solve the problem, you’ll typically be coming from inside it. In lots of subjective cases, how you try to think about and solve the problem is actually part of the problem.
There are lots of examples of this pattern, but here’s a common one: someone who’s habitually clingy in relationships might be acutely aware of how their clinging drives people away. They might try to explain their conundrum to their partner, but with a tone of desperation, like “Please understand that I’m struggling with this so you don’t leave me if I seem nervous!” That effort can have literally the opposite effect than they’re consciously trying to have precisely because they enact the problem in their attempt to solve it.
When you don the role of being a scientist, put any urge to solve the problem in your subject. It’s fine if they strongly want the problem solved! It’s fine if they desperately want this scientific process to address the problem. All of those are part of the subjective system you’re examining. Your job, as the scientist, is to develop and test good explanations for why that system behaves the way it does. Seek to understand the problem rather than to solve it.
(“What good is that?” you might ask. Well, it turns out that in the long run this approach is vastly better at solving most problems! This insight is core to science. E.g., chemistry arose from alchemy by setting aside the goal of turning lead into gold and instead seeking to understand why and how specific chemical reactions happen the way they do.1 Sometimes science shows why your goal is impossible, sometimes it shows your goal isn’t something you actually want, and sometimes it makes the path to a solution vividly clear. All three cases are a win.)
So if you think you’re clingy, and you want to understand it using this method, you might think:
Okay, this clingy person exhibits thus-and-such behaviors. They’re aware of those behaviors, and of some ways those behaviors create pain in their relationships. When they try to do something else, they end up getting clingy through their effort to do something else. And in watching me examine their situation, they’re trying to desperately push toward this process solving their pain too. All of this together is the phenomenon I want to understand in this person.
One possible sign that you understand a problem (i.e. have a good explanation for it) is that you see how to make the problem worse. Not always! But if it’s possible to make the subject’s misery or situation worse, and you understand how the problem works, then you’ll know with simple clarity how to increase the badness of the situation. Making the problem worse can sometimes be an excellent crucial experiment.
For instance, if you think your subject’s brain fog is due to how they’re solving a hostile telepath problem, one test you could have them do is find a potential hostile telepath and try to be transparent with them. That should make the brain fog much worse, possibly even from just considering talking to the person.
Tests like that one tend not to come to mind if you’re focused on solving the problem. But they’re natural if you’re honestly curious about how the problem works.
Bear in mind, of course, that your subject has to be willing to conduct whatever experiments you actually want them to run. And some experiments are too costly. (E.g. if the possible hostile telepath is an abusive partner, it might be a very bad idea to try to approach them with honesty.) In general I find it’s helpful to aim for experiments that:
…don’t have any irreversible consequences; and…
…your subject finds the results acceptable to live with, regardless of which results the experiment yields.
(Just to be clear: I’m being careful to keep the subject in third person when talking about your role as the subjective scientist. But “your subject” is what you normally think of as you.)
A subjective scientific method
So with that context, I’ll spell out the method. You might be able to derive it yourself if everything up above made sense to you! But in practice I find that most people I talk to in fact don’t spontaneously derive it, so I’ll just lay it out here:
Pick a phenomenon connected to your subjectivity that you want to understand. I find that starting with problems (i.e. the domain of what’s usually “self-help” or “personal development” or “life coaching” or “therapy”) is often really good. But anything you’re really curious about is fair game!
Imagine you’re examining a being just like you from the outside, whose subjectivity you can peer into. This is the “become a scientist looking at the subject from the outside” move. Be careful with language, and with the thinking that produces language: you are trying to understand them. You are never trying to understand yourself with this method.2
Come up with good possible explanations for what you observe in your subject. The word “good” here is doing some heavy lifting; I really recommend watching this TED Talk to get some precision about the idea. This step isn’t mechanical and requires a lot of creativity. One quirk worth being aware of though: it’s fine to use introspection to generate hypotheses, but remember that you’re doing so as the subject, and afterwards as the scientist your good explanation should explain why your subject’s attempts at introspection caused those hypotheses to arise in their awareness. (It’s more plausible than you might think for how a hypothesis came to mind to prove that the hypothesis is false.)
Come up with experiments your subject could do that could potentially show that your explanations are wrong. The more decisively those explanations could be proven wrong, the better. Typically that part won’t be perfect, just suggestive; just be honest with yourself about how crucial the experiments are or aren’t. The classic error here is to do experiments that are basically intended to give results that are compatible with the explanation; the problem with that approach is that it puts the wrong evolutionary pressure on your ideas. Also, like I mentioned earlier, it’s helpful to make sure the experiments are ones the subject is willing to bear the costs of, no matter how they go.
Drop imagining you’re looking at a subject, and then just run the experiments as yourself. You have to be the subject in order for the experiments to work. You might find that you’re unwilling or unable to do the experiments, in which case go back to the previous two steps: you-as-scientist have learned something about your subject, or maybe have overlooked some detail about what they can do that your hypotheses in fact imply.
Become the scientist again, and think through what the experiments actually teach you about your subject. This step really is critical. Sometimes the experiment is expected to induce confusion, for instance, and you need time outside the experiment to notice that confusion in fact arose. Likewise, the theory being tested might predict that the subject will deny that the experiment produced the results it did. If you don’t do this “become the scientist again” step, you might try to draw conclusions as the subject instead of as the scientist, and the phenomenon you’re trying to understand is more likely to warp your understanding.3
This is really just the usual scientific method, but (a) directed at subjectivity and (b) correcting for challenges that are particular to subjectivities.
For what it’s worth, it looks more daunting than it is in practice. I usually find that it’s relieving to do the core reframe, and there’s something lovely and very simple about the impersonal clarity that comes at the end.
It can be helpful to have someone else’s support when trying to apply this method. They can guide you through the steps and point out where you stumble (e.g. referring to the subject in first person while generating hypotheses). Their help eases off a kind of meta-layer of thinking: instead of being the subject, the scientist, and the person tracking the method, you can offload method-tracking and kind of share the scientist role. You’re also more likely to actually go through the whole process if you have some social support. (Not that it’s particularly grueling, just unusual and a little tricky at first.
This process isn’t meant to be one you engage in all the time, by the way. Same as how you don’t need to use science on literally everything in your life. That’s impractical. The idea is that you engage as a subjective scientist with some phenomenon you want to understand better (or with a problem that’s particularly resistant to being solved), and then when you’re satisfied you understand it well enough, you go back to living your life normally. The understanding might change you of course! But not because you’re stuck viewing yourself in third person.
Some applications
I’ll quickly outline a few use cases so that the method is a little less abstract.
Being “hangry” sometimes comes with insistence that one’s irritation is not from hunger. But sometimes irritation really isn’t due to needing food! A person prone to this confusion might try to introspect on whether their irritation is a form of being hangry. But if they imagine they’re looking at someone else who feels exactly like they do, and they view introspection as an experiment that person could do to try to distinguish being hangry from being externally annoyed… is it a crucial experiment? Clearly not, since the result is the same regardless of the truth. A much better test that that other person could do would be to eat something: if the irritation is from hunger, eating should make it abate; and if it’s not, then eating won’t affect it. Notice that here the aim is to understand the irritation, not to make it go away, so the “scientist” advances in their goal regardless of the experiment’s outcome.
If you have a really upsetting dynamic with your partner, is it because they did something upsetting? Or is it mostly due to a social control disorder (i.e. a habit of unconsciously creating subjective suffering in order to get what you want from others)? It’s predictably hard to think clearly about this in yourself. But if you imagine you’re looking at someone else who’s suffering the way you have been, how could you tease out the nature of their distress? Not help them necessarily, although that might happen anyway. But to understand the structure creating their suffering. One way to test for a social control disorder in your subject is for you (the scientist) to deduce a possible social payoff your subject is getting from their distress, and then have them directly ask their partner for what the subject wants. If their distress dissolves the moment they get a supportive response from their partner, and especially if the distress later results in thinking of asking for what they want, then that indicates your subject’s suffering was one of these disorders. In some cases the subject might fear their partner’s response too much to conduct the test (e.g. in abusive relationships), in which case you in practice can’t have them conduct the experiment and you’ll need to figure out some other approach.
Maybe you have a habit of being attracted to people who play out a painful dynamic, like clingy people tending to reach for emotionally unavailable people. Then you meet someone and get glittering butterflies in your stomach. Are you repeating the pattern? You can twist yourself into knots trying to figure it out. But what if you were looking at someone else who felt this way due to an identical situation? Maybe it’s a “play out the pattern” shadow dynamic, or maybe it’s honest excitement, or maybe it’s just pure lust. It’d be an error to take this 3rd person view and use it to conclude which one it probably is though: that’s likely blurring the boundary between subject and scientist. Instead, take each possibility as a hypothesis and ask: “What could the subject try doing that would tell me, clearly, what the cause of their attraction is?” You haven’t actually learned anything new until your subject in fact conducts the experiment. (It’s really common for people to use this “What would an outsider say?” trick to try to pressure themselves into a conclusion. But there’s really no need for that here. There’s actually extremely good reason not to do that.)
If you’re always late to things, you might think the reason is that you’re just bad with timing. But “I’m just bad with timing” isn’t a good explanation. It’s barely a description of what you don’t do, and quite far from a mechanical hypothesis about what you do do. If you were observing someone else with your timing behavior, including their subjective thoughts, what do you suppose might in fact be causing them to be systematically late? If their patterns of attention reliably have them not noticing the time, then setting alarms and reminders should change their behavior. (But be careful: are you (the scientist) figuring out how to understand the subject’s problem, or are you slipping back into solving it?) Maybe you already know they’ve tried that intervention and it didn’t work. Great, the experiment was already conducted, and the hypothesis falsified! So, what other hypotheses seem plausible about them? What could the subject do that you’d consider reasonably crucial tests of those possibilities?
Some gaps in this method
While I think this approach is very good, and a much better foundation for scientific “inner work” than I’ve seen anywhere else, I don’t think it’s yet complete. It doesn’t get around all the challenges of studying subjectivities.
For instance, it’s possible that someone who’s running strategic self-deception will block their ability to walk through this method, precisely because some part of them recognizes that doing so could make the truth conscious. So they’ll forget about it, or get too tired each time they consider doing it, or get confused and distracted when they start, etc.
Experimental design in that case is a bit more tricky. It requires making sure that a hypothetical and unknown hostile telepath problem isn’t made worse by any step of the process. I’ve succeeded in such a design in a few cases, but I don’t have a general method yet.
But even in self-deception cases, this subjective research method still works much more often than one might think. I think it’s a very good starting place for subjective science.
I’m just wanting to be clear that it doesn’t fully solve the self-reference problem. There’s more methodological work to do.
I also think there’s progress to be made in working out the multi-person version of this approach. I haven’t laid out here exactly what I mean by “subjectivity”, but the definition I haven’t shared implies that pairs and groups have subjectivities too. It’s part of why couple’s counseling works: the therapist can act as a scientist from outside the relationship. But this application is much more hypothetical on my part, and deserves its own future post.
This insight also shows up in computer science. Ken Stanley showed that in a particular very common kind of context, goal-pursuit is actually worse at achieving any given goal than is just seeking interesting novelty. Metaphorically speaking, following your curiosity is often just better at getting what you want than striving directly for what you want is.
The famous phrase “Know thyself” was carved over the entrance of the ancient Temple of Apollo that people would pass through to consult the Oracle of Delphi. In Greek the imperative was written “ΓΝΩΘΙ ΣΕΑΥΤΟΝ” (“GNOTHI SEAUTON”), meaning that the kind of knowing they were naming was gnosis. In that sense it’s quite possible to know (as in intimately/familiarly know, or ken) thyself. What I’m correcting for here is the fact (I claim) that the mind, which does factual/declarative knowing rather than integrative knowing, literally cannot do global self-reference. And therefore in that sense you cannot know (as in factually know) thyself. Had the Oracle of Delphi’s entrance commanded that kind of paradoxical self-knowledge, it would have used ειδησις (eidesis) rather than γνωσις (gnosis), and the inscription would have been written “ΙΣΘΙ ΣΕΑΥΤΟΝ” (“ISTHI SEAUTON”).
A quirky example of this error shows up in an old Less Wrong post. Scott Alexander brought up some studies that claimed that dripping ice water in a patient’s ear can abruptly halt some kinds of self-deception. But about ten minutes after doing so, the patient would go back to rationalizing, and would deny having changed their mind under the influence of the ice water. Eliezer Yudkowsky then went on to try it, but without keeping this subject/scientist division clear in his experimental design beforehand or his analysis afterwards. The result was that the experiment might as well not have been conducted:
