Categories
Education Lifestyle Productivity

Our Reward Value System

“Everyone thinks of changing the world, but no one thinks of changing himself.”

Leo Tolstoy (1828 – 1910)

My recent days of research and reading have led me to unpack the unexpectedly dense world of rewards and reward systems. I’ve been trying to understand how our brains decide what’s rewarding and what isn’t. This has lead to me ask questions like –

Why do we prefer donuts to spinach?

Why are some things more rewarding than others?

My last post was about the importance of understanding rewards and how rewards can trigger consummatory behaviors within us. This post is going to focus more on why we like some things more than others. Hopefully, with this understanding, we can hack our brains into actually enjoying things that are good for us and reduce the friction to creating a life by design.

A big thanks to Dr. Jud for helping me understand this.

In order to understand how our brain’s reward system works, we have to first look at habits. I’ve written a few posts on habits, I recommend checking them out. They are Types of Habits and Designing Our Lives and Understanding Habits and The 1% Rule. Habits are fundamental to our lives and understanding how they work gives us the ability to design our lives.

Basically, we need habits to get through our everyday life. We use habits as a way of saving energy. Let me put it like this, if we had to learn every single thing we did every day, then we’d be exhausted by noon! It takes a lot of energy to do or learn something we haven’t done before and it takes little energy to do things that we’re familiar with. This is why it isn’t too exhausting for most people to get up and get ready for the day. It’s a habit and habits don’t take much energy to do.

But not everything we do is turned into a habit, only some are.

So how we do know which actions to turn into habits and which ones to not?

It all depends on how ~rewarding~ it is.

Our brains have a way of rank-ordering rewards as more valuable and less valuable. This is known as reward-based learning and it has 3 parts.

Trigger

Behavior

Reward

Let me give a few examples of this: Let’s say our alarm clock goes off and we hit the snooze button to stop it. The trigger was the alarm sound. It’s annoying so we want to do whatever we can to stop it. The behavior is hitting the snooze button to stop the alarm as fast as we can. The reward is the alarm stops. This is known as a negative reward – we got our payoff when something is removed from the situation, in this case, the alarm. Now that we got our reward, we are more likely to use this method again in the future to deal with the same situation. This is why hitting the snooze button is so addictive. Every time we hit it, we get our negative reward which reinforces the behaviors to get it.

Let’s look at this from another angle: Let’s say I study really hard for my exam and I get a higher score than I was expecting. The trigger is the awareness of the exam. The behavior is studying for the exam. The reward is a high grade. This is known as a positive reward – we get the payoff when something is given to us or when something is added to the situation that we wanted. In this case, the high grade is something that we got as a reward for our studying. Now in the future, we are more likely to study when an exam comes up.

I want to emphasize that the reward reinforces the behavior that led up to it regardless of what it was. If we cheated and got the grade we wanted, we are going to be more inclined to cheat again. Rewards will reinforce anything, it doesn’t matter what it is.

These rewards can also be intrinsic or extrinsic. I talked a little about that in Consummatory Behavior and Rewards. Intrinsic rewards are rewards that relate to improving the self or other internal gains. These are extremely motivating and rewarding in the long term, but we have to want the intrinsic reward by our own volition. Extrinsic rewards related to anything that is externally given as a result of an accomplishment. These are great for motivating people who aren’t interested in the intrinsic gains from a given activity.

Bottom line: extrinsic rewards are great for the short game. Intrinsic rewards are great for the long game.

Additionally, the more rewarding the behavior, the stronger the habit. I touch on this slightly in my last post as well.

This plays off a system in our brains that we used for survival as cavemen. Back when food was scarce, our brains would prioritize eating sugars and fats so we can get the highest calories possible. This means that when we’re presented with choosing between donuts and spinach, we’re wired to want the donuts every time.

But it doesn’t just stop there.

We also assign reward values to all the people, places, and things around us. Our brain can combine good feelings of donuts, the fun of celebrations, and the friends around us all into one composite reward value which we also give to the donuts. So to us, donuts are much more than delicious balls of fat and sugar, they are also everything great about eating a donut.

In addition to the caloric bias, most of the associations we make with donuts are more rewarding than spinach. There are subliminal factors that play into our love for donuts, and they come from everything around us. Not to mention, we form positive associations with donuts more frequently than we do with spinach, and reward value increases with repetition.

Over time, these associations can become habits as well. We can mindlessly associate eating donuts with a good time and equate eating donuts as feeling good. This leads to mindless consummatory behavior, which can spiral out of control.

Consummatory behavior on its own is natural, but when it becomes mindless it starts to become dangerous.

So how do we stop automatically consuming things?

Some people say “just use willpower” but that doesn’t work in the long term. I’m sure most of us know this from experience. Every time I try to change a behavior purely off willpower, I end up going back to my old ways in about two weeks.

To change a behavior, we can’t just focus on the behavior itself. We have to pay attention to how it makes us feel, specifically how rewarding it is. If we could just focus on the behavior, then we could just tell ourselves to stop doing any of our bad habits and we could live happily ever after.

Updating our Reward Value System

We can update our system by adding one simple thing to the situation – our awareness and attention. I talk a fair bit about the importance of attention and awareness in my post The Heroes of Hero’s: The Osiris Myth & Attention. Attention is like our superpower! It gives us the ability to cast out will into the future, but more importantly, we can use it to change what we find rewarding.

The only way we can update our brain systems is if our brain determines that what it already knows is outdated and doesn’t work.

This requires giving it new information.

This new information will come in the form of mindful consumption, as opposed to mindless consumption.

According to Dr. Jud, “paying attention to the results of the behavior in the present, we can accurately determine how reward a behavior actually is rather than just run our old automated reward values.”

Let me give the example of smoking a cigarette. I’m using this example because I used these methods to quit my fairly heavy cigarette habit back in the day.

To the habitual smoker, smoking is the behavior that fixes everything. The smoker’s reward value system places cigarettes at the top, smoking is the ultimate reward. But if that smoker were to practice mindful consumption – paying attention to all of the sensations and feelings we get when smoking – the smoker will find that the cigarette isn’t actually very rewarding at all. The smoker will rediscover that smoking makes it difficult to breathe, the chemicals are strong, there’s tightness in our chest, the smell lingers, it costs money, and so many other new things.

Our brains can now take this new information and use it to update its reward value system and place the cigarette in a more accurate position, probably (and hopefully) somewhere near the bottom.

When we practice mindful consumption, we give our brains a chance to rediscover how rewarding (or unrewarding) something is for us now.

No longer do we have to be chained to our past experiences. Through mindfulness we can create new associations.

When I practiced mindful consumption with cigarette smoking, I was able to see that I had so many other associations with smoking. For example, the satisfaction of my oral fixation, the feelings of acceptance I felt from my peers, and the opportunity to be someone who was cool and rebellious. Maybe smoking was rewarding to me when I was younger, maybe even necessary, but today it’s not so much. Once I internalized this realization, I stopped smoking naturally. I remember the moment when I took a drag and immediately felt this disgusted feeling. I thought “what the hell is this doing for me?” I noticed that it really just smelled like stinky cheese and it made it hard to breathe. I was able to put it down cold turkey with a little craving for it later.

Awareness can reset our reward value system.

We can change bad habits by paying attention.

There have been studies that demonstrate that cravings and habitual consumption lowered by as much as 40% just after practicing mindful consumption as little as 10 times. This means we can change our habits without resorting to using serious willpower and it’s all using the already built-in systems that our brain has.

Understanding our built-in systems and how they work gives us an edge in creating our lives by design. We don’t have to work uphill. Our bodies, our brains, our minds are beautiful inventions that have stood the test of time.

Let’s use its miraculous engineering to supercharge our intentions.

Categories
Education Productivity

Optimizing Environments for Studying and Other Things

“There are nearly endless opportunities to improve each day and finding them largely boils down to being curious. People who are better, in the end, are usually curious in the beginning.” 

James Clear (Author of Atomic Habits)

“Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction, and skillful execution; it represents the wise choice of many alternatives.”

William A. Foster (United States Marine & Medal of Honor Recipient)

A huge part of being socialized is being able to regulate our emotions and there are tons of ways to do this. Exercise, diet, meditation, vices, art, work, etc. But the most effective way to regulate our emotions is to regulate our environment.

The best way to not be stressed is to stay away from stressful places.

The Best Study Environments

If you’ve read my other posts on studying, I frequently emphasize learning principles to design systems that work best for ourselves. One of the fundamental principles for creating the best study environments is known as State-dependent Learning. This is the idea that people can more easily recall information if their physical or mental state is the same as their time of encoding.

This suggests that our physical and mental states are influenced by our environment. For example, when students are in a classroom they may be feeling more alert and aware than at home which influences how they encode and recall.

When I was younger, I noticed that I was much better at math inside my math class than at home or anywhere else. In class, I was sharp as a tack but the second I was anywhere else, it took me more than twice as long to answer the same questions. I didn’t understand it then, but now I know that it was because I learned the material in that classroom, everything about the classroom became subconsciously associated with the material. When I took in all the stimuli from the classroom (i.e. decorations of the class or the arrangement of the chairs) I was able to fire the specific neural pathways faster because of all the cues affecting my internal state. I could still fire the same neural pathway at home, it just took a little more energy and time because I didn’t have all the external cues from the classroom (where I encoded the information) influencing my internal states or acting as anchors for recognition.

Implement Cues

When we’re getting started it’s nice to have a cue that lets us know when it’s time to get to work. Cues are the 1st stage of the habit cycle and are crucial in creating (or destroying) habits.

When I was in high school, I had a small study lamp that would click whenever I turned it on. Since I had so many days where I sat at that desk and did my homework, I created a subconscious connection between the click of the lamp and getting to work. It’s like once the lamp clicked, so did my mind and I was ready to study.

The click of the lamp was my cue to get started.

Now, the cue doesn’t have to be a click of a lamp. It can be anything really, as long as we associate it with studying or getting to work.

In college, I had a study playlist that would get me into full flow once it got going. After college, I created new playlists with different kinds of music. (Today, the Hamilton instrumentals have been keeping me on track). However, cues can be more than auditory. It’s also lighting and position, the decorations in the room, smells., or tastes.

Any and all of our senses can be used to create cues that trigger our habit cycles.

Simulate the Test Enviornment

This is one of the most fundamental principles for creating a study environment. There have been studies that show studying in an environment similar to test helps with remembering. This is, in part, because of the cues and state-dependent learning. While we’re learning, the signals from the room are subconsciously connected to the concepts, and seeing those signals may help trigger the neural pathway we want. This is partly why I suggest that silence is better than music while studying because we usually take tests when it’s silent.

Simulating the test environment gets more important as we get closer to the exam date. Now, I want to emphasize that it’s still possible to study in a completely different environment and perform well on an exam. It would require more effort and active recall, but it still works. Studying in a similar environment as the test simply makes things easier to recall later.

Routines

Location and time matter. Learning in the same place makes it easier to encode and retain information. Recalling that information would be even easier if we tried to remember it in the same place that we learned it.

This is partly why I performed so well in high school. I had a rigid schedule where I learned the same subjects every day at the same time in the same place. I noticed that students had a hard time retaining information during distance learning and part of that was because many of the students took it upon themselves to do their work “whenever they felt like it.”

While I’m a huge proponent of people creating their own schedules, without a routine our brain will have a more difficult time filing that information away.

Let me give an example: if we learn math in the morning at our kitchen table, then we will subconsciously associate the kitchen with that specific math concept. Now, let’s say we learn the next topic at the kitchen table again. Our brain will create a little folder in our minds where we associate learning math at the kitchen table and it will be a litter easier next time to learn math there. But let’s say we decide to do our math assignments in the evening in the living room. We’ll notice that we’re still able to do the lesson, but there will be slight resistance. Our brain will have a harder time recalling the math concepts when we’re in a different location or at a different time. This may not seem like much, but over time we’ll have to put in significantly more energy just to keep it all straight.

Music

So here’s the deal with music and studying. Studying in silence is better for two main reasons:

1) We can dedicate more of our cognitive load to the course material

2) We are simulating the test environment by studying in silence. In almost every exam I’ve taken, it was silent so it helps to have silence associated with the concepts, at least in preparation for the test.

But if I have to be honest, I love studying and working to music.

Music is a huge part of my life and it brings me so much pleasure.

Yes, there is a slight productivity hit from studying to music but I think it’s worth it if it makes the experience a little more enjoyable. I’m all about prioritizing my experience of the process over productivity.

If I’m constantly excited and enjoy the process, then I’m naturally going to be working more often.

So I want to get into which music is best for studying. Typically songs with lyrics tend to distract our minds. Even if we aren’t trying to pay attention to the lyrics, our minds will be subconsciously trying to decode the messages in the songs which takes up a significant amount of cognitive load. So studying while listening to music without lyrics or instrumentals is the best of both worlds.

Additionally, songs with more rhythmic qualities tend to be more distracting than songs with less. So I recommend listening to songs with relatively little drum instrumentation. Unfortunately, this means one of my favorite genres, hip hop, isn’t ideal for studying. I tend to work and study to classical or movie scores.

Listening to music also helps mask feelings of effort, our brains release dopamine when we listen to music which helps with our reward systems. Music has also been proven to have an analgesic effect, so it literally hurts less to study with music on.

Lighting

The best lighting for studying is a well-lit room where everything is easy to see. Low or dim light makes it difficult to see and it also signals to our brain that it’s time to go to bed. In low lighting, we’re extremely susceptible to slipping into stage 1 of sleep. Our brain emits alpha waves and lowers our levels of mental arousal. This prevents us from focusing or paying attention as well as we could.

If lighting the entire room isn’t feasible, I recommend having a study lamp that at least illuminates the workspace. Not only for the lighting but having something that’s associated with just studying helps with creating habit-forming cues.

Desks

Sitting or standing, it doesn’t matter which team you’re on. Naturally, we don’t want to sit for long periods of time, but we also don’t want to study for long periods of time either.

I’ve seen a lot of conflicting stuff when it comes to sitting vs. standing desks, but I recommend splitting up studying into modified Pamadoros and getting up and walking around, changing up the environment a little bit during the breaks. I don’t think the issue lies in which desk we have, but in doing things for long periods of time when we’re designed to switch things up every once in a while.

The desk needs to have adequate space to work. It also should be inviting and functional. When I finish my work, I try to clean up my workspace so it’s nice and clean which encourages me to get to work the next time. When my space is cluttered (which is when I’m in the middle of projects), I’m less likely to work again. I call cleaning my workspace after I finish resetting to zero. I try to do it whenever I can because it helps me hit the ground running the next time.

Libraries vs. Coffee Shops

Which is better for studying? This depends on a few different things:

1) Our goals and where we are in relation to it.

Let’s say we’re learning something new, the deadlines are pretty far away, and we have a study group that wants to meet at a coffee shop. Sure! This would probably work. But let’s say we’re not really strong on the material and the test is coming up in a few days, then we should probably spend some time in the library alone running through active recall drills. For me personally, I used libraries to learn new information and coffee shops to review old information. Where we are and where we want to go play a huge role in how we choose or design our environments.

2) Our internal states.

Renowned neuroscientist, Dr. Andrew Huberman, says our brains are constantly determining if our internal states are aligned with our external surroundings. We can think of the library as quiet and still while the coffee shops are busy and loud. If our internal state is still and quiet, then studying at a library may be our best bet. If we’re feeling a little more stimulated and anxious, maybe studying at a coffee shop would be better for us because we will have that matching of the internal and external.

Discover how we feel and go to the place that matches that.

For some, knowing how we are feeling can be a difficult thing. Mindfulness practices are great for developing the skills and abilities to identify what we’re feeling. Making the best choices for ourselves always starts with knowing what we want and need.

Study Groups vs. Studying Alone

I’ve written a post on Group Studying vs. Solo Studying that goes over the 80/20 on that subject, but bottom line is that each has its advantages and it depends on what we want to accomplish. I recommend studying alone if you are learning the material for the first time or if you’re still shaking on most of the main ideas. Groups can be powerful for reviewing content that we are already familiar with or learning how to approach 1 or 2 main ideas.

Know what’s needed and plan accordingly.

More On Environments

“If you are distressed by anything external, the pain is not due to the thing itself, but to your estimate of it; and this you have the power to revoke at any moment.”

Marcus Aurelius (Emperor of Rome, Stoic Philosopher, & Author of Meditations)

Environmental influence is nontrivial and determines our constraints, but it also goes deeper than what’s physically around us. Our environment has physical, social, and mental components.

Our physical environmental influence consists of all the tangible aspects around us. This includes, but is not limited to, the location, lighting, desk, and music. Changing our physical environment is probably the easiest to change compared to mental and social environments. Spending some time cleaning or reorganizing will usually do the trick.

Our social environments are created by the people involved in our pursuits. The people may be in our physical space or not, they could be involved with the project from afar, or they could even be consumers. The thing to keep in mind is that the social influences of our environment come from our relationships with those people. Tending to the relationships, or adjusting our perception of the relationships is our key to optimizing our social environment.

Our mental environmental influences consist of all the nontangibles and other things floating around in our heads. If we can remember that our environment isn’t just what is around us, but also what is goes on in our minds, then it’s a little easier to determine what is part of our mental environment. Optimizing our mental space is a little more difficult than our physical space. In order to do this, we have to adopt new philosophies and perspectives. I highly recommend Stoic philosophy for this. However, the most influential factor of our mental environment is our perception of the physical space. Our perceptions are within our control and with some practice, we can shift our perceptions so they create an environment better suited for our pursuits.

Our environments influence us and determine our constraints, to an extent, but our perceptions could transform an otherwise destructive environment into something supportive.

“When jarred, unavoidably, by circumstance, revert at once to yourself, and don’t lose the rhythm more than you can help. You’ll have a better group of harmony if you keep on going back to it.”

Marcus Aurelius (Emperor of Rome, Stoic Philosopher, & Author of Meditations)

The key to controlling our environment is finding the strength to accept that our interpretation of our environment is just one of many and to actively seek a framework that transforms the challenges of today into what makes us phenomenal.

There are infinite ways to perceive something and if we’re in a position where our environment seems to be crushing us, then our only way out is to look for other ways to see the situation.

“Everyone thinks of changing the world, but no one thinks of changing himself.”

Leo Tolstoy (1828 – 1910)

Our environments are a combination of what’s going outside of us, but also inside as well. As long as we know ourselves and what we want to accomplish, we have the power to transform our environments into beautiful places, or at least less miserable places.