Sentimental Analysis

Earlier, we said that price should theoretically accurately reflect all available market information. Unfortunately for us traders, it isn't that simple. The markets do not simply reflect all the information out there because traders will all just act the same way. Of course, that isn't how things work.

Each trader has his own opinion or explanation of why the market is acting the way they do. The market is just like Facebook - it's a complex network made up of individuals who want to spam our news feeds.

Kidding aside, the market basically represents what all traders - you, Pipcrawler, Celine from the donut shop - feel about the market. Each trader's thoughts and opinions, which are expressed through whatever position they take, helps form the overall sentiment of the market.

The problem is that as traders, no matter how strongly you feel about a certain trade, you can't move the markets in your favor (unless you're one of the GSs - George Soros or Goldman Sachs!). Even if you truly believe that the dollar is going to go up, but everyone else is bearish on it, there's nothing much you can do about it.

How to Develop a Sentiment-Based Approach

As a trader, it is your job to gauge what the market is feeling. Are the indicators pointing towards bullish conditions? Are traders bearish on the economy? We can't tell the market what we think it should do. But what we can do is react in response to what is happening in the markets.
Note that using the market sentiment approach doesn't give a precise entry and exit for each trade. But don't despair! Having a sentiment-based approach can help you decide whether you should go with the flow or not. Of course, you can always combine market sentiment analysis with technical and fundamental analysis to come up with better trade ideas.
In stocks and options, traders can look at volume traded as an indicator of sentiment. If a stock price has been rising, but volume is declining, it may signal that the market is overbought. Or if a declining stock suddenly reversed on high volume, it means the market sentiment may have changed from bearish to bullish.

 Unfortunately, since the foreign exchange market is traded over-the-counter, it doesn't have a centralized market. This means that the volume of each currency traded cannot be easily measured.

 Commitment of Traders Report

The COT Report: What, Where, When, Why, and How

The Commodity Futures Trading Commission, or CFTC, publishes the Commitment of Traders report (COT) every Friday, around 2:30 pm EST.
Because the COT measures the net long and short positions taken by speculative traders and commercial traders, it is a great resource to gauge how heavily these market players are positioned in the market.

Later on, we'll let you meet these market players. These are the hedgers, large speculators, and retail traders. Just like players in a team sport, each group has its unique characteristics and roles. By watching the behavior of these players, you'll be able to foresee incoming changes in market sentiment.
You're probably asking yourself, "Why the heck do I need to use data from the FX futures market?"
"Doesn't the spot FX market have a report that measures how currency traders are positioned?"
"I'm a spot FX trader! Activity in the futures market doesn't involve me."
Remember, since spot FX is traded over-the-counter (OTC), transactions do not pass through a centralized exchange like the Chicago Mercantile Exchange.
So what's the closest thing we can get our hands on to see the state of the market and how the big players are moving their money?
Yep, you got it...
The Commitment of Traders report from the futures market.
Before going into using the Commitment of Traders report in your trading strategy, you have to first know WHERE to go to get the COT report and HOW to read it.

 3 Simple Steps to Access the COT Report

Step 1:

Open up the address below in your web browser. (http://www.cftc.gov/marketreports/commitmentsoftraders/index.htm)

Step 2:

Once the page has loaded, scroll down a couple of pages to the "Current Legacy Report" and click on "Short Format" under "Futures Only" on the "Chicago Mercantile Exchange" row to access the most recent COT report.

Step 3:

It may seem a little intimidating at first because it looks like a big giant gobbled-up block of text but with a little bit of effort, you can find exactly what you're looking for. Just press CTRL+F (or whatever the find function is of your browser) and type in the currency you want to find.
To find the British Pound Sterling, or GBP, for example, just search up "Pound Sterling" and you'll be taken directly to a section that looks something like this:

Yowza! What the heck is this?! Don't worry. We'll explain each category below.

• Commercial: These are the big businesses that use currency futures to hedge and protect themselves from too much exchange rate fluctuation.
• Non-Commercial: This is a mixture of individual traders, hedge funds, and financial institutions. For the most part, these are traders who looking to trade for speculative gains. In other words, these are traders just like you who are in it for the Benjamins!
• Long: That's the number of long contracts reported to the CFTC.

• Short: That's the number of short contracts reported to the CFTC.
• Open interest: This column represents the number of contracts out there that have not been exercised or delivered.
• Number of traders: This is the total number of traders who are required to report positions to the CFTC.
• Reportable positions: the number of options and futures positions that are required to be reported according to CFTC regulations.
• Non-reportable positions: open interest positions of traders that do not meet the reportable requirements of the CFTC like retail traders.

If you want to access all available historical data, you can view it here.
You can see a lot of things in the report but you don't have to memorize all of it.
As a budding trader, you'll only be focusing on answering the basic question:
"Wat da dilly on da market yo?!"
Translation: "What's the market feeling this week?"

Understanding the Three Groups

In order to understand the futures market, first you need to know the people making the shots and those who are warming up the bench. These players could be categorized into three basic groups:

1. Commercial traders (Hedgers)
2. Non-commercial traders (Large Speculators)
3. Retail traders (Small Speculators)

Don't Skip the Commercial - The Hedgers

Hedgers or commercial traders are those who want to protect themselves against unexpected price movements. Agricultural producers or farmers who want to hedge (minimize) their risk in changing commodity prices are part of this group.
Banks or corporations who are looking to protect themselves against sudden price changes in currencies or other assets are also considered commercial traders.
A key characteristic of hedgers is that they are most bullish at market bottoms and most bearish at market tops.
What the hedgehog does this mean?

Here's a real life example to illustrate:
There is a virus outbreak in the U.S. that turns people into zombies. Zombies run amok doing malicious things like grabbing strangers' iPhones to download fart apps.
It's total mayhem as people become disoriented and helpless without their beloved iPhones. This must be stopped now before the nation crumbles into oblivion!
Guns and bullets apparently don't work on the zombies. The only way to exterminate them is by chopping their heads off.
Apple sees a "market need" and decides to build a private Samurai army to protect vulnerable iPhone users.
It needs to import samurai swords from Japan. Steve Jobs contacts a Japanese samurai swordsmith who demands to be paid in Japanese yen when he finishes the swords after three months.
Apple also knows that, if the USD/JPY falls, it will end up paying more yen for the swords.
In order to protect itself, or rather, hedge against currency risk, the firm buys JPY futures.
If USD/JPY falls after three months, the firm's gain on the futures contract would offset the increased cost on its transaction with the Japanese sword smith.
On the other hand, if USD/JPY rises after three months, the firm's loss on the futures contract would be offset by the decrease in cost of its payment for the samurai swords.

In It to Win It - The Large Speculators

In contrast to hedgers, who are not interested in making profits from trading activities, speculators are in it for the money and have no interest in owning the underlying asset!
Many speculators are known as hardcore trend followers since they buy when the market is on an uptrend and sell when the market is on a downtrend. They keep adding to their position until the price movement reverses.
Large speculators are also big players in the futures market since they hold huge accounts.
As a result, their trading activities can cause the market to move dramatically. They usually follow moving averages and hold their positions until the trend changes.

Cannon fodders - The Small Speculators

Small speculators, on the other hand, own smaller retail accounts. These comprise of hedge funds and individual traders.
They are known to be anti-trend and are usually on the wrong side of the market. Because of that, they are typically less successful than hedgers and commercial traders.
However, when they do follow the trend, they tend to be highly concentrated at market tops or bottoms.

The COT Trading Strategy

Since the COT comes out weekly, its usefulness as a market sentiment indicator would be more suitable for longer-term trades.
The question you may be asking now is this:
How the heck do you turn all that "big giant gobbled-up block of text" into a sentiment-based indicator that will help you grab some pips?!
One way to use the COT report in your trading is to find extreme net long or net short positions.
Finding these positions may signal that a market reversal is just around the corner because if everyone is long a currency, who is left to buy?
No one.
And if everyone is short a currency, who is left to sell?
What's that?
Pretty quiet...
Yeah, that's right. NO ONE.
One analogy to keep in mind is to imagine driving down a road and hitting a dead end. What happens if you hit that dead end? You can't keep going since there's no more road ahead. The only thing to do is to turn back.
Let's take a look at this chart of the EUR/USD from TimingCharts:

On the top half, we've got the price action of EUR/USD going on. At the same time, on the bottom half, we've got data on the long and short positions of EUR futures, divided into three categories:

• Commercial traders (blue)
• Large Non-commercial (green)
• Small non-commercial (red)

Ignore the commercial positions for now, since those are mainly for hedging while small retail traders aren't relevant.
Let's take a look at what happened mid-way through 2008. As you can see, EUR/USD made a steady decline from July to September. As the value of the net short positions of non-commercial traders (the green line) dropped, so did EUR/USD. In the middle of September, net short positions hit an extreme of 45,650. Soon after, investors started to buy back EUR futures. Meanwhile, EUR/USD rose sharply from about 1.2400 to a high near 1.4700!

 

Over the next year, the net value of EUR futures position gradually turned positive. As expected, EUR/USD eventually followed suit, even hitting a new high around 1.5100. In early October 2009, EUR futures net long positions hit an extreme of 51,000 before reversing. Shortly after, EUR/USD began to decline as well.
Holy Guacamole! Just by using the COT as an indicator, you could have caught two crazy moves from October 2008 to January 2009 and November 2009 to March 2010.
The first was in mid-September 2009. If you had seen that speculative traders' short positions were at extreme levels, you could have bought EUR/USD at around 1.2300. This would have resulted in almost a 2,000-pip gain in a matter of a few months!
Now, if you had also seen that net long positions were at an extreme in November 2009, you would have had sold EUR/USD and you could have grabbed about 1,500 pips!
With those two moves, using just the COT report as a market sentiment reversal indicator, you could have grabbed a total of 3,500 pips. Pretty nifty, eh?

 Picking Tops and Bottoms

As you would've guessed, ideal places to go long and short are those times when sentiment is at an extreme.
If you noticed from the previous example, the speculators (green line) and commercials (blue line) gave opposite signals. While hedgers buy when the market is bottoming, speculators sell as the price moves down.
Here's that chart again:

Hedgers are bearish when the market moves to the top while speculators are bullish when the price is climbing.
As a result, speculative positioning indicates trend direction while commercial positioning could signal reversals.
If hedgers keep increasing their long positions while speculators increase their short positions, a market bottom could be in sight.
If hedgers keep adding more short positions while speculators keep adding more long positions, a market top could occur.
Of course, it's difficult to determine the exact point where a sentiment extreme will occur so it might be best to do nothing until signs of an actual reversal are seen. 

We could say that speculators, because they follow the trend, catch most of the move BUT are wrong on turning points.
Commercial traders, on the other hand, miss most of the trend EXCEPT when price reverses.
Until a sentiment extreme occurs, it would be best to go with the speculators.
The basic rule is this: every market top or bottom is accompanied by a sentiment extreme, but not every sentiment extreme results in a market top or bottom.

Your Very Own COT Indicator

Having your very own COT indicator is like having your own pony.
Using the COT report can be quite useful as a tool in spotting potential reversals in the market.
There's one problem though, we cannot simply look at the absolute figures printed on the COT report and say, "Aha, it looks like the market has hit an extreme... I will short and buy myself 10,000,000 pairs of socks."
Determining extremes can be difficult because the net long and short positions are not all relevant. What may have been an extreme level five years ago may no longer be an extreme level this year. How do you deal with this problem?
What you want to do is create an index that will help you gauge whether the markets are at extreme levels. Below is a step-by-step process on how to create this index.

1. Decide how long of a period we want to cover. The more values we input into the index, the less sentiment extreme signals we will receive, but the more reliable it will be. Having less input values will result in more signals, although it might lead to more false positives.
2. Calculate the difference between the positions of large speculators and commercial traders for each week.

The formula for calculating this difference is:

Difference = Net position of Large Speculators - Net position of Commercials

Take note that if large speculators are extremely long, this would imply that commercial traders are extremely short. This would result in a positive figure.

On the other hand, if large speculators are extremely short, that would mean that commercial traders are extremely long and this would result in a negative figure.

3. Rank these results in ascending order, from most negative to most positive.
4. Assign a value of 100 to the largest number and 0 to the smallest figure.

And now we have a COT indicator! This is very similar to the RSI and stochastic indicators that we've discussed in earlier lessons. 

Once we have assigned values to each of the calculated differences, we should be alerted whenever new data inputted into the index shows an extreme - 0 or 100. This would indicate that the difference between the positions of the two groups is largest, and that a reversal may be imminent.
Remember, we are interested in knowing whether the trend is going to continue or if it is going to end. If the COT report reveals that the markets are at extreme levels, it would help pinpoint those tops and bottoms that we all love so much.

 Getting Down and Dirty with the Numbers

Now that we know how to determine sentiment extremes, what's next? Recall that not every sentiment extreme results in a market top or bottom so we'll need a more accurate indicator. Calculating the percentage of speculative positions that are long or short would be a better gauge to see whether the market is topping or bottoming out.
The equation to calculate for the %-long and %-short is indicated below:

To illustrate this better, let's go back a few years and see what happened with Canadian dollar futures.
Going through the COT reports released on the week ending August 22, 2008, speculators were net short 28,085 contracts. On March 20, 2009, they were net short 23,950 contracts. From this information alone, you would say that there is a higher probability of a market bottom in August since there were more speculators that were short in that period.
But hold on a minute there... You didn't think it would be THAT easy right?

A closer look would show that 66,726 contracts were short while 38,641 contracts were long. Out of all the speculative positions in April (66,726 / (38,641 + 66,726)), 63.3% were short positions.

On the other hand, there were just 5,203 long contracts and 25,875 short contracts in March. This means that (25,875 / (5,203 + 25,875)) 83.3% of the speculative positions were short positions during that period.
What does this mean? There is a higher chance that a bottom will occur when 83.3% of all speculative positions are short as opposed to just 63.3%.
As you can see on the chart below, the bottom in fact did not occur around August 2008, when the Canadian dollar was worth roughly around 94 U.S. cents. The Canadian dollar continued to fall over the next few months. By the time March came around and the %-short ratio hit 83.3%, the Canadian dollar had hit a bottom around 77 U.S. cents. Then what happened? It started to steadily rise! A market bottom? Yep, you got it.