One thing I’m meticulous about is stats. I try to maximize return to my customers and work hard to ensure my systems are the best possible.
In that regard, one thing I’ve implemented this week – invoice notifications. From here on out, I am notify every customer of how much their payment will be and if they would like to provide feedback (for one month free), they can.
With that in mind, some quick stats:
Of the 40 people who signed up for a trial last month (at $25 – $50 per month), 28 converted to users
Of the 28 converted, 15 provided feedback and were provided the next month free
Of the 12 who didn’t convert to “paying users”, 4 provided feedback and are still participating
That makes a total of 32 people
Those numbers may not sound great to you. However, given I haven’t advertised or otherwise done much of anything besides occassionally mention ProjectPiglet in a comment and let my current users share, that’s pretty exciting. It’s a pretty expensive product for a consumer. What’s getting me more excited, is with the influx of users over the last 6 weeks (~100), my emails stats have stayed consistent.
Last month, we sent 1,718 emails (trade, opinion change, invoice notifications), that’s up from the 767 we sent the prior 30 days.
The good news about that, is although we’ve had an uptake, we’ve maintained the exact same open and click rates between both 30 day segments:
Which I’m reading as, we have a product people want – as those open & click rates are high for the average website. Especially, considering we send an average of 2.2 emails a day for a user with notifications set.
Which brings me to my last point, only 50% (~50 users) are following trends / topics. Of those, if user followed at least five trends / topics, they were 100% likely (thus far) to convert to a full customer. Thus my goal for the next month is to improve the new customer experience to ensure I can improve the following of trends / topics / assets.
In this article: We discuss causality in crypto markets. Specifically, when is a good time to buy / sell a cryptocurrency; this analysis will be conducted from (at the time) live data from ProjectPiglet.com.
Intro to Trends on ProjectPiglet.com
For this article, we will be applying Granger Causality to a few cryptocurrencies for the past few months. We’ll use live trend and pricing data from ProjectPiglet.com, between April 1, 2017 and December 1, 2017.
The goal of ProjectPiglet.com is to provide insights from experts to the masses; bubble up the knowledge, if you will. In the case of crypto-markets, it’s often the reverse, where the masses tell the experts how they feel. Specifically, (crypto)currencies are massively impacted by how confident masses feel. This is part of the reason for the federal reserve in the U.S., as it stabilizes the U.S. Dollar. In contrast, cryptocurrencies have no such system, so a boom-bust cycle is inevitable.
If you’re interested in the specifics regarding the chart, feel free to checkout our explanation. In summary, trend’s are the normalized volume of discussions between 0 and 1, as tracked by ProjectPiglet.com.
Code for Granger Causality
Now, lets get to the fun part – coding!
First, we create a function that takes in trend and pricing data, normalizes the data using zscore, and formats the data for the granger causality function we will be using (from the python package statsmodels):
From there it’s relatively easy to ingest into the Granger Causality function (from the python package statsmodels). The only part we have left to do is defining the “max lag”, or how far back in time should be assessed by the function. In this case, 60 days was chosen as we are only looking at a few months of data. Just make sure it is less than one third of the data’s size (I recommend choosing <10% of the data set size):
With the output from that function it is possible to pull out the F-test and p-value from the “gc” variable (Granger Causality). Both the F-test and p-value are used for determining correlations, however I wont dive into that at the moment. The full documentation on the “grangercausalitytest” function is here, and the full code (in the github, below) will provide further insights.
In addition to the F-test and p-value, the Granger Causality function will provide us with a list of “lags”, which is let us know the offset to look for. For this example, the “lag” is how many days prior a change in trend will impact the price.
With this code it is now possible to run our Granger Causality assessment!
Trend and Price Causality in CryptoCurrencies
Using trend data from ProjectPiglet.com, we will review the following cryptocurrencies from April 1, 2017 to December 1, 2017 for “probable causality” (i.e. Granger Causality) based on trend and price data:
Bitcoin Cash (BTC)
When we run a modified version of code above; we end up with the following chart representing the number of days lag in price after a trend event, for the six cryptocurrencies above:
For reference, a p-value < 0.05 is typically considered the threshold for the correlation, between datasets. In this case, there’s also what we are calling a “Strong Correlation” which is a p-value < 0.01. The strange part is the uniform band of “Strong Correlations” at the bottom.
Upon inspection, it appears that one cryptocurrency in particular is the culprit Bitcoin Cash (BCH):
The large amount of no data leading to a high amount of correlation. This is one of the common pitfalls we mentioned Limits of Granger Causality, and is an excellent example of why you always verify your data sets! Once we remove BCH from our dataset we now have the following cryptocurrencies we are searching for Granger Causality:
Bitcoin Cash (BTC)
We then appear with the following chart representing the number of days lag in price after a trend event:
I’ve personally verified each of the remaining datasets. I believe there is indeed a probable causality (where the trend impacts the price). You’re welcome to do the same with the code / data provided below.
Causality in Crypto Markets
The more specific outputs (i.e. which currencies have causality) from our code are as follows:
Importantly, it appears there is a pattern – i.e. something we can trade off of. Most of the currencies show a strong movement roughly five days after there is a trend event; averaging major cryptomarket movements eleven days after a trend event and with standard deviation of 7 days.
Further, the one outlier Ripple (XRP) is mostly owned by a single entity. Meaning it might make sense there is no correlation between the price of Ripple (XRP) and the number of people discussing Ripple (XRP); where as the other currencies are mostly owned by various groups / the masses.
Verification of Granger Causality
For safe measure, I typically will do an additional assessments to verify theories and confirm Granger Causality is working. Searching for correlations between trends for each of the cryptocurrencies and a “random walk” (i.e. random data), which outputted the following chart:
For comparison purposes, it’s clear there is no pattern at all; where as the live data has a clear pattern we can follow. In addition, it’s highly recommended to review the output of the individual timeseries datasets being compared. It’s extremely common to get results such as we did with Bitcoin Cash (BCH).
Our goal in sharing our analysis using Granger Causality was two fold:
Help explain Granger Correlations and how to use them
Intice you to try ProjectPiglet.com! (Use coupon code: pigletblog2018 – 25% off for 12 months)
In this article: We discuss Granger Causality works and some of the common issues, drawbacks, and potential ways to improve the method(s). Building primarily off our previous article Pitfalls of Backtesting and insights gained from building ProjectPiglet.com.
One of the most common forms of analysis on the stock market is Granger Causality, which is a method for indicating one signal possibly causes another signal. This type of causality is often called “predictive causality”, as it does not for certain determine causality – it simply determines correlations at various time intervals.
In other words, it’s popular and Clive Granger won a Nobel on the matter. That being said, there are quite a few limitations. In this article, we’ll be covering a brief example of Granger Causality, as well as some of the common pitfalls and how brittle it can be.
What is Granger Causality?
Granger Causality (from Wikipedia) is defined as:
A time series X is said to Granger-cause Y if it can be shown, usually through a series of t-tests and F-tests on lagged values of X (and with lagged values of Y also included), that those X values provide statistically significant information about future values of Y.
In other words, Granger Causality is the analysis of trying to find out if one signal impacts another signal (such that it’s statistically significant). Pretty straightforward, and is even clearer with an image:
In a sense, it’s just one spike in a graph causing another spike at a later time. The real challenge with this is that this needs to be consistent. It has to repeatedly do this over the source of the entire dataset. This brings us to the next part: one of the fragile aspects of this method is that it often doesn’t account for seasonality.
Granger Causality and Seasonality
One common aspect of markets is that they are seasonal. Commodities (as it relates to the futures market) related to food are extremely impacted by seasonality. For instance, if there is a drought across Illinois and Indiana during the summer (killing the corn crop), then corn prices from Iowa would likely rise (i.e. the corn from Iowa would be worth more).
In the example, there may be decades where some pattern in the market holds and Granger Causality is relevant. For instance, during summer heat waves in Illinois, corn prices in Iowa increase. On the other hand, with the advent of irrigation methods that deliver water underground, heat waves may no longer impact crops. Thus, the causality of heat waves in Illinois may no longer impact the corn prices in Iowa.
If we then attempt to search for Granger Causality on the entire time range (a) pre-irrigation and (b) post irrigation, we will find there is no causality!
However, during the pre-irrigation time range we will find probable causality, and for post-irrigation time range we likely won’t find probable causality. Any time you combine two timeframes like this, the default is no Granger Causality (unless it’s a very small portion of the dataset). Bringing us to the conclusion, that:
Granger Causality is very sensitive to timeframe(s)
Just a few data points in either direction can break the analysis. This makes sense, as it is a way to evaluate if two time series are related. However, it does lead one to note how brittle this method can be.
Granger Causality and Sparse Datasets
Yet another potential issue with Granger Causality is sparse datasets. Let’s say we have dataset X and dataset Y: if dataset X has 200 data points and data set Y as 150 data points, how do you merge them? Assuming they are in (datetime, value) format, if we do an inner join on “datetime”, we get something that looks like the following:
Then we will have 150 data points in a combined X and Y dataset, i.e.: (datetime, x, y). Unforunately, this also means if the data is continuous (as most timeseries data is), then we have completely broke our Granger Causality analysis. In other words, we are just skipping over days, which would break any causality analysis.
In contrast, we could do an outer join:
We will have 200 data points in a combined X and Y dataset. Again, there’s an issue – it means we probably have empty values (Null, NULL, None, NaN, etc. ) where the Y data set didn’t have data (recall Y only had 150 data points). The dataset would then have various entries that look as such: (datetime, x, NULL).
To fix the empty values, we can attempt to use a forward or back fill technique. A forward/back fill technique is where you fill all the empty values with the previous or following location(s) real value.
This code could look like the following:
From the sound of it, this method sounds promising! You’ll end up with something that’s continuous with all real values. You’ll actually get a graph like this:
As you can see, there are large sections of time where the data is flat. Recall the seasonality issue with Granger Causality? This method of outer joins + forward / back filling will definitely cause issues, and lead to minimal to no meaningful correlations.
Sparse datasets make it very difficult (or impossible) to identify probable causality.
Granger Causality and Resampling
There is another option for us, and that is “resampling”. Where instead of just filling the empty values (Nulls / NaNs) with the previous or following real values, we actually resample the whole series. Resampling is a technique where we fill the holes in the data with what amounts to a guess of what we think the data could be.
Although there are quite a few techniques, in this example we’ll use the python package Scipy, with the Signal module.
At first glance, this appears to have solved some of the issues:
However, in reality it does not work; especially if the dataset starts or ends with NaN’s (at least when using the Scipy package):
If you notice, prior to the ~110 data point, the values are just oscillating up and down. The resampling method Scipy is using does not appear to be functional / practical with so few data points. This is because I selected data set for Bitcoin Cash (BCH) and the date range is prior to Bitcoin Cash (BCH) becoming a currency (i.e. there is no price information).
In a sense, this indicates it’s not possible (at least given the data provided) to attempt Granger Causality on the given date ranges. Small gaps in time can have dramatic impacts on whether or not “probable causality” exists.
When determining Granger Causaily it is extremely important to have two complete overlapping datasets.
Without two complete datasets, it’s impossible to identify whether or not there are correlations over various time ranges.
Resampling can cause artifacts that impact the Granger Causality method(s).
In fact, the most recent example was actually positive for Granger Causality (p-value < 0.05)… That is the worst scenario, as it is a false positive. In the example, the false positive occurs because when both datasets are resampled they had a matching oscillation… it wouldn’t have even been noticed if the raw data sets weren’t being reviewed.
This is probably the largest issue with Granger Causality: every dataset needs to be reviewed to see if it makes sense. Sometimes what at first appears to make sense, in reality the underlying data has been altered in some way (such as resampling).
Granger Causality and Non-Linear Regression
Changing gears a bit (before we get to a real-world ProjectPiglet.com example), it’s important to note that most Granger Causality uses linear regression. In other words, the method is searching for linear correlations between datasets:
However, in many cases – especially in the case of markets – correlations are highly likely to be non-linear. This is because markets are anti-inductive. In other words, every pattern discovered in a market creates a new pattern as people exploit that inefficiency. This is called the Efficient Market Hypothesis.
Ultimately, this means most implementations of Granger Causality are overly simplistic; as most correlations are certainly non-linear in nature. There are a large number of non-linear regression models, below is an example of Gaussian Process Regression:
Similar, non-linear regression techniques do appear to improve Granger Causality. This is probably due to most linear correlations already being priced into the market and the non-linear correlations will be where the potential profits are. It remains to be seen how effective this can be, as most research in this area is kept private (increasing profits of trading firms). What we can say is that non-linear methods do improve predictions on ProjectPiglet.com. They also require a larger dataset than their linear regression counterparts.
Overall, Granger Causality has quite a few potential pitfalls. It is useful for indicating a potential correlation, but is only a probable correlation. It can help to identify market inefficiencies and open the opportunity to make money, but will probably require more finesse than simple linear regression.
All that being said, hope you’ve found some of the insights useful!
One thing considered absolutely essential when developing trading strategies or models is backtesting. Backtesting is the term used to test trading strategies/models on historic data, and can also be applied to other fields such as oceanography or meteorology.
I am a cynic at heart. So, typically I evaluate issues with a method prior to starting an implementation. In backtesting, there are a lot of issues. Unforunately, it’s also the best we have.
In this article, we might break hearts, but we will explain some methods for mitigating issues.
Regardless of what name it goes by (Multiple Comparison Problem or Data-Mining Bias), there is an issue with searching historic data for predictors. The more we think we find, the higher likelihood we are just seeing something erroneous.
Problematic Backtesting Example – Technical Analysis
It’s the same reason why “Technical Analysis” of stock trends are pretty questionable. The lines drawn on a chart, are essentially just making things up (might be right some of the time, but we don’t know). Here’s an exacmple, from stockcharts.com:
These kinds of “Technical Analysis” methods originally developed in the 1920’s and 1930’s. All evaluations had to be done by hand, and they are prone to hindsight bias in their stock selection. Personally, I highly doubt it’s effectiveness overall. The real-life research into the field of “Technical Analysis” has had mixed results and seems to be fairly split on the methodology and outcomes.
The Efficient-Market Hypothesis
You are probably asking yourself, what’s the alternative!? I watnt to prove my trading strategy, financial engine or AI financial advisor works!
Unfortunately – there really isn’t a better way to test any sort of model or strategy.
Stocks always trade at their fair value, making it impossible for investors to either purchase undervalued stocks or sell stocks for inflated prices. As such, it should be impossible to outperform the overall market through expert stock selection or market timing, and that the only way an investor can possibly obtain higher returns is by chance or by purchasing riskier investments.
The truth is, we know it’s possible to gain some advantage; if you have more information or can execute a trade faster. For instance, you can gather information through ProjectPiglet.com as we track, in real-time, experts, insiders, and the public. That’s really the key to ProjectPiglet.com‘s success, although you might not make money on every trade, we improve your chances by providing more information (until we saturate the market).
Which brings us to the next issue related to backtesting…
Today, have more information now about any historic event or price than we did at the time. This is one of the causes of the Data-Mining Bias, where we use data that we didn’t have or didn’t even exist.
More important, historic stock prices were set using prior information and prior algorithms and strategies. Meaning, the strategy being backtested may have already been discovered (at a date after our backtesting data)! There is no way we would know, and it would effectively make the strategy or algorithm in question obsolete.
For reference, this has been dubed anti-inductive – meaning as soon as a pattern is discovered the market creates a more complex pattern, correcting for said pattern. Personally, I like to call this market entropy.
Mitigating the Issues
Although there issues with backtesting, there are ways to mitigate the issue(s).
One such mitigation is called a Bonferroni Correction, and it is used to mitigate the Multiple Comparisons Problem. The purpose of the Bonferroni Correction is to reduce the likelihood of rejecting the null hypothesis prematurely (due to the increased number of tests). In other words, it’s a way to reduce the likelihood your strategy isn’t just getting lucky.
In addition, there are also methods for mitigating the Efficient-Market Hypothesis (although honestly, to a minimal extent). This can be accomplished identifying / indicating causality between two timeseries events, such as Granger Causality:
This is useful for indicating there is a predictable event, and is useful over various time ranges to ensure the inefficiency in the market is still present. An interesting application of this method is also discovering when the market identifies some pattern. Granger Causality works best on linear regression, and is difficult to use for the nonlinear case (AKA it’s most useful for basic causality comparisons).
Backtesting is difficult, error prone, and will not gurantee success; however it is the best we have. Generally, it’s the only way to test if a strategy has any merit, but any insights should be reviewed carefully.
Over the next several articles we will explain our methods for mitigating the issues, as well as share our results. To-date our results typically beat or match what’s called “buy and hold strategies”, but we bias towards reducing risk, as opposed to maximizing potential profit; another whole domain.