SEE ALSO POSTS 7 AND 12
Preliminary Remark about Pareto and Log-normal distributions.
It is custom in the economist to model the financial inequalities with the Pareto distribution (see e.g. https://en.wikipedia.org/wiki/Pareto_distribution ) which is essentially a polynomial function. The exact model of the inequalities is even worse and is closer to the log-normal distribution (see e.g. https://en.wikipedia.org/wiki/Log-normal_distribution ) where the severity of the inequalities is modeled with exponential functions.But here in this book we may keep talking about the Pareto distribution which is celebrated term among the economists
There is a thin line that separates business and investments as gambling that destroys the human spirit from business and investments as applications of scientific statistical knowledge under general principles that protects and reinforces the human spirit. This book contributes to see the difference and put the investors from the side of protected human spirit.
VALID STATISTICS AND PREDICTABILITY
We must make here some remarks about the robust application of statistical predictions in the capital markets.
1) The theory that the efficient markets and in particular that they follow a pure random walk is easy to refute with better statistical experiments and hypotheses tests. The random walk would fit to a market where the sizes of the economic organizations are uniformly random. But the reality is that they follow a Pareto or power distribution, therefore this is inherited in the distribution of the volumes of transactions and also in the emerging trends or drifts.
2) The statistical models of time series are more robust , when they apply to the entity MARKET as a whole and are better as non-parametric , and not when they apply to single stocks and are linear or parametric. The reasons is that a time series as a stochastic process , requires data of a sample of paths, and for a single stock is available only a single path. While for all the market the path of each stock or security is considered one path from the sample of all paths of all the stocks. Linear time series models or derived like ARMA, ARIMA, SARIMA etc are destined to fail for particular patterns like those described in the post 32, because the true equations are non-linear and in addition with random, time varying coefficients that derive the random emergence of the 4 basic observable patterns (see post 32 ). In addition the standard application of the time series by the researchers, focuses on stationary time series after they extract a stable exponential trend, while in the reality the main concern should be the random path of the average value of the prices that shapes the patterns and is neither constant exponential trend neither zero ! The "statistical momentum conservation" (see post 10) might then be nothing else than an hypothesis that the random and time varying 1st order in time steps , partial correlation of the prices , is always positive. This can be easily tested statistically. E.g. in the cross exchange rate EURUSD but also in the indexes, the partial correlation of the current to the previous time step bar is measured indeed positive, in almost all time frames, except at the daily time frame, where the cyclic behavior prevails. In the daily time frame the partial correlation is negative , which means if one day is up the next day it is more probable that it is down. In addition, the cyclic behavior is even stronger in pairs of two days with negative partial correlation (two days up two days down etc)
4) Multivariate statistics, like factor analysis, discriminant analysis , logistic regression, cluster analysis , goal programming etc are possible to utilize for a more detailed theory of predictability and of portfolio analysis, and sector analysis of the market and not only H. Markowitz theory.
5) In applying of the above applications of statistics, the researcher must have at first a very good "feeling" of the data, and should verify rather with statistics the result rather than discover it.
6) The "Pareto rule of complexity-results" also holds here. In other words with less than 20% of the complexity of the calculations is derived more than 80% of the deduction. The rest of the 20% requires more than 80% more complexity in the calculations.
The less hypothesis we use in applying statistical hypotheses, the better. That is why non-parametric statistics is better. An exception is our knowledge of the application of the Pareto distribution in various aspects of the market which we is parametric.
That is why we avoid applying very complicated with many hypotheses and time consuming to estimate models to forecast the markets, but we prefer to respond to the market, by measuring only in a valid statistical way, the average position of the price, and the channel around it, the velocity (trend, 1st derivative) and acceleration-deceleration (2nd derivative) of the prices.
The statistical quantities from the front-office in trading need to me measured are
1) the price position in the channel around the average, 2) the velocity (1st derivative) and
3) the acceleration-deceleration (2nd derivative), which is done as statistical quantities by a hypothesis test or confidence interval.
4) The support-resistance levels can be measured also by action-volume histograms. The measurements are done with convenient indicators, and can also define in a statistically valid way, not only , the channels , the trend, reversal, and
5) (Eliot or zig-zag) waves but also
6) the spikes.
7) It is required also an in advance in the past measurement and discovery of the basic stable cycles in the markets (see post 5)
8) An in advanced in the past measurement and discovery that trends duration and length, and volumes follow the Pareto distribution (see post 11,25 etc).
In addition for the back-office of trading we need to measure the
9) probability of success of trade based on the past history of trades, to apply the Kelly criterion, and also
10) the average rate of increase of the trading funds and
11) its variance again from the past history of trading.
THE OVER ALL STATISTICAL BEHAVIOR OF PRICE PATTERNS
We notice that although the price patterns are essentially 4 categories in details there are 6 distinct statistical patterns of the random or statistical position, velocity and acceleration of the prices (see post 32).
A class of stochastic processes can be defined as the behavior of the markets based on these 6 basic patterns P1, P2, P3, P4, P5, P5, P7. The Pi i=1,2,3,4,5,6,7 are essentially random statistical patterns with random and variable parameters of size duration, and relative analogies that define them. E.g. we may have a Pareto distribution of the duration and price height of the patterns because of the inequalities in the markets. The 7th pattern P7 of stationary behavior we may call intermittency pattern. We may then assume a class of Markov processes with random and variable transition probabilities, where each random type of pattern occurs and then a next one occurs. Bu the transition probabilities are not arbitrary! E.g. spikes occur usually at the begging (initial spikes) and the end (terminal spikes) of trends, up and down trends with stationary channels in between them, shape cycles, that in the average of stable period, and some times of fixed beginning and end.
The probability that a type of pattern occurs changes also according to the time scale. In 2, 5 or more years annual bars time frames, the non-waving trend pattern is dominating, while say in 5-minutes bars time-frame flat patterns are dominating.
We notice that, by utilizing only pattern P3, of non-waving trend, and intermittency P7 we may derive all, other patterns with appropriate patterns of transition probabilities of the P3! I have coded a simulator of such a class of stochastic processes, superimposed on many time frames called Multi-time scales Rainbow Walks stochastic processes.
This is the overall behavior of patterns of the markets, and there are some invariant properties like
1) Cyclic behavior as alternation with up and down trend patterns with flat channels at the bottoms and tops
2) Statistical momentum conservation (see post 10) where the 1st time-step partial correlation of a price is almost always positive,
3) A Pareto distribution of the duration and height of the patterns, due to the inequalities of the enterprises in the economic system (see post 10, 25, 57,63 )
There are many who complain that the indicators have lag, and prefer not to use indicators at all, but only the prices. This is rather stupid! The indicators when are measuring a statistical quantity MUST have lag, because we are not interested for the price at the now only, but in a short-term past horizon too, which defines the statistical momentum which is statically conserved. The science of statistics is the best fr the moment one can have , from the collective scientific thinking, and we must trust , respect it and be confident and humble when applying it. When we are applying the statistical mode of thinking for the markets, we never run serious dangers of being "burned"and "busted" in our deductions, as statistics claims everything only up to some probability, or probability inequality and interval.
Most of the traders think that intraday manual trading is radically more profitable than day-to-day trading where only once per day (e.g. for 15 minutes) a control and a decision is taken. They think so because they estimate that profits will increase in direct analogy to the shorter time scale they will use. But it is not so at least for two reasons. a) The shorter the scale the more the “noise” (=non profitable and non-tradable fluctuations of the markets due to unpredictability because of c) below) b) In intraday manual trading, one has to spend many hours in front of the screen as if he was working at office while in day-to-day trading only 15-20 minutes per day, while he may have a normal work and normal day with other non-trading activities. c) Intraday and short time scales waves and patterns depend on a small number of people (mainly some packets of transactions by employees of the big banks) and therefore are subject to unpredictable changes, of the actions of these employees. But long term waves depend on a very larger volumes , and global populations involved in the economy, therefore are more stable. From the psychological point of view, if someone believes in the natural guessing abilities of every one , not though the observable patterns of the charts, but through consciousness telepathy, then it is obvious that this small "window" of guessing which is not based on the observable patterns of the charts, works better for the larger scales where large populations are involved rather than to small intra-day scales where only a few people and in particular staff of the largest banks are involved. This is one more argument in favor of the better predictability of the larger time scales. Assuming that the ability of the small "window" of guessing through consciousness telepathy is inversely proportional to the probability of the size of the population that determines the movement of the prices, we get the Pareto diagram of the unpredictability among the scales
It is also important to realize, that this small "window"of guessing, based also on the news, is easy on the index funds as they represent, the overall feeling of the real business, of the economic system, and by far more difficult for individual enterprises and business. The trend of an index fund is macroscopic and artificial, as whenever an enterprise ceases to be growing according to its balance sheets, then it is substituted in the index by a new growing enterprise. Strictly speaking the tend of the index fund is the surplus demand in the society to put money in the big and larger business that are represented with companies in the stock exchanges compared investing to smaller business outside the stock exchanges. It is like a law of universal attraction where big money attracts more money. Strictly speaking again such a an excess demand for big and larger business is not the best in society, as it increases the inequalities, while it would be more desirable investing in small and medium size business usually outside the stock markets. But from the point of view of psychology the duality of the psychology in investing in index funds is simple and asymmetric. When betting for up it is betting on hopes, when betting for down it is betting on fears.
Similarly, for the same reason, using this factor of predictability, is by far more difficult, for the commodities compared again with the index funds. It is rare that there is a clear situation of demand-supply interplay, of a particular commodity, that can be found quantitatively in the news, and be early enough to apply in trading. Most often there are no news useful, and there is no clear expected trend for a commodity. Cross-rates of foreign exchange currencies are a little bit less difficult compared to other commodities. Currencies f go up or down compared to the totality of the rest following the growth of the underlying national economy. But cross-rates simply record their relative differences. Of course larger economies have a lower growth rate compared to smaller economies (exactly as in biological organisms, see end of the current post ) and the distribution of rates relative to sizes is a power low. But again these cross-rate currencies with no strong expected trend, compared to index funds.
In the same rule we may be led even without assuming "intuitive guessing" abilities. It is known that the statistical distribution of the sizes enterprises follows the Pareto distribution, and so does the distribution of the size of packets of orders inherited from the enterprises, therefore of the volumes too. And the same we may assume for the size of populations of demand-supply that create the price patterns in the markets. Therefore that "inertia" which is due to the volumes ,of the movements of price patterns follows also the Pareto distribution. But this "inertia" due to the volumes, is the basis of the "statistical momentum conservation", and also the basis of predictability of the moves of the markets. So large scale and large volume patterns and small scale and small volumes patterns predictability is like the predictability of the moves of the planets compared to the predictability of the moves say of asteroids that also collide more often between them. Obviously the latter is more difficult to predict, even though the same physical Newton's law of momentum conservation holds. Now as the difficulty of predictability of asteroids is proportional to the size and mass of the asteroids, and the size and mass follows a Pareto distribution, then so does the difficulty of predictability too (or unpredictability here). The situation is also similar to the difference of predictability of say motion of celestial bodies in classical physics and the difficulty of predictability of particles in quantum physics of micro-world of atoms.
It is also important to realize, that this small "window"of guessing, based also on the news, is easy on the index funds as they represent, the overall feeling of the real business, of the economic system, and by far more difficult for individual enterprises and business. The trend of an index fund is macroscopic and artificial, as whenever an enterprise ceases to be growing according to its balance sheets, then it is substituted in the index by a new growing enterprise. Strictly speaking the tend of the index fund is the surplus demand in the society to put money in the big and larger business that are represented with companies in the stock exchanges compared investing to smaller business outside the stock exchanges. It is like a law of universal attraction where big money attracts more money. Strictly speaking again such a an excess demand for big and larger business is not the best in society, as it increases the inequalities, while it would be more desirable investing in small and medium size business usually outside the stock markets. But from the point of view of psychology the duality of the psychology in investing in index funds is simple and asymmetric. When betting for up it is betting on hopes, when betting for down it is betting on fears.
Similarly, for the same reason, using this factor of predictability, is by far more difficult, for the commodities compared again with the index funds. It is rare that there is a clear situation of demand-supply interplay, of a particular commodity, that can be found quantitatively in the news, and be early enough to apply in trading. Most often there are no news useful, and there is no clear expected trend for a commodity. Cross-rates of foreign exchange currencies are a little bit less difficult compared to other commodities. Currencies f go up or down compared to the totality of the rest following the growth of the underlying national economy. But cross-rates simply record their relative differences. Of course larger economies have a lower growth rate compared to smaller economies (exactly as in biological organisms, see end of the current post ) and the distribution of rates relative to sizes is a power low. But again these cross-rate currencies with no strong expected trend, compared to index funds.
In the same rule we may be led even without assuming "intuitive guessing" abilities. It is known that the statistical distribution of the sizes enterprises follows the Pareto distribution, and so does the distribution of the size of packets of orders inherited from the enterprises, therefore of the volumes too. And the same we may assume for the size of populations of demand-supply that create the price patterns in the markets. Therefore that "inertia" which is due to the volumes ,of the movements of price patterns follows also the Pareto distribution. But this "inertia" due to the volumes, is the basis of the "statistical momentum conservation", and also the basis of predictability of the moves of the markets. So large scale and large volume patterns and small scale and small volumes patterns predictability is like the predictability of the moves of the planets compared to the predictability of the moves say of asteroids that also collide more often between them. Obviously the latter is more difficult to predict, even though the same physical Newton's law of momentum conservation holds. Now as the difficulty of predictability of asteroids is proportional to the size and mass of the asteroids, and the size and mass follows a Pareto distribution, then so does the difficulty of predictability too (or unpredictability here). The situation is also similar to the difference of predictability of say motion of celestial bodies in classical physics and the difficulty of predictability of particles in quantum physics of micro-world of atoms.
The truth is as it seems to me that for manual trading the golden scale is the seasonal cycles (2-6 months) as sub-cycles of a 5.55 Kitchin cycle or 22.2 years global climate cycle (Kuznet cycle, see post 5). Of course by programming automate trading it is possible to trade intra-day without spending human time. But the rate of return of such automated intraday trading is not higher than the seasonal manual trading if in the latter, the human pattern recognition is involved which is superior to automated trading pattern recognition. Some sellers of automated trading show the results of their programs for a limited time intervals (some months only) which appear very high so as to sell or rent them. But sooner or latter such automated trading had also significant failures so that in the long run (5 years or more) have less rate of return that the manual seasonal trading.
The way that unpredictability increases as the time scale becomes shorter, seems to be a result of the law of inequality. The very shape of the Pareto distribution could be used to present how unpredictability increases when the time sale decreases. In the next diagram of the Pareto distribution, the x-axis is the time scale, and the y-axis is the unpredictability. A hint of why this is so is the next: A wave in shorter time scale takes less volumes of transactions to be shaped. And the volumes of transactions, follow the Pareto or power distribution. The less the volumes the smaller the population (of transactions but also of traders) the less the predictability and stability.
We may have a different situation of the above arguments if the short term prediction of a trend (e.g. trend within hours or within a day in 4-hours bars) is an inherited prediction from the larger scale trend (e.g. 6 months or 2.75 years) in which case the stability of the larger scale prediction is inherited also in the shorter scale, but for sequences of limited only small time intervals. But in such a case we are supposed to trade only one side of the trend (always the one that is in accordance with the seasonal trend) and it is not therefore a genuine independent shorter scale prediction. In such cases the inverse pyramiding or moyen, or adjustments technique is the appropriate (see about this method in post 3). Notice that the adjustments or moyen or inverse pyramiding technique is based on the assumption of a constant infinite time trend. While the direct pyramiding technique is based on the assumption of a constant finite time trend following a power or Pareto distribution of time duration.
It is crucial to realize that cycles may emerge in the price changes , in a random way with a hazard rate of appearance at each period , and furthermore that they may appear not directly on the prices changes but on the rate of growth of prices.We must make clear here that we are not talking of exact periodicity but rather for randomly emerging temporary periodicity.
From the cycles (see post 5) (not including their harmonics, that is their sub-multiples of the periods) the order of intensity of effect on the price movements, and therefore the order of predictability also is approximately the next:
Daily (1 Day earth)>>
1 Year (12 months, earth)>>
11 years global climate (Sunspots) >>
Month (4 weeks, sun+moon)>>
2 weeks solar magnetic cycle (Parker Spiral)>>
160 mins Helioseismologic cycle >>
55 mins Helioseismologic cycle>>
5 mins Helioseismologic cycle.
And the most predictable effect modulated by such cycles (after the long term permanent trend) is the reaction to an super-exponential moves (a blow-up at the end of trend in the form of super-exponential move or terminal spike). (See e.g. https://www.ted.com/talks/didier_sornette_how_we_can_predict_the_next_financial_crisis and http://www.er.ethz.ch/ Such super-exponential terminal patterns of trend may occur usually as result of overgrowth of the one of the two populations in a demand-supply coupling rather that of domination and not so much of competition or cooperation. See also post 22. The frequency of emergence and the size of such super-exponential blow-ups follows the law of inequalities in other words the Pareto or Log-normal distribution and is thus by fat more often than pure randomness would predict! )
THUS THE ORDER OF BETTER PREDICTABILITY IS
1) LONG TERM PERMANENT TREND
2) A CYCLE IN THE ORDER OF PREDICTABILITY DESCRIBED ABOVE AND REALIZED AS REACTION TO SUPER-EXPONENTIAL TERMINAL MOVE (OR SPIKE).
1 Year (12 months, earth)>>
11 years global climate (Sunspots) >>
Month (4 weeks, sun+moon)>>
2 weeks solar magnetic cycle (Parker Spiral)>>
160 mins Helioseismologic cycle >>
55 mins Helioseismologic cycle>>
5 mins Helioseismologic cycle.
And the most predictable effect modulated by such cycles (after the long term permanent trend) is the reaction to an super-exponential moves (a blow-up at the end of trend in the form of super-exponential move or terminal spike). (See e.g. https://www.ted.com/talks/didier_sornette_how_we_can_predict_the_next_financial_crisis and http://www.er.ethz.ch/ Such super-exponential terminal patterns of trend may occur usually as result of overgrowth of the one of the two populations in a demand-supply coupling rather that of domination and not so much of competition or cooperation. See also post 22. The frequency of emergence and the size of such super-exponential blow-ups follows the law of inequalities in other words the Pareto or Log-normal distribution and is thus by fat more often than pure randomness would predict! )
THUS THE ORDER OF BETTER PREDICTABILITY IS
1) LONG TERM PERMANENT TREND
2) A CYCLE IN THE ORDER OF PREDICTABILITY DESCRIBED ABOVE AND REALIZED AS REACTION TO SUPER-EXPONENTIAL TERMINAL MOVE (OR SPIKE).
Here in the chart below which is with the prices after taking the logarithm, we may watch deviations from the linear moves (super-exponential moves) and their highly predictable reaction lasting in the average 3.75 years. during 30 years!
The idea that the appearance of price patterns is the same among different time scales is literally wrong. It is true of course that the basic price patterns like waving channels , non-waving trend, spikes, and flat channels appear in almost all time scales. (For the 4 basic patterns see post 22) but it is not true that their relative probability of appearance is the same in all time scales! THERE IS SYSTEMATIC DEVIATION AS WE MOVE ACROSS THE TIME SCALES. The shorter the time scale the more often the patterns of flat channels and waving channels. The spikes appear in time sales of hourly or 4-hours bars, and appear less often as we go to large and smaller time scales. As we go to larger time scales the pattern of constant non-waving trend appears more and more often, and at a scale of months and years it seems to be the only pattern at annual tome intervals.
From this we deduce easily that
prediction and trading is much easier at the larger scales, than the short term
intraday scales. The only phenomenon which is universal e.g. for
index funds indexes, is that they reflect the laws of growth of
enterprises and organizations, which is according to many studies Logy=a*Log x
(super-linear of super-exponential growth) where x is the size of the
organization, y is it time rate of growth (non-percentage but absolute growth)
, and a is a constant. For biological organizations the a is less than 1 it is about 75% but for economic organizations the a is
greater than 1 , it is about 115% and then there is collapse of
the growth. Nevertheless the index funds contain companies before
their collapse, and any time a company starts
to collapse, they substitute with a younger in the index
fund.
Deviations of the above law, is called by some researchers Dragon-crises and are used to predict the reactive collapse either down or up.
EXPOSURE OF POSITIONS FOR THE LONG TERM TREND AS IT CHANGES BY TIME SCALE.
Finally, we we remark that as the sample variance changes by the sample size by the rule of n^(-1/2), so also changes the ratio R/a^2 , where R is the average trend or drift, and a^2 is the variance of this trend. And this ratio determines the exposure of the portfolio, in the portfolio adjustment (see post 3) . E.g. as we calculated in post 3, if the annual R is 10% and the adjustment is annual and a is 34%, then the exposure R/a^2 for the adjustment is 66%. But if the same trend and variance is calculated not annually but daily and the adjustment is daily, then this exposure becomes about 3,5% !! See also post 12.
Of course we are calculating here how the stable long term trend appears in short time scales. As we have seen in other posts, in short time scales there appear temporarily micro-trend patterns with much larger R/a^2, even larger than 66%.
THE TOP 6 FACTORS OF ATTENTION IN MANUAL TRADING ARE
1) NEVER USE ALL YOUR FUNDS FOR TRADING. DIVIDE THEM TO TRADING AND NON-TRADING FUNDS BY THE RATIO f=R/a^2 RULE (see below for this ratio or in posts 3,13,33). THE DIVISION OF FUNDS AT EACH PERIOD IS ADJUSTED TO CONFORM WITH THIS PERCENTAGE RATIO. NEVER WITHDRAW PER PERIOD FROM THE NON-TRADING FUNDS MORE THAN HALF OF THE AVERAGE PROFITS OF THE TRADING FUNDS PER PERIOD. This division and adjustment of the funds has been applied for many years in buy and hold investments by professor Michael LeBoeuf.
2) THE ONLY CERTAINTY, WHILE TRADING IS ALSO OUR FIRST PRIORITY: WE MAY DETERMINE THAT OUR LOSSES AT EACH POSITION WILL NOT BE LARGER THAN A SPECIFIED PERCENTAGE DEFINED BY THE KELLY CRITERION (see posts 3, 13, 33)
3) FOCUS ON MACROSCOPIC INSTRUMENTS LIKE STOCK INDEXES WITH PERMANENT STRONG LONG TERM TREND, even if you want to trade at short time scales. (e.g. of the American Economy which is young and strong and indexes like Dow Jones, SnP500, Nasdaq etc)
4) FOR VERY LOW RISK AT OPENING POSITIONS ON THE PREVIOUS INDEXES WITH PERMANENT STRONG TREND, OPEN BETTING UPWARDS AT TERMINAL SPIKES AGAINST THE TREND. This is the Bill Williams technique.
5) READ THE NEWS AND FINANCIAL STATEMENTS BUT THE ASSESSMENT OF THE PATTERNS OF THE MARKET REQUIRES THAT IT IS DONE IN MANY SUCCESSIVE TIME FRAMES CHARTS. This is a basic recommendation by Alexander Elder, which, by now, it is a common knowledge to traders
6) BE FLEXIBLE IN FORECASTING THE MARKET AND DO NOT HESITATE TO FOLLOW PROMPTLY ANY UNEXPECTED CHANGES OF THE TREND OF THE MARKET.
The most essential tool for successful and profitable above the average, trading from the three that the title of the book suggests (Law of growth, law of cycles, law of inequalities) is the law of cycles and the awareness to discover cycles in the charts of prices, that are not directly apparent. Especially when the cycles are 1) daily cycles to be traded with hourly or 4-hours bars and 2) Weekly cycles to be traded with hourly or 4-hours or daily bars 3) Monthly cycles to be traded with daily bars 4) Seasonal 3-months cycles to be traded with daily bars.
No comments:
New comments are not allowed.