Frame doubling interpolation for 1080p x264 mkv's for superfluid 48 fps

http://forums.guru3d.com/showthread.php?t=288017

Guys I have found something interesting. This will transform your watching experience thanks to the interpolation you can watch 48 fps instead of 24 frames per second.

"It is great for scenes where the camera pans over the scenery at a constant speed. Fast action scenes tend to be more clear as well. The whole movie gets a very realistic feel to it. It's just awesome. Actually IMAX HD is at 48 fps as well. Beware though, it is not easy to get this working. But once you got it, you never want to go back to 24!"

A friend of mine showed me, it is really amazing stuff.

This forum page is important showing how to change the script regarding the number of cores. If you had at least a quad core you should not have a problem, but even on my modest two core laptop with Intel 2.2 Ghz I get it running.

with: SetMTMode(1,2)

No really it is hard but if you get it running you will transform your film watching experience.

Optimization of trading strategy: is there something wrong Part 2

Is there something wrong with optimizing trading strategies. I think the whole conception of optimizing may be a wrong paradigm. In fact during optimization we make in the same time the strategies very vulnerable to randomness.

That is the whole point. All trading strategies are very  vulnerable to randomness but the problem is that y we do not know to what extent they are vulnerable.

That is why the use of any fancy digital filters or enigmatic time series preprocessing methods: SSA, wavelets, PCA does not add much.

On the other hand the use of different optimization algorithms does not add much too: back propagation, genetic algorithms, particle swarm optimization etc.

So what is the reaction of the big guys. They were investing in infrastructure in order to use the infrastructure edge together with complex data mining methods. And that worked and works. However as many others will do the same the competition will increase in the low latency market. Even more some countries are planning to tax the HFT, in France this is already a reality.Big boys are also using traditional macro strategies but this is another beer.

My point here is that a successful trading strategy needs to balance two pillars:

a. An adequate optimization regarding the market conditions. In practice that means that the strategy needs to learn from the market conditions and not to over - fit the past market conditions. This is a hard stuff.

b. The randomness robust trading strategy. There is a need to secure that the trading strategy is robust regarding the randomness of the market.

There are different ways to achieve it.

- To measure the state of predictability of the market and trade only when the market is sufficiently predictable. This is the core concept of the Spinal Implant EA. In this EA the algorithm takes a trade only if the fractal dimension is below a certain value.

-To apply walk forward testing.

This is a traditional approach and maybe is the state of the art in the trading community. But unfortunately quite a few algorithms pass this test, only to prove later that they are not so good.

-To apply Monte Carlo methods

So I really think that this should be implemented by everybody that seeks consistent results.

The Discussion will be here: only for logged users, sign up for free.

Kathy's trade and comments

Here I will post a Kathy Lien video about the current situation on 19.12.2012.

She has a long position targeting the break-out (I was talking too about a break-out).

From a fundamental perspective she makes some comments about fiscal cliff and an optimism about Euro crisis and after all she has a long bias.

Anyway, this was the first thread in beathespread, and we somewhat forgot this thread. Now I am reading the posts most of them discretionary analysis. And hey I was right most of the time about big moves coming and Euro going to hell again.

Mostly I was surprised LOL.

Here I want to make a comment  and a warning about the current situation.

What Katy does is classical analysis. The classical analysis has two parts.

a. Short Fundamental analysis

Fundamental analysis of the main driving forces and deriving a bias long or short based on the expert and discretionary knowledge.

Here in this case Katy does so and she has a long bias. From what Jaguar is talking there are others who have the same bias. But I really do not have an opinion on it because I am not a fundamental analysis specialist.

b. Technical analysis

She makes a technical analysis and we are clearly in a break-out situation that is so obvious. And last but not least the technical analysis is giving the risk control for placing the exit stops.


My comment is that that is the traditional way to do things. You can add also some more exotic ways of technical analysis as the Elliott waves for example. Here is a screen shot of an Elliott wave analysis with simplified counting (the simplified counting is used in order that everybody could get the reading and not the specialists alone).

So the Elliott waves picture is that we are in a global complex triangulat pattern in EUR/USD that is really hard to predict on the long term but do have some local predictability.

My additional comments

c. Elliott waves (simplified counting)

Here is the chart and of course regarding the position entry it mirrors the normal technical analysis regarding the break-out opportunities.

However in beathespread some more methods ans approaches have been proposed.

The method we propose is that we identify the market conditions locally.

As for the current situation we can just observe and measure the local market conditions. Thr trchnical analysis may be just smoke and mirrors as we jave observed so much sharp V-turns.

So what can be done is to use an EA model that is adpated locally to this market conditions. So if the break - out continues the EA model will follow it. If there is a sharp tern the EA would reverse just in time and follow the reversal.

I think this is what professionals do, I am simply explaining in plain English.

Have a nice day.

Optimization of trading strategy: is there something wrong

What is a central question of all trading strategies is the optimization. However is there something wrong with the optimization or at least with our perception of optimization of trading strategies.

I think there is something elusive here.

I think that the main problem is that we look at the market as a problem.

Yes, I think so, we think that the market is a problem and if there is a problem we look for a solution. Applying mathematical models for searching a solution is just the right thing to do.

But what if, there is something wrong in our paradigm?

Of course the trading rule with the highest observed performance is likely to perform in the future provided we have sufficient observations and we do properly performance statistics.

However the problem is that the best rules are highly positively biased. Aronson writes in his book the Objective Technical analysis that technical analysis methods "have the potential to be valid knowledge but only if back-tested results are considered in light if randomness (sampling variability) and data-mining bias.

This screen shot is from the book of Aronson Evidence based technical analysis. Here you can see out-of-sample performance deterioration, that means that the rule which performed well in the past did not perform in out-of-sample trading.

Here in the first part is the in-sample data: the data used for data mining or simply put rule back-testing. In this period you see 50 % Return on on investment. However the out of sample is a different thing. The out-of-sample actually means that the data is not used for data mining and is the actual performance.

The reality is more complex as we may assume. There is not only a problem that needs a solution to be found but there is something else too. That explains why the data - mining normally fails in out of sample trading. I will stop here for the moment.

PAMR: Passive Aggressive Mean Reversion Strategy for Portfolio Selection

PAMR: Passive Aggressive Mean Reversion Strategy for Portfolio Selection by Bin Li, Peilin Zhao, Steven C. H. Hoi, Vivekanand Gopalkrishnan

On this link you can download the Matlab Source code. Otherwise the article is here.

The Bin Li's doctoral dissertation is about "on-line portfolio selection", which intertwines machine learning and portfolio selection. The key issues in the topic is to find effective patterns among markets and then effectively allocate the capital for portfolio trading. He has mainly involved in two types of approaches.


1. He firstly uses nonparametric learning to extract a set, whose patterns are expected to be similar to future pattern. And then, he exploits the patterns via portfolio trading (using Kelly's approach).

2. After realizing that mean reversion is a common pattern among the markets, then he designs learning algorithms to trade on such pattern.

The backtest results of the two approaches on real market are real encouraging according to him.

For other techniques beyond his thesis, he thinks that the main usage of machine learning is to automatically identify (unknown/unknown) patterns among markets. Then feeding the related inputs is critical, such as neural network, or genetic algorithms.


You can contact Ben Li in linkedin.

How can we determine ahead of time when a certain EA would be suitable for upcoming market conditions?

That a trading holy grail question. Much more than the one million question LOL.

What we can do is to know if it is working now.

Based on that you can evaluate how much draw-down is logical and calculated within the system. You can use a Monte Carlo simulation based on that knowledge. Check the post about Monte Carlo simulations and the appropriate risk control.

You can use market state analysis as an independent tool too. This a discretionary approach based on the best practices among discretionary traders and this is how far this pragmatic field can go by now (in my eyes of course).

And you can use your fundamental knowledge to have some insight. Sometimes the market state is closely related with fundamental information. Here I can give a historical example: the Swiss/Euro EA, this strategy was based on the Swiss Central bank decision setting a minimum exchange rate of 1.20. Until we were under the same circumstances the EA was expected to work and it worked too well for several months doubling the account many times.

And you can use data mining only for predicting market conditions. That is also possible, however it is not an easy task.

Predicting the future performance of an Expert adviser in Metatrader is not an easy task. Before doing this you need to know what is really the worth of the EA model. 

Suggested books:

The Evaluation and Optimization of Trading Strategies (Wiley Trading) by Robert Pado
Trading Systems: A New Approach to System Development and Portfolio Optimisation by Emilio Tomasini and Urban Jaekle

Machine learning in trading strategy?

Question: Does anyone here apply machine learning techinques (such as Neural networks, PCA, local sensitivity hashing) to his trading strategies?

There are two main commercial vendors of AI models for retail traders:
Neuroshell and TradingSolutions.

I think that they are offering a very easy tutorials allowing to grasp the use of those models in trading systems.

On both sites there is a lot of free information about basic models. In trading solution there is even a demo version which gives you several models and gives you the opportunity to make new models.

If I can summarize it in really rough way basically the basis of everything is:

* To select the predictive inputs (mainly technical indicators) and try to predict with them something else output.

* Use the output as a trading signal.

For example:

You have 2 inputs
You use a moving average of period of 20 as input.
You use a second moving average with period 10 as input.

You want to predict the next day close.

So the rule may be if the predicted next day close is higher than the next day open then buy.

If the predicted next day close is lower than the next day open then sell.

You train the model. And you backtest the model over a past unseen data.


So that is the basis and from there start the variations including their use in portfolio.


I have two remarks:

* The problem is that trading with neural networks is not an easy task. My opinion is applying a model over an already viable strategy or idea will allow to squeeze more profit from the idea compared to the rough model.

* Another problem I may say is luck related. Very often you can confuse the optimization with the luck. Very often the results you get are purely due to the luck. Monte Carlo simulations may help a lot.

Please check the book http://www.evidencebasedta.com/ and Monte Carlo
Permutation Evaluation of Trading Systems
http://www.evidencebasedta.com/MonteDoc12.15.06.pdf

I just want to add another remark that by using commercial software within the software is integrated a fool proof.

In Neuroshell for example they are offering to you some default settings that will limit the overfitting. In trading solutions it is the same. If you use your own model or an open source software and you do everything by yourself you need to be carefull what are you doing, for example adding too many weights to the model and too many neurons and hidden neurons will increase the risk to overfit the data and not to learn from the data.

Get real-time feed of live data into Excel

I have found this list of sources of data for real time feeds of live data. I think it may be useful that is why I share the list here.  

Get real-time feed of live data into Excel
Get free historical data into Excel, Matlab or Java
Foreign Exchange, Forex, FX

www.forexite.com (Forex, 1 minute ticks), CSV download
ratedata.gaincapital.com (Forex, tick data), CSV download
www.oanda.com (Forex, daily), CSV download
www.truefx.com (Forex tick data) after login
www.dukascopy.com (Forex, ticks), CSV
Debt/Interest Rates

historical time series
www.euribor-ebf.eu (Interest-Rates: “Euribor, Eonia”), Excel download
www.databank.rbs.com (Exchange Rate, Interest Rate, Consumer Price Inflation (CPI), Commodities: “Brent Spot, Henry Hub Spot, I P E Brent crude futures 1-pos, I P E Natural Gas Index, OPEC Fixing, US EIA Crude Imports, WTI Daily”), Excel download
www.ecb.int Interest-Rates and Government Bonds, web tables and partly Excel download
(German) www.boerse-duesseldorf.de (Bonds, daily), Website table only
(German )www.bundesbank.de German money and interest-rate statistics
current
(German) www.baadermarkets.de (Bonds)
(German) www.westlbmarkets.net (2012-07-12: Just went offline, hopefully online again soon), Swaps, Caps, Floors, CSV download
Stock / Indices

historical time series
finance.yahoo.com/(Stocks, Indices), Excel download
www.google.com/finance, Website table only
www.nasdaq.com (Nasdaq), Excel download
mba.tuck.dartmouth.edu (portfolio data), text download
(German) finanzen.handelsblatt.com
www.investopedia.com, Website table only
www.nasdaq.com, Website table only, intraday, daily
www.standardandpoors.com (S&P 500, daily, yearly)
current
finance.yahoo.com (Stocks, Indices)
www.bloomberg.com (Stocks, Indices)
Energy

EU http://www.epexspot.com/de/marktdaten/intraday-handel (German), Website table only
US http://www.gdfsuezenergyresources.com/index.php?id=33 CSV download
General Historical

(German) www.onvista.de (Stocks, Indices, Warrants, ETFs …, daily Quotes) Website
www.indexmundi.com World-wide econometric data and commodities
www.measuringworth.com US, UK, Japan, China with interest rates, GDP, inflation, gold and silver prices, cool long term, Website table only
Sites with Summaries

scraperwiki.com (The community collects free data from the web and converts it into CSV, etc.)
wikiposit (The author collects free data from the web and converts it into CSV, etc.)
Matlab Trendy (The community collects free data from the web and converts it into plots)
Economic data

World

www.indexmundi.com World-wide econometric data
OECD.StatExtracts includes data and metadata for OECD countries and selected non-member economies.
Google Data e.g. Market Capitalization in GDP: Google Data on Marcet Cap in % of GDP
European Union
Eurostat
www.ecb.int
www.destatis.de Statistisches Bundesamt
United Kingdom
www.statistics.gov.uk
United States
Federal Reserve Economic Data – FRED
www.census.gov
www.bls.gov
www.ssa.gov
www.treasury.gov
www.sec.gov
www.economagic.com (Web, Excel download after login)
www.standardandpoors.com/indices/sp-case-shiller-home-price-indices (Shiller-House Price Index)
www.data.gov Public data of all US government organizations
Live Cams

www.boerse-frankfurt.de (German Dax)
Buy Data:

(I did not try any of these, but they look interesting.)

www.cmegroup.com (derivative prices)
www.ivolatility.com (Implied volatilities)
commodities.euronext.if5.com (Cocoa Future, Robusta Coffee Future, White Sugar Future), 15 min delay, XML
www.eoddata.com (End-of-Day Stock), Excel download and more
www.historicaloptiondata.com (Option price data)
www.livevol.com (Implied volatilities)

Trend following EA performance results

I will post some recent performance tables of Brain Trend EA on its default settings og lenght 50. The results show clearly that it performed well on 30m, 1 h and 4 h time frame.

On 15 m time frame it did not perform on any settings.

I somewhat missed the performance because I was focused exclusively on the 15 m time frame and I knew it was not working there. So I am glad with the results because the idea here is not to produce EAs and hoping to find the holy grail by chance ;).

The brain trend is a sound system with its limitations.



От EA tests




От EA tests




I will add the results on 1 h time frame and the 4 hour time frame with the same parameters of lenght of 50.

A you can look the results it is typical for trend following system: the % profitable trades is below 50 % but the average profit trade is bigger than the average loss trade.

As you can see 43 % is very good performance for trend following system.

Also the largest gain (77.4) has to be much bigger than the largest loss (-41.10). However you need to see if all the good results are not due to just one trade.

Also it is important to have a look at the consecutive losses. This is also a critical parameter for trend following trading systems.

In the case they are 6. This is a lot but hopefully the consecutive loss (-132.27) is not very high compared to the biggest profit trade (230.39).

What is a bittle bit surprising of this system is the consecutive profit of wins of 4 (398.89). That is surprising for a trend following system. In fact the brain trend is an upgraded trend following system, I would say it is an impulse following system. When the market state get more predictable a persistent it is able to follow it. Check about the market states analysis.

Well this is not much, but this is what we get from Metatrader as statistics.

Now I will add the screen shots:



От EA tests




От EA tests


So you can argue that the data is not big. And yes it is true. The performance of the system is closely related with the predictability of the current market state. I was thinking about a seasonal pattern but I do not find any clue about it.

The discussion and the download files are here.

Forex Remote Viewing

Can you apply remote viewing for Forex trading? Theoretically Remote viewing can be applied everywhere. Of course it is obvious that the tools will be used to suck a lucrative activity as trading. Historically the first methodologies were developped for the Stock market.

And guess what if you google search those key words that would mean that you had the idea that the remote viewing techniques may be useful for trading. And if you had the idea that means very often that somebody else had the same idea too. Here I will give you a link to a you tube promotion video that matches what you are looking for.

However I will bring the discussion a little bit further.

The video and the course is from Ed Dames. Who is Ed Dames and how is he related with remote viewing.

Major Ed Dames is a distinguished military intelligence officer from United States Army (ret.).

After the declassification of the Project Stargate many of the military personnal went into the private sector.That explains how this and other courses are created.

So with Ed Dames you will get first hand experience in remote viewing. As this is very technical stuff the details are what matters most. So if you are going to put real money into it be serious and learn from the best.

Here I end with the recommendations and I will show the limitations of the method. Check the previous post about the risk.

The problem is related with the multiple time lines. There is a theory that there are multiple time lines and if you get a result you can't be sure that the result is withing your time line.

For more explications you can check this video from Courntey
Brown: 



So who is Courntey Brown? He has a  Ph.D. degree from Washington University (St. Louis) in 1982 in political science with an emphasis on mathematical modeling and he is a leading scholar on the subject of "remote viewing" and he is representing the non profit public research with the Farsight Institute.

So he is another authority in the field if Ed Dames is former military agent and turned toward the practical use and teaching, Courtney Brown is theoretical and scientific.

The discussion is here:

The Stanford Discovery and the remote viewing

The Stanford institute were trying to understand what makes the psychics succesful in their predictions and what makes them damn wrong. They tried everything for spi purpouses even out of body experiences.

As I am from the french school of though and we disnguish in everything two twings all the time I can say that there are two ideas.

The firt way is ESP (extra sensory perception) is a a particular state of mind ans special mind conditions are necessary in order to be a psychic. And you will find none of this in those documents, even if Ingo Schwan uses an alternate mind states.

However in those condition you really cannot understand what is real and what is imagination. On the other hand you are very vulnerable for psychic attacks. Some magick practitionors know that very well.

The other theory is that no particular state of mind is necessary. We are constantly in the flow of information. They call that the matrix, long before the film. What is needed is a process to get the information before the creative imagination take over. They call all picture that is coming in the imagination an Analytical overlay and is most of the time rubish. So you will not find any special meditation techniques. Why? Because there arent any special mind techniques. The only thing you can do is to take bit per bit information from the matrix and record it on paper. And you make more and more iterations. And you process the information before the imagination takes over. There is the secret and that is a BIG discovery and most of the people interested in the new wave are completely unaware if it and they are a simple pray of their imagination and not so friendly non material influences. As traders are particularly vulnerable it is not a good idea to experiment with those kind of stuff, especially if you are not under supervision and control. Anyway it is dangerous.

The truth is ()just an opinion LOL) that we are in a big ocean of information. There is a a barrier between it and us. Any force attack of the barrier is protected (the creative imagination under alternate state if mind). That is why other methods have been developped taking information bit per bit, per bit, hacking it, this is what remote viewing is about.

Have you ever solved a problem in less than 4 seconds? If you did that is probably because you had an access to that source of information.

Read more: