Consulting, Machine Learning, Predictive Engines
Quantized ML Classifier, Elastic Search, Lucene, Product Search, Enterprise Search, Auto Suggest.
A general purpose, high performance machine learning classifier. Source Code
Examples / Tests cover:
Continue reading “Quantized classifier Machine Learning in GO”
Digital Elevation Models are the height maps of the Earth taken by Satellites. This data was produced using the Shuttle Radar Topography Mission (SRTM)
The STRM3 data is available for most of the world while the SRTM1 data is available for most of the USA. The SRTM1 data provides finer resolution than the SRTM3.
The basic quick start and topography docs are OK but working code is always easier to use.
I have included Sample Scala to download the entire world wide set of SRTM3 files along with the SRTM1 files for the USA. There is also sample Scala code to read the .hgt files and convert them into grey scale renderings. Continue reading “DEM – Digital Elevation Model work with Scala”
Scala example of fetching a string from remote URI, saving results in local file using command line parameters.
would normally use Curl for this. Continue reading “Scala fetch string from URI and save results in file.”
Scala Example to parse almost ISO Date time string, convert to proper time zone and regenerate as ISO standard string.
See Also: Helpful Scala Resource Links
Continue reading “Scala parse almost ISO Datetime String with TimeZone adjustment”
When I learn new programming languages I first choose a non-trivial programming problem and then develop pieces of that system in the new language then go back and re-factor as I learn more. Solving a real problem with the language is the fastest way I have found to gain demonstrated competence in a new language. Continue reading “Scala Resource Links Getting started”
Lua Source demonstration of using Kelly Criteria to figure out appropriate amount to risk for any single trade based on your current capital, odds of winning and win versus loss payout. Continue reading “Lua Source to calculate Kelly Criteria Investment Level”
For those of you who are testing our Forex Mini strategies you need to talk to your Tax professional before you make your first trade during the fiscal year. There is an internal election you need to make and if you don’t you can be stuck with a tax treatment you do not like. Continue reading “Talk to your tax specialist before making your first Forex trade”
I recently started making Forex Micro-strategies available for sophisticated users under the general terms that they agree to pay 15% of their net profit. Those that pay will also gain the benefit of continued enhancements and new micro strategies. Contact us for access. Continue reading “Precision VS Recall, a Net Trading Profit Perspective”
I started using LuaJIT© after first using F#, Python, Julia and C for stock and Forex related predictive work. I am always on the lookout for a language that is high speed as close as I can get to C without having to write in low level C all the time.
Lua is a language that feels somewhat like a cross between BASIC and Ruby and has been around for a long time. Lua may embedded or used stand-alone. It has been embedded into many games, entertainment consoles and other devices as a scripting language. The LuaJIT is a new compiler technology and takes what was already fast as an interpreted language and in some of our tests made it run over 20X faster with a few tests reaching 80X faster.
LuaJIT seemed like the ideal combination since it provided a language any ruby or python programmer would find readable with fast start-up times, excellent run-time speeds and good error messages. Continue reading “LuaJIT Access 20 Gig or More of Memory”
I needed this utility to save 1 minute bar data tick data in a form where our AI / ML prediction engine could access it. I needed the history but even more important I needed the something that could deliver new ticks to the prediction engine fast as they became available.
I cheated with this utility and open the file in shared mode. This allows my Lua code read new lines as they become available as if the data was delivered by a more traditional pipe. I tested the lua code which reads to the end of the file and then recognizes when new data is available simply by continuing the read process. When no data is available it returns immediately so you can do a very low cost poll without having to re-open or re-read any part of the file. When I move to linux I can replace the file with a named pipe and eliminate the overhead of writing to disk.
Not a elegant architectural approach but it does keep the interface between my proprietary code and the 3rd party cAlgo product as thin as possible so I can change brokers easily in the future.
I was able to get it to deliver 1 minute bar data back to Jan-2011 which is the best source I have found for that much data. Still has a flaw in that I could not figure out how to calculate actual transaction volume for the bar but will comeback to that when it is critical. .
I needed this to save tick data in a form where our AI / ML prediction engine could access it. I needed the history but even more important I needed the something that could deliver new ticks to the prediction engine fast as they became available. I cheated with this utility and open the file in shared mode. This allows my Lua code read new lines as they become available as if the file were a more traditional pipe. Not perfect but I wanted to keep the code in cAlgo as small as possible so I can change brokers easily in the future. It only delivers about a year of back tick data but that is still over a gig of data.
http://ctdn.com/algos/cbots/show/591
A version of this code is used in our Production Forex Trading by our Forex Prediction Engine.
I started testing Julia as a possible alternative because Julia advocates claimed the interpreter loop was nearly as fast a C and it was similar in concept to Python which I love but which was too slow for our application. I recently ran across a blog entry mentioning a new Lua Jit. I found it intriguing because Lua did quite well during our last round of tests.
| Relative Execution Time. Lua Jit as baseline – lower is better | ||||
| Operation | Lua52 | LuaJit | Julia | Julia Typed Array |
| Parse File into Data Frame | 2.42 | 1.0 | 5.64 | 5.64 |
| Compute SMA(14) | 2.81 | 1.0 | 6.87 | 0.70 |
| Compute SMA(600) | 33.32 | 1.0 | 80.00 | 1.30 |
| Compute SMA_slice(14) | 2.42 | 1.0 | 11.87 | 1.83 |
| Compute SMA_slice(600) | 33.32 | 1.0 | 15.52 | 5.90 |
| Did not implement slice in Lua so re-used the timing from nested loop version. Response times are in seconds. |
||||
The only function where Julia out performed Lua Jit was in the SMA(14) all other items tested were slower. I think the reason it did better in this instance is that the SMA function must allocate a new array with 71K rows to store the results. In Julia you can do this as a typed Array of float. In Lua this is done as an append to list so it is allocating memory in little pieces. In the SMA(600) the Lua jit was faster again because it is doing more work compute in a tight loop relative to the memory allocation overhead.
Continue reading “Lua jit tests faster than Julia for Stock Prediction Engine”
Article in Inside Market Data – Bayes Analytic Preps Price Prediction Engine
We have an article in Inside Market Data. Bainbridge Island WA-based startup Bayes Analytic is preparing to launch a new stock analysis and trading recommendation engine that leverages advanced trading algorithms and machine learning to predict future price movements based on past trading patterns.
See The article at Inside Market Data
Contact us
Continue reading “Article in Inside Market Data – Bayes Analytic Preps Price Prediction Engine”
Moved to the Reference page