Quantized classifier Machine Learning in GO

ML Quantized classifier in GO

A general purpose, high performance machine learning classifier. Source Code

Examples /  Tests cover:

  • ASL Sign language Gesture recognition
  • Classify breast cancer from 9 features. 96% accuracy at 100% recall first pass with no optimization. Using 623 training rows.
  • Predict death or survival of Titanic passengers.
  • Predict Diabetes
  • Please send me data sets you would like to add to the test.

Continue reading “Quantized classifier Machine Learning in GO”

Node JS formula parser recursive decent

Available for download at:  BitBucket

Javascript based formula elevator that does not require use of eval.   I wrote this code to test some ideas about how to use function pointers to completely decouple the implementation from class specification.

I used the function pointer style of the command pattern rather than inheritance because I originally intended to use this problem as an experiment in GO and RUST where function pointers are first class citizens.

Once I started playing with ideas in JavaScript the rest just kind of fell out. I don’t like the recursive descent mechanism in JavaScript due to the stack overhead but it would be great for languages that do a better job of tail recursion optimization like Scala.

It works but I would modify to remove recursion before heavier use. Other TODO are in the Readme file at bitbucket.

Continue reading “Node JS formula parser recursive decent”

DEM – Digital Elevation Model work with Scala

n51e001-london-ukDigital Elevation Models are the height maps of the Earth taken by Satellites.  This data was produced using the Shuttle Radar Topography Mission (SRTM)

The STRM3 data is available for most of the world while the SRTM1 data is available for most of the USA.   The SRTM1 data provides finer resolution than the SRTM3.

The basic quick start and topography docs are OK but working code is always easier to use.

I have included Sample Scala to download the entire world wide set of SRTM3 files along with the SRTM1 files for the USA.  There is also sample Scala code to read the .hgt files and convert them into grey scale renderings.  Continue reading “DEM – Digital Elevation Model work with Scala”

Scala fetch stock bar data from Yahoo finance service

Scala Example to retrieve yahoo stock bar data from Yahoo finance service.  Saves  resulting BAR data in local CSV file. Fetches each year from server and saves in file name symbol-yy.day.csv where yy is replaced
with the year. The .day indicates this is a day level bar. The Yahoo service does not offer anything smaller than day. Continue reading “Scala fetch stock bar data from Yahoo finance service”

LuaJIT Access 20 Gig or More of Memory

Access 20 Gig or more from LuaJIT while coding in native Lua  and minimizing GC speed penalties.

I started using LuaJIT© after first using F#,  Python, Julia and C  for stock and Forex related predictive work.     I  am always on the lookout for a language that is high speed as close as I can get to C without having to write in low level C all the time.

Lua is a language that feels somewhat like a cross between BASIC and Ruby and has been around for a long time.   Lua may embedded or used stand-alone.  It has been embedded into many games,  entertainment consoles and other devices as a scripting language.    The LuaJIT  is a new  compiler technology and takes what was already fast  as an interpreted language and in some of our tests made it run over 20X faster with a few tests reaching 80X faster.

LuaJIT seemed like the ideal combination since it provided a language any ruby or python programmer would find readable  with fast start-up times,  excellent run-time speeds and good error messages.   Continue reading “LuaJIT Access 20 Gig or More of Memory”

Sourcecode to save Forex Bar data to CSV via cAlgo bot

I needed this utility  to save 1 minute bar data tick data in a form where our AI / ML prediction engine could access it.     I needed the history but even more important  I needed the  something that could deliver new ticks to the prediction engine fast as they became available.

I cheated with this utility  and open the file in shared mode.  This allows my Lua code read new lines as they become available as if the data was delivered by a more traditional pipe. I tested the lua code which reads to the end of the file and then recognizes when new  data is available simply by continuing the read process.   When no data is available it returns immediately so you can do a very low cost poll without having to re-open or re-read any part of the file.   When I move to linux I can replace the file with a named pipe and eliminate the overhead of writing to disk.

Not a elegant architectural approach but it does keep the interface between my proprietary code and the 3rd party cAlgo product as thin as possible so I can change brokers easily in the future.

I was able to get it to deliver 1 minute bar data back to Jan-2011 which is the best source I have found for that much data.   Still has a flaw in that I could not figure out how to calculate actual transaction volume for the bar but will comeback to that when it is critical. .

http://ctdn.com/algos/cbots/show/599

Contact us

Sourcecode to Download Forex Tick data to CSV via cAlgo

I needed this to save tick data in a form where our AI / ML prediction engine could access it.     I needed the history but even more important  I needed the  something that could deliver new ticks to the prediction engine fast as they became available.    I cheated with this utility  and open the file in shared mode.  This allows my Lua code read new lines as they become available as if the file were a more traditional pipe.   Not perfect but I wanted to keep the code in cAlgo as small as possible so I can change brokers easily in the future.  It only delivers about a year of back tick data but that is still over a gig of data.

http://ctdn.com/algos/cbots/show/591

A version of this code is used in our Production Forex Trading by our Forex Prediction Engine.

Contact us

Why Language Performance Matters & Some Measurements

An optimization pass that takes 10 minutes in one language could take 15 hours in the slower language.

I work from the philosophy that I want the highest performance I can afford but this is traded off against development costs and delivery time because a great solution 6 months later  may be less valuable than a good enough solution 6 months sooner.

Continue reading “Why Language Performance Matters & Some Measurements”