Posted on November, 14, 2016

Written by IMC , Chicago

Posted in Technology

The world of trading is changing. Gone are the days where traders bought and sold stock over the phone. Now, sophisticated computer programs are being used to make trades at an astounding speed and frequency impossible for any human trader to achieve.

It’s called algorithmic trading. For firms like ours, it creates a big data challenge.

At IMC we crunch 20-40 terabytes of market information daily. That’s the equivalent amount of data as 2.5 times the printed assets at the Library of Congress every day. The only way to process and store this huge volume of data was to wipe it at the end of each day, leaving little opportunity to analyze the data to ensure that we are constantly learning and improving.

Or at least that’s what happened before we handed the big data challenge to our crack team of software engineers and developers. They started looking for a way to store this huge volume of data long enough for meaningful analysis, which would allow traders to identify market patterns and swiftly test algorithms to make the maximum amount of good trades.

Having that long-term access to stored data across a global network has reaped great advantages; it’s allowed our team to revolutionize the accuracy of time sources for trading. Events of all types are now recorded in nanoseconds.

At IMC the team is always pushing to achieve the best results possible, which often means not getting attached to legacy systems and traditional ways of thinking. To stay one step ahead of the market, we need to create solutions that are one step ahead of technology developers. It’s all in a day’s work at IMC.

Find out how you could be solving the next big challenge at IMC.