This article was written by and originally published on Built In Chicago.
Writing efficient code is important across the tech industry, but for developers at IMC, wasted nanoseconds can add up to lost opportunities. To maintain a competitive edge as a market maker, the firm encourages its technologists to be on the lookout for inefficiencies in its systems and to implement improvements whenever possible.
Built In Chicago spoke with two members of IMC's tech team about the technologies they employ, the company’s culture and what they look for in potential hires.
What does IMC do?
Andy Adams, Chicago head of technology: We're market makers at our core. Across every financial instrument that we trade, we're both willing to buy and sell the product, which provides liquidity for the marketplace. So, if someone wants to buy stock in their 401(k), we'll sell it to them. And if they want to sell stock to get cash to buy a house, we'll buy it from them.
What do you need to build to make that happen?
Adams: Our systems are responsible for determining the value of every financial product that we trade. We come up with that using different types of algorithms, models and quantitative characteristics, as well as data on prices correlate with each other. Then, once we know what we want to trade, we have to be the fastest to execute. That means we rely heavily on low-latency infrastructure. Some companies talk in milliseconds or microseconds — we get into nanoseconds.
Tyler McDougall, Chicago team lead: We need extremely efficient systems that also have to be very safe, both to prevent loss and our reputation. People who work here take the craft of writing software very seriously.
How do you build a system that performs at that level without hiccups?
Adams: The biggest challenge is doing it at scale. Generally, the more specifically you can tailor your code to do one thing, the faster it can run. If we try to make things too generic, that creates overhead, which means we can’t compete. Our job is to create tailored solutions that can scale across the 50 or so exchanges and dark pools that we trade on — and that do so efficiently.
Our algorithms also deal with a lot of complex calculations, so we have to strike the balance between being really smart and being really fast.
How do you decide where speed overrides complexity, and vice versa?
Adams: We try to measure everything we can with different types of metrics. We measure latency in different areas of our system, as well as how much money we think is available in the market. The more data we have, the better we know where to focus our efforts. And we also find out where it makes sense to give up some flexibility to gain a little extra edge.
What data do you draw on to determine the value of a financial product?
Adams: There are product groups that are very highly correlated. You could look at all stocks, for instance, or at airline stocks as a group. Then there are products that correlate more tightly because they’re derivatives of the same underlying security. There are gold futures, and there are exchange-traded funds that also have gold as their underlying security. If the price of gold moves, both of those are going to move in tandem.
And your trading decisions are based on that data?
Adams: The cool thing about it is that all of this is automated. We build systems that manage all of these complex algorithms. Throughout the course of the day, we are listening to billions of market updates from exchanges and making decisions within milliseconds, or even nanoseconds. We also look at how the prices of different products are correlated to manage our risk internally.
What has been the most surprising thing that you’ve learned after starting?
McDougall: The scale at which latencies have gone down over time is bizarre. If you could take our systems from today and move them back in time 10 years — saying it would be no contest doesn't even begin to approach it.
At this point, you’ll get messed up by something like excess wiring between servers.
Adams: Yeah. We know how many nanoseconds a metre of wire is, because that matters to us. Speed also matters for the technologies we use. We're a Java-based company, and the bulk of our software still is Java. But where latency is important, we've moved to C++ and field-programmable gate arrays. With FPGAs, we're developing low-level trading strategies directly onto the hardware itself. FPGA is actually a really old technology, but it’s new to this industry because speed is becoming so critical.
Do the engineers you hire usually know these technologies, or do people learn them while working here?
Adams: It's a mix. We have a number of developers who were doing Java, but who have transitioned to become C++ experts, or who transitioned from C++ to FPGA.
How has your company’s approach to technology changed over time?
Adams: We used to do sprints with releases every two weeks or so. But we found that we weren’t getting fast enough feedback on our work. Moreover, if we’re sitting on useful functionality for two weeks before releasing, we’re basically losing money. So we invested heavily in automated testing and we bolstered automation around deployment. At this point, we deploy new code to our production system every day, so long as our automated tests are successful.
Traders used to be nervous when we were doing upgrades. Now, if we go a day or two without being able to upgrade, traders start craving the new thing that’s coming.
What was the transition from two-week cycles to daily deployment like?
McDougall: Building the infrastructure is one thing, but you also have to change developers’ mindsets. The code they check in will be in production tomorrow — that changes how they approach the problem. But if you look at it correctly, it's actually considerably safer to upgrade every day. If something does go wrong, it’s easy to fix it because you were working on it yesterday.
Adams: We’ve evolved a lot as a company. I’ve been at IMC for nine years. Back then, we were a trading company with good technology. Now we’re a technology company that trades.
How can you tell if a candidate is a good fit for your team?
Adams: There is definitely a base of technology skills that we need them to have, but they also need a lot of problem-solving skills because of how we work. We have very little micromanagement, so people have to make decisions on how to implement things. They have to understand our trading strategies and work with our traders to solve their problems, which requires high-level abstract problem solving skills.
You also need to have a drive to win. This is a really competitive industry, and it’s only getting more competitive.
Do people tend to specialise in specific parts of your technology stack, or do people jump around?
Adams: It's a mix, for multiple reasons. In some places, it's beneficial to have specialist who really knows all the details. But a lot of our systems depend on each other, so it helps to have people who can work across different areas, too.
At the end of the day, we want people working on what they’re passionate about. If you're passionate about what you're working on, you're going to be more engaged, and you’ll have the drive and perseverance to solve problems. You’ll also be happier and you’ll thrive in your role. That’s good for you as an employee, but it’s also good for us because you’ll be more productive.
What do you think is the biggest difference between IMC and other places you’ve worked?
McDougall: Often with engineers, you can run into a problem where they want to refactor things over and over, and it's never done. Usually you have to let it go at some point and move on, because you’ll hit the point of diminishing returns. Here, if you can make something cleaner, prettier, more efficient or easier to read — so long as you’re not making the code uglier and you’re still adhering to good standards — go nuts.
Adams: Our traders really understand that at times we may need to spend some effort on refactoring some code to clean it up, or on improving our test coverage. When I first started here, I remember pointing out a bug to my supervisor. In response, they asked me why I hadn’t already fixed it. If I spot an opportunity to make something better, it's my responsibility to do that.