Excerpt from my book All About High Frequency Trading:


I always wondered when word would get out.

I got my own look in 2003. That’s when I went to work for the Citadel Investment Group, Ken Griffin’s stealthy Chicago hedge fund, to help them build a high-frequency trading system for exchange-traded equity options. There was only one fully automated options exchange in the United States at that time, the International Securities Exchange in New York. Before the ISE opened for business in 2000, no options exchange would allow market-makers to submit their all-important quotations–bids to buy and offers to sell–electronically. For the most part, those were still communicated verbally by human traders with loud voices and sharp elbows, standing all day in open outcry trading pits wearing sensible shoes. ISE founders David Krell and Gary Katz knew it was time to change that, and the success of their enterprise proved to any remaining doubters how absolutely right they were. The ISE was the quintessential game changer. Their explosive success forced the traditional, floor-based options exchanges to make their own plans for electronic quoting. And Ken Griffin wanted Citadel to be all over it.

I had been managing financial systems development for several years by this time, mostly for the pricing of derivative securities. The Citadel system, though, would not only calculate hundreds of thousands of option prices simultaneously–an impressive feat in its own right–but also inject streams of bids and offers into the markets at literally superhuman speed. The custom-built quoting engines would tirelessly inject many millions of quotes into the markets every day, each of them a binding commitment to buy or sell a listed option contract at some specified price, each one the result of a software program running on a computer. And while the quoters were busy doing that, “electronic eyes” would scan everyone else’s quotations and orders–hundreds of millions per day–all in real time. It would be like standing at the end of an open fire hose and examining each drop of water before it hit the ground. When the electronic eye (or EE) found someone offering to buy an option for more than it was worth, or to sell it for less, it would immediately submit an order to take the other side of the trade for a tiny profit.

It was a dazzling sight, watching these machines pick the markets clean of its inefficiencies. I would have loved to talk about it back then, to tell friends and family what was going on in the gleaming glass tower at the intersection of Dearborn and Adams. But the confidentiality agreements one has to sign for employers like Citadel are very, very effective. In this business, everyone knows that loose lips get pink slips. So like everyone else, I kept my mouth shut and talked only with my small group of colleagues on the 37th floor. Even within the sanctum sanctorum at Citadel, information was purely need-to-know. I once asked a quantitative analyst about one of the factors that went into the all-important volatility model. “What does ‘v’ mean?” “It means ‘v,'” she replied. “Ahh. What about ‘h’?” “It means ‘h.'” This little badinage went on for quite a few more letters of the alphabet.

By the time I left Citadel in 2005, their options market-making system–the work of a team not much larger than the Chicago Cubs starting lineup–was responsible for more than 10 percent of all options trading in the United States, or more than a million contracts a day. Within three years, its market share had reportedly grown to a commanding 30 percent. The U.S. options market had become dominated by the extraordinary machines of just a handful of secretive firms like Citadel. Still, nobody on the outside seemed to have a clue–or a care–that trading was no longer done by traders.

That all changed in 2009.

As people licked their wounds in the aftermath of the 2008 market meltdown, wondering where all the money went, word got out that something like $20 billion of it went to these folks known as high-frequency traders. The term was well known inside firms like Citadel but not so much outside. Now, it was bad enough that anyone made out like bandits in the horrible year that was 2008, but a far more frightening contemplation caused more than a few people to go grab a pitchfork from the shed: Was high-frequency trading somehow culpable? Did it cause the mother of all crashes, or at least accelerate it once it began? After all, the 1987 market crash was widely attributed to automated trading, then known as program trading. Did the computers do it again?

Word went around that 50 percent of all stock trading–maybe 60 percent or even 70 percent–was attributable to HFT computers trading with each other, supposedly just to collect tiny kickbacks, known as rebates, from the exchange. The HFT firms weren’t even holding onto their stock. Once bought, they’d immediately turn around and sell it, sometimes buying and selling the same stock hundreds or thousands of times a day. What was up with that? All this trading at ungodly speeds, it was said, was creating massive price volatility that otherwise wouldn’t exist. Could this be good? Nerves were not settled when a former Goldman Sachs employee was arrested for allegedly stealing proprietary computer code for high-frequency trading, with the bank asserting ominously “there is a danger that somebody who (knows) how to use this program could use it to manipulate markets in unfair ways.” Computer code to manipulate markets? What the hell was going on here?

Anyone following the HFT stories in 2008 learned a handful of new terms from the modern trading lexicon–none of them particularly comforting. The HFT firms were supposedly using something called “flash orders” to get advance looks at customer trade orders before the rest of the market, then using those peeks to make their own trades at a profit. Wasn’t that front-running and wasn’t it illegal? The flash order robbers supposedly had only 30 milliseconds to do their dirty work, but this was plenty of time because they “colocated” their computer servers in the same data centers as the exchange computers, at great expense. This also let them get their own orders in before any investor possibly could. Uneven playing field, anybody?

Unsatisfied with flash order thievery, the HFT smarties supposedly submitted something called IOC orders with no intention of trading, but only to force investors to reveal the true prices at which they were willing to trade, information the HFT guys would use to move market prices against the investor. Whoa. Were the HFT firms even qualified to be so close the exchange and trade at lightning speeds? Nobody could say, because it was nearly impossible to know even the identities of high-frequency traders. They didn’t need to make the infrastructure investments themselves. They could use “direct market access” or “naked access,” using their broker’s exchange connection to get in anonymously, then perform their lightning-fast derring-do as if wearing a mask. SEC Chairman Mary Schapiro likens the practice of DMA to lending the car keys to your unregistered Ferrari to someone who may not even be licensed to drive. It’s not a bad analogy.

“Dude,” you could almost hear people asking, fatigued and more than a little ticked off, “What happened to our stock market?” Was it no longer what it used to be, a place to simply invest in companies with the idea of holding onto that stock for a while? Were we all naive to still think like that? Maybe we had all been reduced to easy marks for sharpies with fast computers and math skills far better than our own, like dummies on the boardwalk, sized up and taken by the hucksters. Do the markets still work? Or have they been hijacked by cutthroat information technology and greed?

It can sure seem that way.

The year 2008 was indeed the year of wonders for high-frequency traders, especially options traders, and this struck plenty of folks as somehow wrong. A head-hunter told me that his client, a high-frequency options market-making firm, had made over $800 million in 2008. Another well-known firm was said to have cleared $1.3 billion of net profit–and that was just trading options. Who knows what the stock HFT desk pulled in. Now just because some people made out like kings in 2008 when most people suffered, however, does not mean all those gains were ill-gotten. When technology revolutionizes an industry, be it personal computing or automobile manufacturing or oil refining, it’s not unheard-of for a small number of pioneers (Gates, Ford, Rockefeller) to make bazillions from their investments.

There’s more mileage to be had from the automobile analogy, this time relating to safety. When horseless buggies first hit the streets in the early twentieth century, top speeds were on the order of 10 or 15 miles per hour, and there weren’t all that many of these novelties on the road. As technology, demand, and free-market enterprise found their confluence, however, those top speeds climbed steadily and the roads started filling. Cars crashed, people died, and the lethality of these contraptions became frightfully obvious. Government and industry began addressing safety in the design and regulation of automobiles. Speed limits were posted, seat belts were invented, laws were written, and cops started writing tickets. It’s not unreasonable to say that high-frequency trading requires that same sort of rethinking to keep people safe in light of the new capabilities brought on by technology.

And there is yet one more obvious parallel between the automotive revolution and HFT. Some folks in the early years of the automobile saw cars not as conveniences but as tools to help commit crime. Would John Dillinger have fared so well were his getaway vehicle a trolley? Bank robbers used cars to get away with crimes they might not have otherwise. In turn, the cops themselves were equipped with better and better cars and laws were expanded and revised accordingly. Can people use the tools of high-frequency trading to get away with things they might not have, say, ten years ago? It’s not inconceivable. Is it time to reconsider regulations and law enforcement in the securities markets? Probably.

It must be said that pulling off the high-frequency part of high-frequency trading is no stroll through the mall. It’s exceedingly difficult setting up an HFT system that actually works, and there is no one thing to be done, no single task to master, any more than there is only one thing to ensure the successful construction of an ocean liner. High-frequency traders leave no stone unturned in pursuit of their wispy profits, starting with the hiring of just the right number of rock stars from the fields of trading, mathematics, and software system development. They write their own software rather than license packaged products from third parties, investing the time and money required for a system catered to their exact needs, one they can fix and modify on their own schedule. Oh, and for the building and maintaining of these systematic goliaths, they spend sums of money that would make even Warren Buffet raise his eyebrows.

HFT firms apply software engineering practices that facilitate the development of so-called distributed, real-time systems, borrowing patterns from the field of complex event processing, with thousands of individual programs running on just the right number of computers in just the right number of data centers. They buck the decades-long trend of packing more and more processing onto a computer’s CPU, favoring the seemingly backward approach of delegating some computing tasks to specialized hardware. They even take over the massively parallel processing capability of graphical processing units–game cards, in essence–for financial computations. An HFT firm would not dream of receiving market data–the crucial flow of “ticks”–or submitting trade orders by way of third parties and their latency-consuming connections. They insist on direct connections to the exchange for these purposes. They also know where the exchanges house their computers, or matching engines, and collocate their own computers in the very same data centers, thus getting their work done just a few billionths of a second sooner than the next guy.

Exhausting? Expensive? You betcha. But with the prospect of rolling a billion samolians a year off of one these money-making machines, it’s no wonder more than one firm is willing to pay whatever it takes to have one.