When I started in the stock market business on the sellside at Lehman Brothers in the 1990s, it felt like the research analysts did extensive work, rigorous analysis and were the market movers the impressionable buy side relied upon. This was before data scraping, during the introduction of excel spreadsheets and after most offices had PC’s linked together by a local area network. I pivoted to the buyside working for one of the largest long short equity managers in New York City. They had similar tools as the sellside, but they ran harder to get the latest data point to determine market and individual stock direction. The race to perform became one of access to management teams which the SEC figured out was essentially inside information and produced regulation FD (full disclosure). Now? Reg FD still exists but data scraping services have replaced the need to speak with management teams as funds deploy data scientists to develop trend change models for almost every type of business.
If you play almost any game on your cell phone or use any app for that matter, the programmers have inserted tracking devices to scrape your habits and predict the trends of the masses at large. Sure, Apple has inserted some privacy settings to negate these trackers, but the dirty little secret is that there are work arounds. Similarly, data scrapers can access publicly available filings such as new capital equipment purchases to determine sales trends at companies such as Caterpillar. If you want to track almost anything, there is now someone who can build a model to do so, as long as the data set exists. With the advent of AI, look for this to become even more precise, predictive and speedier in the years ahead. For this investment funds, mutual or hedge with the largest management fees, this should differentiate performance in the years ahead. Sobering for us mere mortals who trade huh? No, it doesn’t have to be. Let’s discuss.