In the spring of 2017, Yahoo pulled a fast one on the crowd by suddenly changing the technical coding rules for accessing its financial data, leaving countless R users high and dry, including yours truly. Numerous R files that had been meticulously written, revised and maintained over months and years were suddenly broken. Not fun.
As frustrating as that day was (not to mention the weeks that followed), the event served a purpose by dispensing the mother of all wake-up calls on coding matters related to Internet-based data sources. In one fell swoop it became painfully obvious that relying on a single data source was the programming equivalent of driving drunk – you’re going to hit a wall… eventually.
The good news is that we’re wiser now (hopefully). The first rule: always have a back-up plan at the ready. Nothing’s perfect and so it’s critical to plan accordingly. In particular: be prepared to run your code with alternative data sources.
I’m often asked which databases I favor. The short answer: several. That includes my preferred day-to-day source for data dives: Tiingo.com. Why? Several reasons.
First, I’ve found it to be reliable. It doesn’t hurt that Tiingo’s CEO and founder, Rishi Singh, has a habit of answering phone calls and responding to emails.
Superb customer service is one thing, but ultimately it’s about the data when it comes to number-crunching. Fortunately for its users, Tiingo is a tough act to beat on this front, too — whether you’re looking for historical or real-time data. One reason, to quote from the company’s web site:
Each feed is made up of at least 3 data sources on average. This means if any one of them goes down, you keep going.
Sign me up! In the wake of the 2017 Yahoo Finance debacle, redundancy ranks high for my data-related priorities.
Singh elaborates a bit in a recent email, telling The Capital Spectator that “our end-of-day data goes through a proprietary error-checking framework that allows us to detect erroneous data points and also find missing corporate actions and dividends.” And for data users who require the upper echelons of speed for data analytics, he adds: “Our stack is optimized to minimize latency; our cloud-server farm is located about 20 miles from the exchange data centers. This means we can expose the raw binary feeds and shave off milliseconds.”
Another plus: Tiingo seamlessly integrates with R for downloading historical data. Two choices that I routinely deploy: a customized function I wrote (for initial guidance see the API documentation) and using
getSymbols() in the widely used (and highly respected) quantmod package. A simple example:
library(quantmod) symbols <-c("SPY", "AGG") getSymbols(symbols, src = "tiingo", api.key = "ABCDE", # change ABCDE to your Tiingo.com API key auto.assign=TRUE, adjust=FALSE, from="2010-01-01") head(SPY) SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume 2010-01-04 112.37 113.39 111.510 113.33 118857369 2010-01-05 113.26 113.68 112.850 113.63 111519230 2010-01-06 113.52 113.99 113.430 113.71 116017127 2010-01-07 113.50 114.33 113.180 114.19 130502017 2010-01-08 113.89 114.62 113.660 114.57 126357911 2010-01-11 114.57 115.13 114.235 114.73 105905179
Note the use of the
adjust=FALSE argument, which selects unadjusted data. For adjusted data (reflecting distributions) use
As an added bonus, a basic account is free for individuals and institutions. As it turns out, that’s a pretty good deal since zero still provides quite a lot of functionality and pricing history. For details, along with pricing for the premium service, take a look at this summary.
I’ve been known to toggle back and forth between the free and premium subscriptions, depending on how much data I need for analytics and consulting in any given month. Fortunately, there’s no contract and so you can easily move from one to another as your data needs evolve.
Nothing’s 100% and so it’s always prudent to have a Plan B. But as reliability and quality go for data providers, Tiingo’s on my short list.
Learn To Use R For Portfolio Analysis
Quantitative Investment Portfolio Analytics In R:
An Introduction To R For Modeling Portfolio Risk and Return
By James Picerno