Retrieving Historical Stock Data in C++

Examples abound of multiple different ways to retrieve historical stock data in Python using many different sources. I have personally written about an API I created which downloads the data from Yahoo Finance and have released a project to do so (available via pip). A quick search of the same functionality reveals a few projects to perform the task in C++ including some GitHub projects and other code samples. However, I am looking for a way to replace my Python stock/time-series analysis pipeline with corresponding C++ functionality. This lofty goal has lead to the creation of a very primitive tabular data structure written in C++ available here and discussed here (note the project is in a very early stage and is under development, hence not EXTREMELY useful).

With my new tabular data structure in hand (DataTables), my goal now is to replace the data retrieval aspect of my pipeline (visualization and modeling being the last two stepping stones). In this post, I will discuss the creation of a YahooFinanceAPI class written in C++ that uses libcurl (a client-side URL transfer library). I will provide example usage of the program to show how easily the historical data can be downloaded and will also develop methods to load the data directly into a DataTable.

Yahoo Finance Data

Downloading data from Yahoo Finance is easily done via the URL tied to the download link on the Historical Data portion of their website. A general template of the link looks something like this

https://query1.finance.yahoo.com/v7/finance/download/{ticker}?period1={start_time}&period2={end_time}&interval={interval}&events=history

where ticker is replaced with the ticker symbol of interest, start_time and end_time are Unix timestamps for the start and end date of the data to be retrieved, and interval is replaced with “1d” (daily), “1wk” (weekly), or “1mo” (monthly) and determines the frequency of the data. Replacing these values in the URL allows us to use curl to download the file that would result from the query.

Dependencies

libcurl

To be able to use libcurl’s C API we will need to install the library. This can be done on Ubuntu (and probably any Debian-based Linux system) via:

sudo apt install libcurl4-openssl-dev

I haven’t used libcurl on any other platforms but here is a link to the installation page for the project.

DataTables

Likewise, if you’re interested in using the DataTables class that I’ve created that will also need to be installed. Installation instructions can be found on the project’s GitHub page. However, I know many people will find it limiting to only be able to utilize the API by using the DataTable structure so functionality is presented below that circumvents the need for installing DataTables.

Note that, if you choose not to install DataTables you will need to remove the C++ logic that references the library to avoid build errors.

The Class

The code for the data download and processing is stored in a YahooFinanceAPI class. The class definition is shown below.

class YahooFinanceAPI
{
    public:
        YahooFinanceAPI();
        void set_interval(Interval interval);
        void set_col_name(std::string col_name);
        datatable::DataTable get_ticker_data(std::string ticker, 
            std::string start_date, std::string end_date, 
            bool keep_file=false);
        std::string download_ticker_data(std::string ticker, std::string
            start_date, std::string end_date);

    private:
        std::string _base_url;
        Interval _interval;
        std::string _col_name;

        std::string build_url(std::string ticker, std::string start_date,
            std::string end_date);
        bool string_replace(std::string& str, const std::string from,
            const std::string to);
        std::string timestamp_from_string(std::string date);
        void download_file(std::string url, std::string filename);
};

As seen here, the class is very simple. Functionality is provided to return ticker data loaded into a DataTable object or to download the data to a CSV file. Downloading to a CSV file does not require the installation/usage of the DataTables library. If using DataTables, the set_column() function provides a way to set the column to be used as the response column in the DataTable. The last function, set_interval(), allows the frequency of the data to be set on the fly. This uses an enum and a method to get the string value for the URL construction both of which are shown below.

#include <string.h>

enum Interval 
{
    WEEKLY,
    MONTHLY,
    DAILY
};

static const std::string EnumAPIValues[] { "1wk", "1mo", "1d" };
std::string get_api_interval_value(int value)
{
    return EnumAPIValues[value];
}

The class implementation is ~100 LOC so I won’t reproduce it here. However, as usual, the full code for the project is provided at the end of this post.

Example Usage

Some basic usage of the API is demonstrated in the code below.

#include "yfapi.hpp"

using namespace std;

int main(int argc, char* argv[])
{
    // create an API object which lives in the yfapi namespace
    yfapi::YahooFinanceAPI api; 
    return 0;
}

To begin we need to create an instance of the YahooFinanceAPI class. This class, as seen below, lives in the yfapi namespace. With the class initialized we can create DataTables, download the data as a CSV file, or both.

#include "yfapi.hpp"

using namespace std;

int main(int argc, char* argv[])
{
    // create an API object which lives in the yfapi namespace
    yfapi::YahooFinanceAPI api; 

    // get 1 month worth of data for the ticker symbol 'spy'
    // return the data in a DataTable and delete the file afterwards
    datatable::DataTable dt = api.get_ticker_data("spy", "2020-09-01", "2020-10-01");
    // datetime not supported in DataTables 
    // (https://github.com/anthonymorast/DataTables/issues/5)
    dt.drop_columns(new string[1]{"Date"}, 1);  
    
    // display the amount of data and the names of the headers for the data
    dt.print_shape(cout);
    dt.print_headers(cout);

    // this function will download the YTD data for ticker 'qqq'
    // the data is saved as "<ticker>_<timestamp>.csv" to prevent 
    // overwriting the files.
    api.download_ticker_data("qqq", "2020-01-01", "2020-10-07");

    // now the interval is set to MONTHLY so the stock data 
    // for the first day of every month is retrieved from Yahoo Finance
    api.set_interval(MONTHLY);

    // in this instance, the stock data for Apple is retrieved and 
    // returned as a DataTable. However, since keep_file is set to
    // 'true', the file will not be deleted and will be named as 
    // described above.
    datatable::DataTable dt2 = api.get_ticker_data("aapl", "2010-01-01", 
                "2020-10-01", true);

    return 0;
}

After reading the comments above, there’s not much left to say for the code’s functionality. The API is very simple to set up and use so one can quickly get started analyzing the time-series or finding any secret trends in the data. The YahooFinanaceAPI’s code is provided below and free to use.

Full Code

Interval.hpp

#include <string.h>

enum Interval 
{
    WEEKLY,
    MONTHLY,
    DAILY
};

static const std::string EnumAPIValues[] { "1wk", "1mo", "1d" };

std::string get_api_interval_value(int value)
{
    return EnumAPIValues[value];
}

yfapi.hpp

#include <string.h>
#include <curl/curl.h>
#include <ctime>
#include <DataTable/DataTable.hpp>
#include <iomanip>
#include <sstream>
#include <iostream>
#include <stdio.h>

#include "interval.hpp"



namespace yfapi
{
    class YahooFinanceAPI
    {
        public:
            YahooFinanceAPI();
            void set_interval(Interval interval);
            void set_col_name(std::string col_name);
            datatable::DataTable get_ticker_data(std::string ticker, std::string start_date, std::string end_date, bool keep_file=false);
            std::string download_ticker_data(std::string ticker, std::string start_date, std::string end_date);

        private:
            std::string _base_url;
            Interval _interval;
            std::string _col_name;

            std::string build_url(std::string ticker, std::string start_date, std::string end_date);
            bool string_replace(std::string& str, const std::string from, const std::string to);
            std::string timestamp_from_string(std::string date);
            void download_file(std::string url, std::string filename);
    };

    YahooFinanceAPI::YahooFinanceAPI()
    {
        this->_base_url =
            "https://query1.finance.yahoo.com/v7/finance/download/{ticker}?period1={start_time}&period2={end_time}&interval={interval}&events=history";
        this->_interval = DAILY;
        this->_col_name = "Open";
    }

    std::string YahooFinanceAPI::timestamp_from_string(std::string date)
    {
        struct std::tm time = {0,0,0,0,0,0,0,0,0};
        std::istringstream ss(date);
        ss >> std::get_time(&time, "%Y-%m-%d");
        if(ss.fail())
        {
            std::cerr  << "ERROR: Cannot parse date string (" << date <<"); required format %Y-%m-%d" << std::endl;
            exit(1);
        }
        time.tm_hour = 0;
        time.tm_min = 0;
        time.tm_sec = 0;
        std::time_t epoch = std::mktime(&time);

        return std::to_string(epoch);
    }

    bool YahooFinanceAPI::string_replace(std::string& str, const std::string from, const std::string to)
    {
        size_t start = str.find(from);
        if(start == std::string::npos)
        {
            return false;
        }
        str.replace(start, from.length(), to);
        return true;
    }

    std::string YahooFinanceAPI::build_url(std::string ticker, std::string start, std::string end)
    {
        std::string url = this->_base_url;
        string_replace(url, "{ticker}", ticker);
        string_replace(url, "{start_time}", timestamp_from_string(start));
        string_replace(url, "{end_time}", timestamp_from_string(end));
        string_replace(url, "{interval}", get_api_interval_value(this->_interval));
        return url;
    }


    void YahooFinanceAPI::set_interval(Interval interval)
    {
        this->_interval = interval;
    }

    void YahooFinanceAPI::set_col_name(std::string name)
    {
        this->_col_name = name;
    }

    void YahooFinanceAPI::download_file(std::string url, std::string filename)
    {
        CURL *curl;
        FILE *fp;
        CURLcode res;
        curl = curl_easy_init();
        if (curl)
        {
            fp = fopen(filename.c_str(), "wb");
            curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
            curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, NULL);
            curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
            res = curl_easy_perform(curl);

            /* always cleanup */
            curl_easy_cleanup(curl);
            fclose(fp);
        }
    }

    /*
     * Returns a DataTable containing data downloaded from YahooFinance.
     * By default this method will delete the tmp file created by the
     * download.
     */
    datatable::DataTable YahooFinanceAPI::get_ticker_data(std::string ticker, std::string start, std::string end, bool keep_file)
    {
        std::string url = build_url(ticker, start, end);
        datatable::DataTable dt;
        std::time_t now = std::time(0); // now
        std::string output_file_name = ticker + "_" + std::to_string(now) + ".csv";
        if(!keep_file)
            output_file_name = "tmp_" + output_file_name;

        download_file(url, output_file_name);
        dt = datatable::DataTable(output_file_name, this->_col_name, true);

        if(!keep_file)
            std::remove(output_file_name.c_str());
        return dt;
    }

    /*
     * Downloads the historical stock data and returns the name of the file
     * created by the download.
     */
    std::string YahooFinanceAPI::download_ticker_data(std::string ticker, std::string start, std::string end)
    {
        std::string url = build_url(ticker, start, end);
        std::time_t now = std::time(0);
        std::string output_file_name = ticker + "_" + std::to_string(now) + ".csv";

        download_file(url, output_file_name);

        return output_file_name;
    }
}

usage.cpp

#include "yfapi.hpp"

using namespace std;

int main(int argc, char* argv[])
{
    yfapi::YahooFinanceAPI api; 
    datatable::DataTable dt = api.get_ticker_data("spy", "2020-09-01", "2020-10-06");
    // datetime not supported in DataTables (https://github.com/anthonymorast/DataTables/issues/5)
    dt.drop_columns(new string[1]{"Date"}, 1);  
    dt.print_shape(cout);
    dt.print_headers(cout);


    api.download_ticker_data("qqq", "2020-01-01", "2020-10-07");

    api.set_interval(MONTHLY);
    datatable::DataTable dt2 = api.get_ticker_data("aapl", "2010-01-01", "2020-10-01", true);

    return 0;
}

Makefile

all: example

example:
	g++ usage.cpp -lcurl -lDataTable -o example -O3

clean: 
	rm example

Leave a Reply

Your email address will not be published. Required fields are marked *