Integration

There are several categories of functionality available from 3rd parties providers:

  • Historic data feeds

  • Live data feeds

  • Broker functionality

  • Exchange rates

In some cases a single 3rd party provides all types of functionality, or sometimes they are just offer a subset. You can combine several of these 3rd parties together in a single run, for example using a data feed from one provider and the broker functionality from another one.

One of the design principles is that roboquant tries to enforce is avoiding exposing 3rd party APIs directly. Typically, these APIs are wrapped in roboquant specific APIs and types. This makes it easier to switch between different providers and also reduces the learning curve.

Modules & Packages

The third party providers have their own modules and are not part of the core roboquant module:

  • roboquant-extra for the "traditional" broker and data feed providers

  • roboquant-crypto for cryptocurrency related 3rd parties

  • roboquant-ibkr for Interactive Brokers

You can include these modules in your Maven or Gradle build. In order to use the 3rd party integration, you’ll have to import the package. All third party integration have their own package directly under org.roboquant.

So for example you can import all the required classes for Alpaca as follows:

import org.roboquant.alpaca.*

This wildcard import is especially useful if you use roboquant in a Jupyter Notebook. If you use roboquant in an IDE like IntelliJ IDEA, the IDE will automatically suggest which packages and classes to import for you.

Configuration

When you want to integrate with a 3rd party, you most likely require credentials to gain access to their APIs. Often this is in the form of an API key and/or secret. You can include these credentials directly in your source code. However, this is not considered good practice since it is easy by mistake to version these files and make the credentials available to a wider audience.

So roboquant offers better and more secure alternatives. The following list shows the order of steps roboquant uses to find these configuration parameters:

  1. If the credentials are provided directly in the code when calling the API, use these, otherwise go to next step.

  2. If the credentials are provided as startup parameter to the JVM with the -D syntax, use these, otherwise go to next step.

    java myapp.jar -Dalpaca.secret.key=... -Dalpaca.pubic.key=...

  3. If the credentials are set as system environment variable (export my_key="some value"), use these, otherwise go to next step. In this case use _ and not . in the key names.

    export alpaca_secret_key=...
export alpaca_public_key=...
java myapp.jar

  4. If the credentials are set in a property file called .env in the working directory, use these, otherwise go to next step.

  5. If the credentials are set in a property file called dotenv in the working directory, use these, otherwise go to next step. The reason to support this variation is that .env files are hidden in Jupyter notebook directories and then dotenv is a good alternative.

  6. If the credentials are set in a property file called .env in the $HOME/.roboquant directory, use these, otherwise not found.

The following provides an overview of the used properties in roboquant:

# This property file contains configuration settings that will be made available to roboquant
# Many data feed providers and brokers require one or more keys to be able to access their APIs,
# and this is one of the ways to provide those credentials.
#
# If you store credentials in here, make sure other people don't have access to it and
# don't version control it to a public repo by mistake.
# One solution is to place this file at ~/.roboquant/.env

# Sample entry
# sample.public.key=some_key_value
# sample.secret.key=another_key_value

# Uncomment the following two lines and change the values for Alpaca API access
# alpaca.public.key=your_api_key_id
# alpaca.secret.key=your_api_secret_key

# Uncomment the following line and change the value for Alpha Vantage access
# alphavantage.key=your_api_key

# Uncomment the following two lines and change the values for Binance access
# binance.public.key=your_key
# binance.secret.key=you_secret

# Uncomment the following line and change the value for Polygon.io access
# polygon.key=your_key

Supported 3rd parties

The following table shows which 3rd parties are supported out of the box and what functionality the integration offers:

3rd party Type Historic feed Live feed Broker Exchange rates

Interactive Brokers

API

✓

✓

✓

✓

Alpaca

API

✓

✓

✓

Polygon.io

API

✓

✓

Alpha Vantage

API

✓

Binance

API

✓

✓

✓

XChange (60+ crypto exchanges)

API

✓

✓

✓

ECB

CSV File

✓

Stooq

CSV File

✓

MT5

CSV File

✓

HistData

CSV File

✓

Kraken

CSV File

✓

Yahoo Finance

CSV File

✓
Roboquant is open source software that is developed independently of these 3rd party companies and has no associations with them. There is also no direct on indirect recommendation using them.

Alpaca

The integration with Alpaca covers their broker API as well as their historic and live market data APIs. In order to use these, you’ll need at least register for a paper-trading account and create an API-key.

val feed = AlpacaHistoricFeed()
val tf = Timeframe.past(100.days)

// You can retrieve historic price-bars, quotes or trades
feed.retrieveStockPriceBars("AAPL", "JPM", "TSLA", timeframe = tf)
feed.retrieveStockQuotes("AAPL", "JPM", "TSLA", timeframe = tf)
feed.retrieveStockTrades("AAPL", "JPM", "TSLA", timeframe = tf)
val feed = AlpacaLiveFeed()
feed.subscribeStocks("AAPL", "IBM")

val tf = Timeframe.next(120.minutes)
roboquant.run(feed,tf)
// instantiation with hard coded configuration, rather than using dotenv property file
val broker = AlpacaBroker {
    publicKey = "123"
    secretKey = "456"
    accountType = AccountType.PAPER
    dataType = DataAPIType.IEX
}
println(broker.account.summary())
println(broker.availableAssets)

// place a market order to buy 100 stocks Apple
val order = MarketOrder(Asset("AAPL"), 100)
broker.place(listOf(order))

Polygon

Polygon has several type of subscriptions available, depending on your needs. The free version only provides access to historic data, while the paid versions also provide live data.

val feed = PolygonHistoricFeed()
val tf = Timeframe.past(100.days)
feed.retrieve("AAPL", "JPM", "TSLA", timeframe = tf)
val feed = PolygonLiveFeed()
feed.subscribe("AAPL", "JPM", "TSLA")

val tf = Timeframe.next(120.minutes)
roboquant.run(feed,tf)

ECB

The ECB (European Central Bank) publishes since the start of the Euro the daily historic exchange rates with almost all other fiat currencies in the world. ECBExchangeRates will automatically download the exchange rates from their website and use them for currency conversions during trading. There is no need for registration or an API key, since this data is available on their public website.

// Download the latest rates from the ECB website
// The results are cached by default
Config.exchangeRates = ECBExchangeRates.fromWeb()

// Create a wallet holding different currencies
val wallet = 100.EUR + 20.USD + 1000.JPY

// convert the wallet to GBP at today's exchange rates
wallet.convert(Currency.GBP)

//  convert the wallet to GBP using 5 years ago exchange rates
wallet.convert(Currency.GBP, Instant.now() - 5.years)

Binance

The integration with Binance covers their broker API as well as their historic and live market data APIs. From an algo-trading perspective, it is also nice to see that the access to (high-frequency) market data is free. Even without a trading account with them, you can still access the data.

val feed = BinanceHistoricFeed()
println(feed.availableAssets.summary())

// Retrieve 1-minute candlesticks for two currency pairs for the last day
val timeframe = Timeframe.past(1.days)
feed.retrieve("BTCBUSD", "ETHBUSD", timeframe = timeframe, interval = Interval.ONE_MINUTE)

To subscribe to live quotes for one or more currency pairs, is easy

val feed = BinanceLiveFeed()
println(feed.availableAssets.summary())

// subscribe to live quotes for two currency pairs
feed.subscribePriceQuote("BTCBUSD", "ETHBUSD")

Currently, the broker integration with Binance is largely untested and core features are disabled to avoid costly bugs. People have been using it as a starting point for their own implementation, but the included implementation should not be relied upon yet.

val broker = BinanceBroker {
    publicKey = "123"
    secretKey = "456"
}
println(broker.account.fullSummary())

XChange

XChange is a library providing a consistent API for interacting with 60+ Bitcoin and other cryptocurrency exchanges, providing an interface for trading and accessing market data.

Roboquant integrates with XChange for retrieving historic data, live data and accessing trading functionality. Because there is support for so many crypto exchanges, the initial setup is a bit more elaborate than with other 3rd party providers.

You’ll always have to add the additional exchange specific libraries to your project. We’ll use Maven and Bitstamp as an example:

<dependencies>
    <dependency>
        <groupId>org.knowm.xchange</groupId>
        <artifactId>xchange-bitstamp</artifactId>
        <version>5.1.0</version>
    </dependency>
    <dependency>
        <groupId>org.knowm.xchange</groupId>
        <artifactId>xchange-stream-bitstamp</artifactId>
        <version>5.1.0</version>
    </dependency>
</dependencies>

To get live data, you’ll first need to acquire an instance of a StreamingExchange. Again using Bitstamp as an example, you can use the following code snippet:

val exchange = StreamingExchangeFactory.INSTANCE.createExchange(BitstampStreamingExchange::class.java)

// Connect to the Exchange WebSocket API. Here we use a blocking wait.
exchange.connect().blockingAwait()

Now you are ready to create an instance of XChangeLiveFeed and subscribe to one or more currency pairs.

val feed = XChangeLiveFeed(exchange)

// subscribe to live quotes for two currency pairs
feed.subscribeTicker("BTC_USD", "ETH_USD")

Alpha Vantage

Alpha Vantage provides a rich set of historic stock prices, not only for stocks listed on US exchanges. Although Alpha Vantage also provides other information besides stock market data, that is not yet supported by the AlphaVantageHistoricFeed.

val feed = AlphaVantageHistoricFeed()

// You can retrieve end of day prices
val assets = listOf(
    // regular US stock
    Asset("AAPL"),

    // stock listed on a non-US exchange
    Asset("DAI.DEX", currency = Currency.EUR, exchange = Exchange.DEX)
)
feed.retrieveDaily(*assets.toTypedArray())

// You can retrieve intra-day prices
feed.retrieveIntraday(Asset("TSLA"), interval = Interval.ONE_MIN)

Interactive Brokers

Interactive Brokers integration requires to run either TWS or IB Gateway on a local machine. So you cannot directly connect from roboquant to their servers.

Other than that, the integration is the same as other brokers and data feeds.

val feed = IBKRHistoricFeed()
val asset = Asset("TSLA")
feed.retrieve(asset)
roboquant.run(feed)

If you run your software in a Docker container and the IB Gateway is running on the host, you still need to be able to connect from your software to the IB Gateway.

ibkr install

One way to achieve this, is to set the IBKR host property to "host.docker.internal" in your software.

Config.setProperty("ibkr.host", "host.docker.internal")

Adding new 3rd party integrations

Adding integration with a 3rd party is not difficult. Integration with 3rd party data providers is straight forward and requires implementing a single interface. Depending on the client library that is available, this could take as little as a few hours to a few days. You can read more about the Feed interface that you need to implement it right here.

Integrating with a new broker is not that difficult either, but requires more effort. Most of the work will be the translation of object exposed by the broker to those used by roboquant. For example an Order in roboquant will be different from an Order expected by the client library of the broker, and a conversion needs to happen in both directions.

For both custom Feed and Broker integrations, it is recommended to first look at some existing integrations and use them as a starting point. And if you have questions, you can always reach out via one of the channels mentioned on the community page.