Integration

There are several categories of functionality available from 3rd party providers:

Historic data feeds
Live data feeds
Broker functionality
Exchange rates

In some cases, a single 3rd party provides all types of functionality, or sometimes they are just offer a subset. You can combine several of these 3rd parties together in a single run, for example, using a data feed from one provider and the broker functionality from another one.

One of the design principles is that roboquant tries to enforce is avoiding exposing 3rd party APIs directly. Typically, these APIs are wrapped in roboquant specific APIs and types. This makes it easier to switch between different providers and also reduces the learning curve.

Modules & Packages

The third party providers have their own modules and are not part of the core roboquant module. For example:

roboquant-alpaca for the Alpaca broker and data feed integration
roboquant-binance for Binance crypto-exchange integration

You can see all the modules at Maven Central

You can include these modules in your Maven or Gradle build. To use the 3rd party integration, you’ll have to import the package. All third party integrations have their own package directly under org.roboquant.

So for example, you can import all the required classes for Alpaca as follows:

This wildcard import is especially useful if you use roboquant in a Jupyter Notebook. If you use roboquant in an IDE like IntelliJ IDEA, the IDE will automatically suggest which packages and classes to import for you.

Configuration

When you want to integrate with a 3rd party, you most likely require credentials to gain access to their APIs. Often this is in the form of an API key and/or secret. You can include these credentials directly in your source code. However, this is not considered good practice since it is easy by mistake to version these files and make the credentials available to a wider audience.

So roboquant offers better and more secure alternatives. The following list shows the order of steps roboquant uses to find these configuration parameters:

If the credentials are provided directly in the code when calling the API, use these, otherwise go to the next step.

// instantiation with hard-coded configuration
val feed = AlpacaHistoricFeed {
    publicKey = "123"
    secretKey = "456"
    dataType = DataType.IEX
}

If the credentials are provided as startup parameter to the JVM with the -D syntax, use these, otherwise go to the next step.
```
java myapp.jar -Dalpaca.secret.key=... -Dalpaca.pubic.key=...
```
If the credentials are set as system environment variable (export my_key="some value"), use these, otherwise go to the next step. In this case use _ and not . in the key names.
```
export alpaca_secret_key=...
export alpaca_public_key=...
java myapp.jar
```
If the credentials are set in a property file called .env in the working directory, use these, otherwise go to next step.
If the credentials are set in a property file called dotenv in the working directory, use these, otherwise go to next step. The reason to support this variation is that .env files are hidden in Jupyter notebook directories and then dotenv is a good alternative.
If the credentials are set in a property file called .env in the $HOME/.roboquant directory, use these, otherwise not found.

The following provides an overview of the used properties in roboquant:

# This property file contains configuration settings that will be made available to roboquant
# Many data feed providers and brokers require one or more keys to be able to access their APIs,
# and this is one of the ways to provide those credentials.
#
# If you store credentials in here, make sure other people don't have access to it and
# don't version control it to a public repo by mistake.
# One solution is to place this file at ~/.roboquant/.env

# Sample entry
# sample.public.key=some_key_value
# sample.secret.key=another_key_value

# Uncomment the following two lines and change the values for Alpaca API access
# alpaca.public.key=your_api_key_id
# alpaca.secret.key=your_api_secret_key

# Uncomment the following line and change the value for Alpha Vantage access
# alphavantage.key=your_api_key

# Uncomment the following two lines and change the values for Binance access
# binance.public.key=your_key
# binance.secret.key=you_secret

# Uncomment the following line and change the value for Polygon.io access
# polygon.key=your_key

Supported 3rd parties

The following table shows the 3rd parties that are supported out of the box and what functionality the integration offers:

3rd party	Type	Historic feed	Live feed	Broker	Exchange rates
Interactive Brokers	API	✓	✓	✓	✓
Alpaca	API	✓	✓	✓
ECB	CSV File				✓
Stooq	CSV File	✓
MT5	CSV File	✓
HistData	CSV File	✓
Kraken	CSV File	✓
Yahoo Finance	CSV File	✓

3rd party

Type

Historic feed

Live feed

Broker

Exchange rates

Interactive Brokers

API

✓

Alpaca

API

✓

ECB

CSV File

✓

Stooq

CSV File

✓

MT5

CSV File

✓

HistData

CSV File

✓

Kraken

CSV File

✓

Yahoo Finance

CSV File

✓

Roboquant is open-source software that is developed independently of these 3rd party companies and has no associations with them. There is also no direct or indirect recommendation using them.

Alpaca

The integration with Alpaca covers their broker API as well as their historic and live market data APIs. To use this, you’ll need at least to register for a paper-trading account and create an API key.

val feed = AlpacaHistoricFeed()
val tf = Timeframe.past(100.days)

// You can retrieve historic price-bars, quotes or trades
feed.retrieveStockPriceBars("AAPL,JPM,TSLA", timeframe = tf)
feed.retrieveStockQuotes("AAPL,JPM,TSLA", timeframe = tf)
feed.retrieveStockTrades("AAPL,JPM,TSLA", timeframe = tf)

val feed = AlpacaLiveFeed()
feed.subscribeStocks("AAPL", "IBM")

// instantiation with hard-coded configuration, rather than using dotenv property file
val broker = AlpacaBroker {
    publicKey = "123"
    secretKey = "456"
    accountType = AccountType.PAPER
    dataType = DataType.IEX
}
println(broker.sync())

// place a market order to buy 100 stocks Apple
val order = Order(Stock("AAPL"), Size(100), 100.0)
broker.placeOrders(listOf(order))

ECB

The ECB (European Central Bank) has published since the start of the Euro the daily historic exchange rates with almost all other fiat currencies in the world. ECBExchangeRates will automatically download the exchange rates from their website and use them for currency conversions during trading. There is no need for registration or an API key, since this data is available on their public website.

// Download the latest rates from the ECB website
// The results are cached by default
Amount.registerConvertor(ECBExchangeRates.fromWeb())

// Create a wallet holding different currencies
val wallet = 100.EUR + 20.USD + 1000.JPY

// convert the wallet to GBP at today's exchange rates
wallet.convert(Currency.GBP, Instant.now())

//  convert the wallet to GBP using 5 years ago exchange rates
wallet.convert(Currency.GBP, Instant.now() - 5.years)

Interactive Brokers

Interactive Brokers integration requires running either TWS or IB Gateway on a local machine. So you cannot directly connect from roboquant to the IBKR servers. Other than that, the integration is the same as other brokers and data feeds.

IB Gateway provides some benefits over TWS, in that you can more easily monitor the messages being exchanged, and it is a lighter on resources.

val feed = IBKRHistoricFeed()
val asset = Stock("TSLA")
feed.retrieve(asset)

If you run your software in a Docker container and the IB Gateway is running on the host, you still need to be able to connect from your software to the IB Gateway.

One way to achieve this, is to set the IBKR host property to "host.docker.internal" in your application.

Config.setProperty("ibkr.host", "host.docker.internal")

Adding new 3rd party integrations

Adding integration with a 3rd party is not difficult. Integration with 3rd party data providers is straight forward and requires implementing a single interface. Depending on the client library that is available, this could take as little as a few hours to a few days.

For both custom Feed and Broker integrations, it is recommended to first look at some existing integrations and use them as a starting point. And if you have questions, you can always reach out via one of the channels mentioned on the community page.

Feed integration

You can read more about the Feed interface that you need to implement it right here.

Broker integration

Integrating with a new broker is not that difficult either, but requires more effort. Most of the work will be the translation of object exposed by the broker to those used by roboquant. For example, an Order in roboquant will be different from an Order expected by the client library of the broker, and a conversion needs to happen in both directions.

To get started, it is most helpful to look at one of the existing implementations, like the BinanceBroker or IBKRBroker. This will give you a good idea of how to approach this.

A few things that might make the task easier:

You can use the InternalAccount class to store the state in the broker implementation. It makes it easy to create an Account instance.
BuyingPower tells you how much money there is to place new orders. You have to make sure that this value is correct.
Open Orders and Positions are also important to decide what to do in your logic and should reflect the latest status.
Closed orders, Cash and Trades are typically not required when making decisions for new orders. So if you don’t map these, typically a few metrics might be wrong, but it should not impact your overall trading logic.

The following snippet can serve as a template:

class MyBroker : Broker {

    private val api = BrokerApi()
    private var id = 0

    // Internal Account is a mutable version of Account
    private val iAccount = InternalAccount(baseCurrency = Currency.USD)


    override fun sync(event: Event?) : Account {
        // All the hard work goes here where you sync the InternalAccount object with your broker
        // The api calls are all fictitious examples

        val now = Instant.now()

        // Optional check that this broker isn't used in a back test
        if (event != null  && event.time < Instant.now() - 1.hours)
            throw UnsupportedException("Cannot place old orders")

        // Optional check to avoid to many calls to the broker
        if (iAccount.orders.isEmpty() && (now < iAccount.lastUpdate + 1.minutes)) return iAccount.toAccount()

        // Sync the positions
        iAccount.positions.clear()
        for (position in api.getPositions()) {
            TODO()
            // iAccount.setPosition(...)
        }

        // Sync the open orders
        for (order in iAccount.orders) {
            val brokerOrder = api.getOrder(order.id)

            // Fictitious implementation
            when (brokerOrder.status) {
                "RECEIVED" -> TODO()
                "DONE" -> TODO()
            }
        }

        // Sync buying-power
        val buyingPower = api.getBuyingPower()
        iAccount.buyingPower = buyingPower.USD

        // Set the lastUpdate time
        iAccount.lastUpdate = now
        return  iAccount.toAccount()
    }


    override fun placeOrders(orders: List<Order>) {


        // Process the orders
        for (order in orders) {
            order.id = id++.toString()
            api.placeMarketOrder(order.asset.symbol, order.size.toBigDecimal(), order.id)
            iAccount.orders.add(order)
        }


    }

}