Metrics & Loggers

Metrics allow you to monitor the progress of a run and get the necessary insights on how a strategy is performing. Metrics are not limited to back testing and can also be used during forward testing, paper trading and live trading.

By default, the configured metrics are invoked at each step in a run. A single metric can return multiple values, each value having its own unique name. Metrics only calculate the values, the storing and/or logging of these values is done by a MetricsLogger.

Standard Metrics

There are several commonly used metrics included with roboquant that can be added to your Roboquant instance:

val metric1 = AccountMetric()
val metric2 = ExposureMetric()
val metric3 = ProgressMetric()
val metric4 = VWAPMetric()
val metric5 = ReturnsMetric()
val metric6 = PNLMetric()
val metric7 = AlphaBetaMetric(sp500Asset, 250)
val metric8 = ScorecardMetric()

val roboquant = Roboquant(
    strategy, metric1, metric2, metric3, metric4, metric5, metric6, metric7, metric8
)

If a metric is very computationally intensive, and you don’t require it at each step, you can wrap it in a MetricsScheduler and configure when it should be invoked.

Custom Metrics

You can also develop your own metrics. All you have to do is implement the calculate method from the Metric interface and return MetricResults, which is just a typealias for Map<String, Double>. Metrics also implement the Lifecycle interface, so you can keep state in a metric if required and clean it when a lifecycle method is invoked.

If you don’t have any results at a certain step, you should return an emptyMap(). In the calculate function you have access to both the Account and the Event objects.

class MyMetric : Metric {

    private val asset = Asset("AAPL")

    override fun calculate(account: Account, event: Event): Map<String, Double> {
        val metric1 = account.buyingPower.value
        val metric2 = event.getPrice(asset) ?: Double.NaN
        return mapOf("buyingpower" to metric1, "appleprice" to metric2)
    }

}

Metrics Loggers

A metrics logger receives the metric values at each step and then stores and/or logs these values. All implement the MetricsLogger interface.

The following metric loggers are available out of the box:

MemoryLogger: this is the default logger used if no other logger is configured. It stores all the recorded values in memory and allows them to be queried afterwards. It also comes with an optional progress bar that shows how much time is left during a run.

This logger is especially useful if you are using a Jupyter Notebook and want to directly plot some metrics after a run.
```
// You can always find out which metrics are available after a run
val logger = roboquant.logger
println(logger.getMetricNames())

// And plot a metric in a Jupyter Notebook
val equity = logger.getMetric("account.equity")
TimeSeriesChart(equity)
```
SilentLogger — This logger just silently ignores all metric values and is mostly useful for unit tests
InfoLogger — Logs metrics using the built-in logger at INFO level
ConsoleLogger — Logs the metrics to the console using the println statement

QuestDBLogger — Logs the metrics to a QuestDB database.

// Without specifying a path, the logger will use
// the default DB path ~/.roboquant/questdb-metrics/db
val logger = QuestDBMetricsLogger()

val rq = Roboquant(EMAStrategy(), AccountMetric(), logger = logger)

// The logger can be used for multiple runs, even concurrently if required.
// Just make sure every run has a unique name
feed.timeframe.split(1.years).forEach {
    rq.run(feed, it , name = "run-${it.toPrettyString()}")
}

You can access the data just like with the memory logger, or use a web console:

docker run --rm -p 9000:9000 -v "$HOME/.roboquant/questdb-metrics:/var/lib/questdb" questdb/questdb:7.3.1

Custom Loggers

You can also implement your own metrics logger, for example if you want to store values into a database or have a custom file format that you want to use. The following example shows a minimal implementation:

class MyDatabaseLogger : MetricsLogger {

    private val database = Database()

    override fun log(results: Map<String, Double>, time: Instant, run: String) {
        for ((key, value) in results) database.store(key, value, time)
    }

    /**
     * Optional, if you want to access the logged metrics via roboquant
     */
    override fun getMetric(metricName: String, run: String): TimeSeries {
        return database.retrieve(metricName, run)
    }

    /**
     * Optional, if you want to access the logged metrics via roboquant
     */
    override fun getMetricNames(run: String) = database.retrieveKeys(run)

}