Replace the SNX dashboard backend with The Graph

[jjgonecrypto]

The Synthetix Dashboard is a web application that uses a closed source API to monitor and track transactions in the Synthetix ecosystem.

Blockchain analytics can get quickly get cumbersome and expensive. On Ethereum, events emitted from smart contracts are not available on public API services (such as Etherscan) and mappings (such as ones that track account balances) cannot be iterated.

The Synthetix team originally decided to create our own service to monitor and track transactions in real time as there weren't alternatives at the time that could supply our Dashboard the information it required. Since then however, the Ethereum ecosystem has evolved and developer tooling has increased significantly.

In an effort to continue decentralizing and opening up the Synthetix system, I propose we replace the closed system that supplies our Dashboard with one that is open source. And, in the spirit of open data and of removing trust, that the same system be queryable by anyone.

What is The Graph

The Graph is a decentralized protocol for blockchain queries on Ethereum. In essence, it supports GraphQL queries of entities. These entities can be created both when smart contract events are emitted and functions are invoked. Anyone can construct and query the user activity of the Synthetix smart contracts (see below). As a decentralized service, The Graph hosts its content on IPFS. Moreover, anyone can create additional subgraphs to be hosted alongside the one we manage.

Here's an example of using The Graph web explorer to query the Synthetix subgraph for sUSD transfers and SynthExchange events (and the underlying HTTP requests):

Why The Graph

By exposing an API endpoint and allowing anyone to construct queries on our data, we allow anyone in the wider Synthetix community to verify, track, monitor, and report on our data in near real-time.

• Our current subgraph is hosted at: https://thegraph.com/explorer/subgraph/synthetixio-team/synthetix
• The code for our subgraph is: https://github.com/Synthetixio/synthetix-subgraph/
• And example of querying our subgraph to get the last 24 hours of SynthExchange events: https://codepen.io/justinjmoses/pen/EzoEbM?editors=0010

Concerns

As it stands, there are two longer term concerns with using The Graph:

1. Processing times: In the past few months, we've noticed that the processing times of new versions of our subgraph have increased from a few hours to several days. This disrupts the develop-deploy-test lifecycle and creates a long turnaround for the entities to be refreshed. We have been talking with members of The Graph's engineering team who are actively looking into this.

2. Cost: The Graph team have said that they plan on charging on a per-query basis in the future, yet their actual plans are unclear. They are VC-funded and in hyper growth mode, so as such are not charging for their hosted service right now. If they do decide to charge for hosting, we can look at forking their indexing service and using decentralized services for network and processing bandwidth.

Proposal

I suggest we move ahead and ensure the Synthetix subgraph can supply all the data required for our current Dashboard. Once done, we then cut over our Dashboard to use these feeds.

[jjgonecrypto]

The Graph have posted some more information on their upcoming network, including how it will be funded:

https://thegraph.com/blog/the-graph-network-in-depth-part-1 and
https://thegraph.com/blog/the-graph-network-in-depth-part-2

In essence, Indexers will appear and vie to index those subgraphs with the most queries. Incentives will be added so that they also index newer subgraphs. To pay for usage, end users will open up state channels to the State Relay Hub, posting ETH and DAI for query support.

In terms of impact to Synthetix, any service we use which queries The Graph - such as our dashboard and even codepens - will need to be behind a caching layer, that will query The Graph on a fixed schedule and cache the results (thereby limiting exposure to per-requests costs that could be swarmed).

[yanivtal]

Thanks for the writeup and amazing work you've done Justin!

I'd like to share some of our plans that will hopefully clarify the path forward for The Graph's monetization. We are not and will never be a revenue seeking company. We are building a decentralized protocol where Indexers can compete in a query market.

While we haven't shared our plans publicly, we understand the need for partners to be able to plan their usage of The Graph. As described in the post, we'll be hosting a gateway at network launch that projects like Synthetix will be able to use to create a seamless experience for users while the transition to Web3 occurs. This gateway will be operated by Graph Protocol, Inc. and will allow Synthetix to pay for queries on behalf of their users.

The Gateway will be responsible for selecting indexers based on price/performance/security and preventing DOS attacks. The Gateway will not set prices, it will just be trusted to make good choices on behalf of its consumers. Anyone can run their own Gateway if they don't trust us. Eventually people will be encouraged to connect to The Graph directly and bypass The Gateway altogether.

Thank you for your amazing contribution to the space!