Introduction
Many moons ago, one of our clients needed to deploy several subgraphs using self-hosted nodes from The Graph—a tool with a name that leaves much to be desired when it comes to debugging... (Note to self: build a logging tool called “The Log” packed with a whole bunch of “un-Googleable” bugs and make sure it becomes a dependency of The Graph—that’ll show them).
As you might expect from such strong hints in the introduction, finding clear documentation and troubleshooting issues was a challenge when we first started working with this service. After navigating a maze of unrelated information, we were finally able to craft a robust solution for The Graph—and now, we’re ready to share our hard-earned expertise with the world.
But wait, what’s “The Graph” and what are these so-called “subgraphs”?
The Graph
The Graph is a decentralized protocol that indexes and queries data from blockchain networks. It streamlines data retrieval by aggregating and structuring blockchain information into a unified API, enabling developers to build robust applications without having to deal with complex data extraction processes. Designed for scalability and transparency, it ensures that data queries remain fast and reliable at all times.
At its core, The Graph leverages a decentralized network of self-hosted nodes to process and serve data. This approach not only decentralizes the indexing process, it also enhances security and fault tolerance, key advantages in today’s rapidly evolving blockchain landscape.
Subgraphs
Within The Graph Network, we find subgraphs—custom indexing layers that define precisely what blockchain data to extract and how to organize it. By creating a subgraph, you specify the schema and relationships of the data you need, allowing for efficient and targeted data queries. This tailored data layer significantly speeds up the retrieval process since you only have to process the data once, letting your application directly access relevant, pre-processed information.
Each subgraph essentially functions as a dedicated data service, abstracting the underlying blockchain complexities. Whether you’re looking for real-time transaction details or historical records, subgraphs provide a streamlined solution for building dynamic, data-rich applications that scale with user demand.
Our Solution and its Features
- Kubernetes Deployment with Helm and Terraform
- RDS for the PostgreSQL Database
- Multi-chain Support
- Grafana Dashboards
- 24/7/365 Monitoring and Alerting
Let’s dive into the details.
Kubernetes Deployment with Helm and Terraform
We deploy nodes for querying and indexing data on The Graph Network using Amazon EKS and RDS, allowing us to leverage autoscaling support for both compute and storage, ensuring high availability and efficient resource usage. By using Kubernetes for container orchestration, our deployment process becomes simpler, more scalable, and resilient. We’ve built our own Helm charts and Terraform modules to standardize and automate this deployment process.
By using Terraform for managing the whole deployment as Infrastructure as Code (IaC) we enable consistent and repeatable deployments, while also easily versioning your infrastructure configurations. Our custom Helm chart further simplifies application management within Kubernetes, making it easy to roll out updates and manage dependencies across various environments without a single minute of downtime.
You’re free to test and push the limits in a staging environment, we’re right there with you, ready to roll back your deployment at a moment’s notice. As soon as you're confident in your latest changes, we can seamlessly replicate them in production while we vigilantly monitor all of your systems to guarantee that your infrastructure continues to run efficiently and without interruptions.
RDS for the PostgreSQL Database
Our solution leverages Amazon RDS to host the PostgreSQL database used by The Graph nodes to store their persistent data, ensuring high availability, automated backups, and seamless scalability. This managed service minimizes operational overhead while providing a secure and robust environment for your data.
We’re dedicated to keeping your data safe at all times and making sure your system scales smoothly as your demands grow.
Multi-chain Support
Our platform is built with versatility in mind. Whether you’re working with Ethereum, Arbitrum, Solana or any other network, you can deploy subgraphs seamlessly across various blockchains without the need for extra configuration—we’ll handle that for you.
That said, it's important to tailor your subgraph to each specific chain. Every blockchain has unique characteristics, so you’ll need to check your subgraph’s schema and configurations to ensure you’re still capturing the relevant events and data structures effectively. We can provide comprehensive guidance to help you customize your subgraph for optimal performance on any network.
Grafana Dashboards
Gain immediate access to comprehensive insights with our Grafana dashboards, which consolidate key performance indicators, log data, and error tracking into a single visualization layer. This setup enables efficient monitoring of both node performance and subgraph health.
Our dashboards are pre-configured to deliver actionable metrics that help you diagnose and resolve issues quickly. With customizable views and real-time data, you can maintain operational efficiency and ensure that your system remains optimized at all times.
24/7/365 Monitoring and Alerting
Our solution includes 24/7 monitoring and alerting, ensuring that any disruptions or issues with subgraph synchronization or Graph nodes are detected immediately. This continuous monitoring is meant to provide peace of mind, knowing that your deployment is always under our ever-watchful eye.
Customizable alert thresholds offer real-time notifications if any anomalies or errors occur, enabling you (or us) to take corrective measures ASAP. This proactive approach minimizes downtime and supports a highly resilient deployment environment, keeping your operations running smoothly around the clock, all year long.
You can rest easy knowing that your production environments are well taken care of.
Final Thoughts
If you need to deploy any number of subgraphs, our solution simplifies the process, freeing you from the hassle of fragmented troubleshooting guides and hard-to-find documentation.
Focus on building your project while we manage your deployment complexities using our battle-tested best practices and standards, refined over our many years as expert engineers.
