Hello Steemians, we’ve been hard at work reducing the costs of running steemd nodes, adapting RocksDB for use in Steem, and splitting condenser into separate Wallet and Social applications. In today’s post we’d like to update you on the progress our engineers are making on these fronts. By sharing this information we hope to provide deeper insights into our operations, while also helping other node operators (like app developers) to reduce their costs as well.
EC2 Instance Cost Reduction
The full stack that Steemit runs is actually comprised of 14 individual pieces of software. Some of the more well known pieces are steemd, condenser, and hivemind but there are also lesser known applications like jussi, conveyor and even redeemer. Each of these pieces of software runs on their own EC2 instances, which can be thought of as cloud based servers.
The Goal: Optimization
The goal for this project is to optimize the types of instances we are using for each piece of software. To accomplish this we analyze metrics like CPU and RAM utilization over a specified period of time for a certain amount of traffic. By discovering which instances are being underutilized, we can achieve significant cost savings simply by switching to a lower cost instance type. After that we can look at the process through which instances are scaled up and down to handle the need for additional processing power. We can reduce costs even further by ensuring that the instances are no bigger than necessary given the computational power that is required at any particular moment to meet the demands of the network.
These changes must be made carefully so that overall response time stays relatively constant, while ensuring that scaling happens quickly enough to support the additional load that results from high-traffic periods.
Status
So far we have completed this task for four of our applications. We were able to reduce our average monthly expenditures for condenser (the software that powers steemit.com) by 60%. We were also able to reduce expenditures on conveyor by 75% while maintaining performance. Once this project is completed we will be able to turn our attention to whether or not we should utilize “reserved instances” which could deliver additional cost reductions in the range of 20-35% with respect to total EC2 spending.
RocksDB and MIRA
2 months ago we announced that we were developing a new piece of software we are calling "MIRA," which stands for Multi Index RocksDB Adapter. The purpose of MIRA is to allow Steem blockchain nodes to store almost all necessary data on disk in a modern database instead of in RAM. At that time our focus was on getting RocksDB working with data consistency.
We have now had a correct implementation of MIRA that passes all of our tests for several weeks and have progressed to focusing on performance optimizations. There are around 200 options we can use to tune RocksDB in order to maximize performance. We are continuing to optimize performance, but we have made enough progress to justify prioritizing the deployment of RocksDB to our development environment so that we can test live performance. Our goal is performance parity on average hardware, which will make running a steemd node much less expensive.
Condenser Split
Finally, we have made significant progress with respect to separating condenser into two separate applications, one social and the other financial. The process involved forking condenser and then carefully cutting whatever features do not belong in the given application. The goal is to have a complete separation of concerns. The financial version doesn’t need any of the social features, and the social version doesn’t need any of the financial features.
We have now eliminated all of the social features from the wallet app, and are in the process of streamlining the user-experience in the wallet application. On the social side, we are removing components which rely on active key as well as any signup API services (which are now part of the wallet service). To learn more about why we’re creating two separate condensers, check out our previous post.
Follow @steemitblog
There is a lot of work left to do, but we are extremely happy with the progress that has been made to date. We look forward to providing continued updates on this process, so be sure to follow @steemitblog if you would like to stay informed about the work we are doing to increase the sustainability and scalability of the Steem ecosystem.
The Steemit Team