implement IO throttle #31

New issue

Open

opened 2021-12-18 01:35:14 +00:00 by forest · 0 comments

forest commented

2021-12-18 01:35:14 +00:00

Owner

One capsul is responsible for more than half of our write IOPS, and its a "tiny" $5/mo one. 🤪

This is an issue because its a monetary cost for us. SSDs wear out after a certain number of writes, and they are not pitching in proportionately to help with the cost of replacing the disks periodically.

All other providers do this. Ideally would not have to, but we aren't interested in manually policing what our users do on their capsuls.

So we will have to implement something like "IO credits" for "IO Burst".

Once your capsul runs out of IO credits, it wil be throttled to a much slower storage performance.

We can do better than providers like amazon in this regard; we can give users a large pool of IO credits and warn them when it's half depleted. A "normal" user with a normal use case will never even get close to seeing this warning, but even the most agressive disk-writing-user will be warned days or weeks before they are throttled. This way we can protect our disks from wearout or at least force users to pay more monthly if they need this kind of IO performance, but at the same time they should hopefully not have to experience the misery of trying to fix a problem while being throttled.

j3s mentioned that qemu supports this natively: https://github.com/qemu/qemu/blob/master/docs/throttle.txt

One capsul is responsible for more than half of our write IOPS, and its a "tiny" $5/mo one. 🤪 This is an issue because its a monetary cost for us. SSDs wear out after a certain number of writes, and they are not pitching in proportionately to help with the cost of replacing the disks periodically. All other providers do this. Ideally would not have to, but we aren't interested in manually policing what our users do on their capsuls. So we will have to implement something like "IO credits" for "IO Burst". Once your capsul runs out of IO credits, it wil be throttled to a much slower storage performance. We can do better than providers like amazon in this regard; we can give users a large pool of IO credits and warn them when it's half depleted. A "normal" user with a normal use case will never even get close to seeing this warning, but even the most agressive disk-writing-user will be warned days or weeks before they are throttled. This way we can protect our disks from wearout or at least force users to pay more monthly if they need this kind of IO performance, but at the same time they should hopefully not have to experience the misery of trying to fix a problem while being throttled. j3s mentioned that qemu supports this natively: https://github.com/qemu/qemu/blob/master/docs/throttle.txt

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: cyberia/capsul-flask#31

No description provided.