This article explains how to identify such tasks and how to ensure progress isn't lost after interruptions on queued computing resources.
01 First, distinguish between real-time and batch tasks.
Tasks where someone is waiting, like conversations and instant searches, must use real-time resources. Background batch tasks, such as data cleaning that just needs completion by the end of the day, are suitable for queued computing.
Only the latter is suitable for the queued tier.
02 Why the queued tier is cheaper.
SpareHive's homepage mentions that supply comes from idle consumer GPUs in China, where electricity prices are significantly lower than in the US and Europe. The site states this price difference is why it can lower costs, but it's a hypothesis pending private testing, not yet realized profit.
The queued tier is cheaper, but the trade-off is that machines may be interrupted, requiring recovery capability.
03 Tasks must support checkpoint resumption.
Break batch tasks into small batches, saving input and output for each to disk. After an interruption, only rerun incomplete batches.
Without this layer, one interruption could waste long runs, negating any savings.
04 Record costs per batch.
For each batch, record start time, end time, tokens processed, and actual cost to calculate cost per million tokens. Compare this with average API prices to determine if the queued tier is truly cost-effective, not just based on listed rates.
05 Tasks unsuitable for queuing.
Online conversations, customer service needing immediate feedback, and time-sensitive deliveries cannot be queued. Putting these in the queued tier saves money not worth the delays you'd incur.
If your batch tasks require stable computing power, you can submit a request on SpareHive for manual platform matching, but first clarify the provider's availability and interruption rules.