Technology architecture challenges of SLT project
In this article I’d like to review multiple challenges one would face when designing and implementing truly super-long-term (SLT) project, aimed to span multiple centuries and human generations.
First challenge: Sustainability
Resistance to hard times
The project must survive in turbulent times like pandemic-caused economic depression with the same confidence as in times of pleasant growth.
In these days there are too many companies which are desperately dependent of persistent growth to keep their profitability. When hard times come, they can’t keep their business on the go while having financial responsibilities to their clients, hence go bankrupt.
From the other side, we never lived in such perfect times in terms of on demand-based services. We can now enjoy Pay-As-You-Go pricing allocation so we pay effectively nothing (or close to nothing) in times of low demand. Of course, if the project is properly designed.
Retention.live leans on a fully serverless and on demand-based setup with every resource generates an expense only when it’s in use (with few exceptions like DNS zones, that is not game changers here).
To benefit from serverless computing, the project technology architecture should lean on event-driven pattern rather than tightly coupled components.
Decoupled workers are sleeping until a new event appears and posting a response back to the event bus after they have completed their tasks.
Ability to consume high demand
A mirror issue is also valid here: as user base of the project might experience exponential growth over many years, any resource limit set in advance will be exceeded earlier or later. Solution here is to not apply any limits at all, which is perfectly doable in serverless computing world.
Similarly, in case of rapid demand growth, the project resources shouldn’t be overflown and remain accessible and functional. This also fits well into serverless approach, which is capable to quickly provide resources on demand.
Second challenge: Cost control
A cost control of SLT project must be total and sufficient enough to secure clients’ property over long periods of time — like centuries. There are two major challenges of keeping the promise: cost accumulation and the service misuse prevention.
Cost accumulation
Small fee grows to a fortune in a century. For example, you have $1 annual fee when using some service. It hardly looks dangerous, doesn’t it? But it adds as much as $1000 to client’s millennial expense. One simply can’t afford this fee if aims to limit the pricing by $100 for a thousand years.
The promise must be carefully estimated against super-long periods of time like millennia to ensure it will sustain even very small increase of expenses because it might grow to unacceptable levels because of multiplication by hundreds of years.
Retention.live guarantees one day of user’s data exposition annually for a thousand of years which means that as many as 1000 days (3+ years) of client’s files exposition must be secured. Traffic and requests cost must be properly estimated and protected from abuse with intelligent CDN solution.
Service misuse prevention
We live in times when DDOS attack costs just a small sum on black market. And sometimes it is not even the attack, but some bright idea of a really smart user who is not aware, that her actions jeopardise the whole project finances.
The only way to defend the project of possible overuse is sophisticated and carefully designed monitoring system. It’s not enough to monitor the project precisely and reliably. The system must react instantly and block compromised resources immediately — before a harm to other users inflicted.
Third challenge: Data durability
A probability of users’ data corruption or loss is growing over the years. There are two major types of data which must be secured and made as tolerant as possible to passing of time: users’ data and users’ accounts.
Users’ data can be as reliable as 99.999999999% durability of objects over a given year according to the modern cloud offerings. This durability level corresponds to an average annual expected loss of 0.000000001% of objects, or a single object from 100 thousands in every millennia.
Users’ accounts are more vulnerable to human mistake, especially if many humans maintain them over the centuries. Blockchain (ledger) technology has arrived just in time to help Retention.live secure users’ accounts from accidental damage or deletion because of human mistake.
Unfortunately, 100% protection against passing of time is impossible as per current understanding of science because of an entropy of the Universe. However, we can not predict if the view will remain the same in the future…
Please subscribe to this publication
This is the second article in a series, explaining why Retention.live project has been created in the first place. The next one will be about practical advice of how to collect and store your priceless personal data.
