Image for post
Image for post

Sending emails is a complicated topic still due to the anti-SPAM war, which lasts for decades already. One can’t simply send emails using the default installation of MTA (Mail Transport Agent).

Well, you can send them, obviously, but your recipient won’t get them as they won’t be accepted by his email service provider (ESP) and delivered to her mailbox. Especially if you mass-mailing.

In this article, I’d like to go through a minimal set of configurations, required to have your emails successfully delivered to their destination. I will use Postfix in Docker image in code examples.

Configure DNS stuff

The main protection mechanism for anti-spam measurements is to use relevant DNS records. You should create two records at least: for SPF and DKIM technologies. …

Image for post
Image for post

To be a maintainer of an email server is hard. One can’t simply apt install default-mta to get a healthy and robust email system. Even now it still requires extra work.

It takes only a short time after the server has been misconfigured as an open relay to have it compromised by spammers and get its IP caught by multiple anti-spam lists!

A modern email server must support SASL, DKIM and TLS to be treated nicely by huge email providers like GMail.

As a DevOps Engineer, I tend to use Docker as much as I can to support uniform and well-maintainable infrastructure. …

Image for post
Image for post
Luca Galuzzi —

In this article I’d like to review multiple challenges one would face when designing and implementing truly super-long-term (SLT) project, aimed to span multiple centuries and human generations.

First challenge: Sustainability

Resistance to hard times

The project must survive in turbulent times like pandemic-caused economic depression with the same confidence as in times of pleasant growth.

In these days there are too many companies which are desperately dependent of persistent growth to keep their profitability. When hard times come, they can’t keep their business on the go while having financial responsibilities to their clients, hence go bankrupt.

From the other side, we never lived in such perfect times in terms of on demand-based services. We can now enjoy Pay-As-You-Go pricing allocation so we pay effectively nothing (or close to nothing) in times of low demand. Of course, if the project is properly designed. …

Image for post
Image for post
Pyramids at Giza, by Ricardo Liberato. CC BY-SA 2.0

Who wants to live forever?

Immortality is a longtime dream of humankind. While as organisms we can only hope to be alive for a century (if something like coronavirus won’t make it shorter), we have already invented immortality — in digital dimension!

Digital revolution’s introduced digital immortality

People of ancient times were keen to save their work for the future generations but, effectively, failed. They leave to us few artifacts like petroglyphs or pyramids but in terms of information it’s nothing. We have no clue how those pyramids were built as there is no README file attached…

And after only a few thousand years we see those signs of disrepair and we know we lost the vast majority of evidence of the past ages already — because of entropy, which rules in the real world. …

Image for post
Image for post

The issue

We don’t expect that kind of incidents anymore in the modern world. In this epoch of DevOps and automated testing it simply should not happen. We also are quite used to cheap and easy backups in the cloud, e.g. EBS volume lifecycle policies of AWS EC2 service…

But sometimes stars come to this rare configuration and my friend, who owns a pet SOHO web-hosting, has woken up by alarm, reporting his CloudLinux server in AWS seems dead from 3am of that unfortunate night. :(

What is even worse, he didn’t configure EBS lifecycle policies for the server, relying on per-client data backups only — hence they are quite useless in this painful situation, when the server doesn’t respond to any requests. …

The idea of this article have come from the following blog of Jeff Barr, in which he wrote:

…CloudWatch Integration — You can write a bit of glue code to run queries, use the results to publish Custom Metrics. Then you can visualize them, set alarms, and so forth, all with the goal of simplifying and accelerating your troubleshooting.

As new and enthusiastic user of recently announced CloudWatch Logs Insights service, I was wondering how it could be implemented.

In my pet project I have a bunch of Lambda functions and wanted to track their errors in most hassle-free and straightforward way. …

Image for post
Image for post

I suppose everyone agrees that as mature infrastructure engineers we won’t allow our CloudFormation stacks to receive incorrect parameters.
This might cause quite serious consequences for downstream resources.

This is why we have the following validation options in Parameters section of CloudFormation templates:

AllowedPattern — A regular expression that represents the patterns to allow for String types.

AllowedValues — An array containing the list of values allowed for the parameter.

MaxLength — An integer value that determines the largest number of characters you want to allow for String types.

MinLength — An integer value that determines the smallest number of characters you want to allow for String types. …

If you’ve come to this article because of its title, you’re most probably aware of what cfn-response is, so you can skip the next section. For others I’m moving to a short introduction of cfn-response module right away.

What is cfn-response module?

To make sense of all in the below, just feel a context: you are building a custom resource in your CloudFormation stack, which is backed by Lambda function. After your service Lambda invoked on CREATE/UPDATE/DELETE events of the stack, it has to send SUCCESS/FAILED response back to CloudFormation service to report the operation result status.

If it doesn’t, CloudFormation stack will get stuck in ‘IN PROGRESS’ state for annoyingly long time — about 3 hours. (Well, you’re still able to unblock it manually if you farseeing enough to post an event received into logs, but it’s a different…

I suppose that everyone beginning with Lambda@Edge function, gets a bit surprised of how it publishes its logs to CloudWatch service. Yes, Lambda@Edge, which exists in us-east-1 region only, distributes its logs across all the regions where it’s invoked at least once. So after you get some traffic into your CloudFront distribution, you could find Lambda@Edge CloudWatch log groups in some regions.

This does not look really appropriate for even basic level of project observability, hence some log aggregation mechanism is required for Lambda@Edge from day one.

There are different ways to approach this challenge (as we used to expect in AWS), and there is the one, recommended by AWS, which implements log streaming to S3 bucket using Kinesis…

I’ve faced a challenge with my AWS pet project runtime. I’m storing objects in S3 Glacier storage class, which allows restoration to S3 Standard class in up to 5 minutes, or 5 or 12 hours, depending on retrieval tier.

After the mentioned period of time gone, I need to invalidate Cloudfront CDN cache to make the objects available to users. Hence, I need some scheduling mechanism to run invalidation Lambda function after delays specified.

After short googling I’ve found great article by Yan Cui (AWS Serverless Hero), in which he is reviewing some methods existing, doing quite interesting experiment and providing useful statistics. …


Danil Smirnov

Containers, Serverless and DevOps

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store