How Logz.io Ensures Our Serverless Architecture Security
While serverless is a relatively new concept, adoption is well underway. Still, many companies are shy about exploring and implementing a serverless architecture, not to mention publishing their successful migration story online.
Droplr’s engineering team shared their story about moving to serverless, covering all the main steps of the process — planning, testing, security, and monitoring. Logz.io plays an integral part of the multi-layered monitoring approach used by Droplr, and is used as the primary logging in the new architecture.
Searching for a Logging Solution
Droplr’s services are now built almost entirely upon a serverless architecture. Lambda functions handle both the processing of background jobs and all the public-facing microservices. The latter are now HTTP-invoked Lambda functions. Requests and responses are passed via the AWS API gateway, which in turn is placed behind CloudFront.
CloudFront enabled Droplr to monitor all incoming and outgoing HTTP traffic using access logs, and the question was what log analysis tool to use to use. Droplr began exploring different logging solutions. AWS lacked the analysis features Droplr required, while other logging solutions did not integrate with AWS to a satisfying degree.
Because of previous experience with the ELK Stack, the team preferred an ELK-based solution that did not require investing resources in maintenance. Offering ELK as a scalable and secure service, Logz.io was evaluated and found to suit Droplr’s requirements and architecture. As Antoni Orfin, Solutions Architect at Droplr, puts it: “Logz.io fit our serverless – and “pay for what you use” – concept perfectly.”
Onboarding with Logz.io
Migrating to Logz.io was seamless and took less than a day’s work. CloudFront logs are shipped to an S3 bucket, from where they are pulled using Logz.io’s built-in support for AWS.
Since Droplr’s CloudFront logs contain some custom fields used for measuring bandwidth abuse, some initial parsing work was required. This was performed with the assistance of the support team at Logz.io and all processing is now performed on Logz.io’s end. CloudFront logs are parsed automatically to allow efficient analysis and visualization.
Monitoring File Downloads
More than half a million users download files from Droplr every day. It goes without saying that these downloads need to be carefully monitored. Limitations on file downloads need to be enforced and abusive behavior identified on time.
Droplr analyzes and monitors file downloads using the CloudFront access logs. These logs contain typical HTTP-related fields (e.g. status, method, IP, result), and as mentioned above have also been customized to include some Droplr-specific metrics. Kibana dashboards have been developed on top of the logs to give the team a good indication of when the system is being used as expected.
Triggering Lambdas with Logz.io Alerts
Droplr has also developed a way to automatically block a user once anomalous behavior is identified in the CloudFront logs.
This method is based on two components that integrate Logz.io with Droplr’s serverless architecture. On the Logz.io side, an alert has been configured based on specifically defined thresholds. Once triggered, this alert calls an http endpoint which is actually a Lambda function. This function is tasked with blocking the user.
The entire IT team at Droplr use Logz.io as their primary log analysis tool. Access is given to the development team as well upon request, for visibility into how the services are performing in production. Approximately 5 GB of CloudFront logs is shipped to Logz.io a day, helping Droplr not only monitor HTTP traffic but also save the company money:
Antoni Orfin, Solutions Architect at Droplr, sums it up: “After a few months of using Logz.io, we were amazed at how great it fits our serverless architecture. Using Logz.io has helped us to effectively analyze abusive usage of Droplr and has ultimately saved us tens of thousands of dollars a month.”
This article appeared originally on: https://logz.io/case-studies/droplr/
If you like what you’ve just read, stay tuned for more feed on the serverless architecture security!