"A generation which ignores history has no past—and no future."
— Robert A. Heinlein

While Heinlein's words refer to human history, they also apply to cloud infrastructure. Most of the time, we care about the current state of resources; but sometimes, we want to know the origin of a resource, when a resource was deleted, or when/how a resource was updated.

Such knowledge is necessary in situations where you need to understand the timeline to investigate a specific system behaviour:

• To perform the post-mortem analysis of an outage, we need to know which cloud resources changed and how they changed to yield the behaviour that we saw in our application. Without the ability to review a change log this becomes impossible.
• To understand cost spikes in your cloud billing dashboard, you need to understand what resources were created, when they were created, and by whom they were created. Not only do you need a list of changes, but also the ability to filter, group, sort, and aggregate the data to see the big picture.
• To check for security issues or compliance violations, you may need to reduce the scope to verify only those resources that were created or updated since the previous scan. Even complex checks can be performed on large infrastructures if they are only run against changed resources.

History is a log of events defining your infrastructure. This event log is important, as it will enable you to answer future questions about the state of your infrastructure retrospectively, including tomorrow's questions that have not yet crossed your mind.

A Resoto install comes with batteries included; Resoto ships with a command-line interface (CLI) that allows for exploration, insights, and manipulation of your infrastructure. With Resoto's CLI, automating tedious tasks becomes a breeze. Think about enforcing a policy, cleaning up resources, exporting data, or alerting on specific circumstances. See How-To Guides to learn more about possible use cases.

Version 3 of Resoto introduces the ability to extend this capability by defining custom commands programmatically in the language of your choice. If you are familiar with Python, this task becomes super easy, since all the necessary boilerplate code is already provided.

In this blog post, we will implement a new command called hello-world in Python, to show the power and flexibility of this new feature. The simple idea of our new command is adding a greeting to the tags of a selected resource.

Cloud tagging strategies and policies are hailed as one of the most efficient ways to keep your cloud infrastructure controllable. But are they really?

Cloud Resource Tagging​

Generally, the idea is that every piece of cloud service gets tagged (or labeled, in case of GCP) by the developers or maintainers who work with it. This could be accomplished with infrastructure-as-code (IaC) tools (such as Terraform), with a command-line interface (CLI), or in the cloud UI.

Cloud Resource Tagging Policies​

Tagging policies could require that each resource needs tags identifying the owner, cost center, product, project, and/or any other metadata. By being diligent about tagging, resources can be managed via their tags and nothing gets overlooked.

Cloud Resource Tagging Challenges​

In theory, this is the correct way to manage resources; in practice, however, this hardly ever works as intended.

Each tag created is a tag that requires maintenance. Tagging policies may change over time and people can make mistakes (in AWS, for example, tag keys are case sensitive).

And, to properly use tagging on a greenfield cloud account is one thing; to retroactively apply tags to sprawling cloud infrastructure is quite another (especially when utilizing a multi-cloud strategy, where you'd need to repeat any operation over multiple interfaces).

Hello folks! A few months ago, we released Resoto Notebook, a library that makes it easy to query, visualize, and analyze Resoto data using pandas, Plotly, and Jupyter Notebooks.

Today, we'll discuss Resoto's new JupyterLite support, which allows you to use notebooks in the browser without installing and launching a Jupyter server.

Want to analyze raw infrastructure data when only platform engineers can access cloud consoles? Or count infrastructure assets without a data scientist? JupyterLite is a JupyterLab distribution that runs entirely in a web browser, and Resoto's JupyterLite support gives you access to popular data analysis tools without the need for any additional installation steps.

We recently released the Resoto AWS CDK construct, which simplifies the deployment of Resoto to AWS.

The CDK construct pretty simple. All you need is a recent version of Node.js (we tested with 18.x.x) and Git. If you don't already have Node.js installed, we recommend using Node Version Manager (nvm).

In my last post, we discussed building actionable cloud infrastructure metrics and how to create metrics, export them into a time series database, and visualize them with Grafana. Today, we'll take a look at how to build an infrastructure app using Streamlit, a framework that turns data into web apps.

If you are not familiar with Python, don't worry—we're going to keep it simple! In Prerequisites, we'll go over installing Python and the coding techniques utilized in this project.

Software engineers working with AWS have every cloud service imaginable at their fingertips, and developer velocity could hardly be higher. But, even the most shiny of coins has two sides.

While developers can freely spin up compute instances and databases in addition to less tangible things like Lambda functions or virtual identities—at some point, someone will ask, "What is all of this?"

And as that person hacks away in the CLI trying to get an overview of resources spanning multiple AWS accounts, they will inevitably get frustrated.

While Amazon has been a pioneer in cloud computing and offers the largest array of services, there are some things that just aren't so ideal. Namely, API consistency.

In this post, I describe a few of the challenges and quirks with the AWS API and why we're building Resoto. (Spoiler alert: It is so that you don't have to!)

Today's world of cloud computing is complex. There are many cloud providers, each with their own set of services. Getting insights out of your infrastructure requires specialized understanding of the data from each service.

Cloud Service Diversity​

Properties in different services may have different names but the same meaning, or vice versa. To interpret properties, we need to ensure that values have a defined unit of measurement and one base unit. You can see the challenge if you imagine the many ways you can specify the size of a volume, the number of CPU cores, or even timestamps.

Kubernetes has dramatically improved the way we manage our workloads. It has become the de-facto standard for deploying and managing containerized applications, and is available in all major cloud providers.

A typical setup consists of distinct Kubernetes clusters for each application stage (e.g., dev, test, prod) or a cluster per tenant, and Kubernetes clusters shared between different users and teams often utilize namespaces and roles to control access. Deploying a single application to a Kubernetes cluster usually consists of tens to hundreds of resources (e.g., deployments, services, ConfigMaps, secrets, ingresses, etc.).

Even a relatively simple setup quickly becomes tedious to manage as the resource count grows. It is difficult for a human to keep track of resources, especially with user access limited to certain clusters in select namespaces.

A cloud asset inventory is a complete representation of the resources in your cloud. The job of the inventory is to continuously discover new resources and store data about each individual resource (such as its properties, configurations, and dependencies). Examples of resources not only include compute instances, storage buckets, Kubernetes pods, but also access keys and user and org policies.

In modern cloud-native environments, developers enjoy freedom and permissions to create new resources. The resources in a company's cloud environment can easily number in the hundreds of thousands or millions, resulting in new challenges for infrastructure engineers. One such problem is "infrastructure fragmentation"—resources are distributed across regions, organizations, accounts, and/or projects, and each resource has unique properties and APIs. Coupled with constant change, this fragmentation makes it difficult to keep track of resources, which opens the door to cost problems, security threats, and compliance issues.

A cloud asset inventory solves the infrastructure fragmentation problem by providing complete visibility into all resources from a single place.