As companies grow, their cloud infrastructure quickly becomes fragmented and gets out of control. Data about what resources exist and how resources they relate to each other is tedious to acquire.

In practice, this means that the infrastructure layer often remains a mystery, and engineering teams are unable to see what's happening in their infrastructure. This makes capacity planning impossible, limits organizations' ability to control cloud costs, and leaves teams in the dark about potential security vulnerabilities.

The data to understand cloud growth exists as cloud resource metadata describing the state, configuration, and dependencies of cloud resources. Acquiring and unifying this "infrastructure data" into a single place is the solution for a lot of the problems that infrastructure engineers deal with today—not just cost, but also security and reliability.

But infrastructure is fragmented. Data is locked behind cloud APIs, and the tools that use those APIs to control the deployment of cloud resources. In this post, I'll explain how Resoto acquires infrastructure data, and then uses that data to write code.

Hi there fellow cloud enthusiast 🖖

In this blog post, we'll delve into infrastructure fragmentation, exploring what it entails, why it impacts every organization utilizing the cloud, and how building a reactive infrastructure brings order to chaos.

Explaining Infrastructure Fragmentation: The Laptop Analogy​

For an analogy let's take your Laptop. Over time, performance declines due to a variety of factors, including the accumulation of unnecessary files, software conflicts, and the buildup of outdated or redundant software. And despite your best cleaning efforts, after a year or two its performance is just not what it used to be and everything is a bit more cluttered and disorganized than what it was like when brand new.

Now instead of just you working with that Laptop, imagine there were tens, hundreds or even thousands of users. Imagine every cloud user in your organization was working on it as well. That is what your cloud infrastructure looks like.

Infrastructure fragmentation, also known as cloud fragmentation or cloud sprawl, refers to the phenomenon where an organization's cloud infrastructure becomes decentralized and uncoordinated due to the widespread adoption of cloud services by independently operating teams or individuals and a lack of centralized governance.

In other words, infrastructure fragmentation is the uncontrolled proliferation of cloud services and providers within an organization. Resulting in a disorganized and difficult-to-manage cloud infrastructure.

Infrastructure fragmentation can create various problems, such as security issues, quota and performance problems as well as increased costs due to redundancies, leaked or abandoned resources and Infrastructure as Code (IaC) drift.

EC2 instances often account for the largest portion of your AWS bill. Yet, it's notoriously difficult to get a simple list of all EC2 instances across all regions and accounts, as threads on StackOverflow and Reddit show.

You also then want to use that list to ask questions about your inventory, such as:

• How many total instances are there?
• Which instances are running?
• Which instances are missing tags?
• Which resources have an expiration date?

In this post, I'll describe how to use Resoto to build an EC2 cloud asset inventory. The baseline inventory is a list with all EC2 instances, which you then can use to create more narrow and detailed views.

In today's cloud-native world, maintaining a resilient and secure infrastructure is crucial to the success of any business. However, as the cloud infrastructure grows in complexity, it becomes increasingly difficult to track all your cloud resources.

This is where Resoto comes in—Resoto ensures that information about your cloud resources is always available by routinely collecting infrastructure data across cloud providers, accounts, regions, and a plethora of services.

However, simply having cloud resource data is not enough; it is also crucial to automate actions based on this data.

For example, if a resource is no longer needed, it should be cleaned up to avoid incurring unnecessary costs. Similarly, if a resource is not properly tagged, it can be difficult to identify its purpose, leading to confusion and making reporting a mess.

Resoto provides tools to cope with such challenges. In this post, we'll explore another category of high-priority issues that often require immediate attention: breaches in the security baseline.

New month, new release! We shipped Resoto 3.2 with a handful of new features and improvements.

Welcome to my first blog post! I'm Raffa and I'm excited to share a new feature that I've been working on: the job editor in Resoto UI.

The Resoto job editor provides a low-code way to create and manage automated jobs.

As 2022 came to a close, we released Cloud2SQL and Resoto 3.0. Resoto 3.0 shipped with a brand new web UI built on the Godot game engine.

Resoto makes it easy to create an inventory of your cloud infrastructure. With just a few commands, you get a snapshot of your resources. And with that data ingested into a time series database, you not only know the current state of your cloud but also have insights into changes and trends.

Resoto also offers simple and human-friendly ways to view this data—you can export the data to Google Sheets to create charts or use Resoto Notebook for interactive search visualization and analysis.

Today, we are introducing a new way to visualize this data: Resoto Dashboards.

With Resoto Dashboards, your searches and queries are executed periodically and the results displayed in widgets automatically. You can also customize and organize widgets in the dashboard to keep the most important metrics always within reach. You can even share dashboards across your organization so others can access the data. And of course, you can access your dashboards anywhere you have access to your Resoto installation.

Cloud2SQL is a tool based on Resoto's collector plugins that allows you to collect data from various cloud infrastructure sources and export it to a database (like Snowflake, PostgreSQL, MariaDB, or MySQL) or write it as Parquet, SQLite, or CSV files for ingestion in your data lake.

In this post, I will guide you through the process of installing Cloud2SQL and demonstrate how to export data from AWS to a local SQLite database file.

Whether you are looking to integrate cloud data into your existing SQL workflows or simply want an easy way to access and analyze data from multiple cloud sources, Cloud2SQL is an excellent tool to consider.

One of the key features of Resoto is its ability to collect data from a wide range of cloud providers, including Amazon Web Services (AWS), DigitalOcean, and Google Cloud Platform (GCP). This makes it easy to get a comprehensive view of your cloud infrastructure, no matter where it is deployed.

But, what sets Resoto apart from other cloud data collection tools is its ability to enrich the data it collects and make additional connections. This means that Resoto not only gathers raw data about your cloud resources, but also adds additional context and information that can help you better understand your cloud environment.