Open Sourcing Backstage Plugins from Segment's IDP

Internal developer platforms (IDPs) are a key tool in unlocking developer productivity. At Segment, we’ve been developing our own Backstage-based IDP since mid-2022. Along the way, we’ve learned a lot about how to build for Backstage—investing in Scaffolder Actions and Templates to enable self-service workflows, ingesting data from internal systems to build a robust software catalog, and developing strategies for quickly integrating with internal APIs to pull valuable workflows and interactions into the portal.

With the launch of CNOE, we began collaborating with other industry leaders to share our learnings. Today, we’re excited to start sharing our code.

Our new monorepo and first Backstage plugin

Following patterns and best practices set by other popular Backstage plugin maintainers, we’ve created a public GitHub repository at segmentio/segment-backstage-plugins structured as a standard Backstage monorepo. Over time, our goal is to lift and shift many of our internally-developed Backstage plugins into this repository and release them under Apache License 2.0.

In the plugins/ directory, you’ll find one of the very first plugins we built to support development of our IDP: the AWS SigV4 Proxy backend plugin (@segment/backstage-plugin-proxy-sigv4-backend).

What is the AWS SigV4 Proxy backend plugin?

Background

We’re heavy users of AWS. One benefit of running so many of our workloads on AWS compute is that it enables near-universal access to expirable AWS credentials for every service for free. Between EKS Pod Identities, IAM roles for service accounts, and Instance Metadata Service, there’s no shortage of options for configuring workloads with AWS-native identities and credentials to boot.

For those unfamiliar, AWS credentials are used to authenticate and authorize requests against AWS services. For example, to download a protected file from S3, or invoke an API Gateway REST API. The way AWS credentials are used to authenticate a request is described by an algorithm: the AWS Signature Version 4 (SigV4) request-signing algorithm.

Why we built it

SigV4 is a sound choice for service-to-service authentication between internal APIs. The combination of automatic credential rotation via STS, broad request-signing support across many languages via the official AWS SDKs, and first-class authorization capabilities via IAM resource policies, make SigV4 a secure, low-cost, low-complexity authentication strategy that can be readily added to existing systems through the introduction of an API Gateway.

We leverage SigV4 in exactly this way. When one internal platform API needs to be made available to another, the service need only create an API Gateway Integration to expose its endpoint(s) to other internal callers. By enabling AWS IAM authorization and attaching a resource policy that allows internal services to perform execute-api:Invoke actions on the API, callers can use their existing AWS credentials to make SigV4-signed requests to the new API Gateway endpoint.

The AWS SigV4 Proxy backend plugin we built aims to dramatically simplify the process of integrating internal APIs built this way with Backstage by moving the request-signing burden and complexity off of the caller and into a slim, flexible backend plugin.

How it works

Inspired by the official AWS SigV4 Proxy project (awslabs/aws-sigv4-proxy), the plugin signs incoming HTTP requests using AWS SigV4 and forwards them to the configured target. Proxy definitions are managed entirely through application configuration and each registered target can optionally specify a custom IAM role to be assumed and used when signing requests for that target’s endpoint(s).

If you’ve ever used the built-in proxy-backend plugin, the configuration format should feel familiar:

With this configuration, requests to  /api/proxy-sigv4/service-a/PATH  will be signed using credentials from the default credential provider chain and then forwarded to https://abcdefg.execute-api.us-west-2.amazonaws.com/PATH. Requests to  /api/proxy-sigv4/service-b/PATH will use credentials from an sts:AssumeRole call for arn:aws:iam::12345:role/custom-role-for-service-b-requests before forwarding to https://hijklmnop.execute-api.us-west-2.amazonaws.com/PATH.

By moving the request signing process into a general purpose proxy plugin, we accomplish two important things:

1. Secrets used for request signing never leave the backend

2. Proxied services maintain existing API contracts for their callers

Conclusion

We’re excited about our evolving collaboration with CNOE and eager to begin sharing some of the innovative and high-value plugins that we’ve developed to power Segment’s IDP.

The AWS SigV4 Proxy backend plugin is just the first step. It has greatly simplified the integration of internal APIs with Backstage at Segment, and we hope it helps speed time to value within your organization as well.