A Makeshift Service Discovery for Lambda Function URLs

🗓️ May 28, 2024

Lambda function URLs not only provide a very fast and easy way to build HTTP endpoints, but also provide streaming functionality to these endpoints. Unfortunately, as soon as we use any kind of buffering reverse-proxy like AWS’s own Application Load Balancer or API Gateway, streaming responses do not work.

The simplest way to use Function URLs with streaming responses, is to use the provided URL directly. Security challenges like DDoS protection and firewalling aside, this also poses a service discovery challenge whenever a Function URL changes. This may happen if we tear down a Lambda function and recreate it.

To provide a more stable interface to our Function URLs, we must provide a service discovery that is more stable in its configuration than our Function URLs. Storing our service catalog in a JSON containing the current Function URLs that can be pulled and utilized by our application may suffice for simple use cases. Such a JSON can simply be distributed via S3 Static Website Hosting. The only infrastructure component that must remain stable in this scenario is the name of the S3 bucket storing the service catalog and the region of the S3 website endpoint.

In order to generate the JSON file used for service discovery, we create a Lambda function (I use Python) that iterates over a list of Lambda function names, pulls the Function URLs via the AWS API, creates a JSON and saves it to the S3 bucket:

import json
import boto3
from botocore.exceptions import ClientError


def lambda_handler(event, context):
    function_names = [
        "foobar-value-estimator",
        "foobar-value-estimator-image"
    ]
    
    lambda_client = boto3.client("lambda")
    s3_client = boto3.client("s3")
    
    bucket_name = "foobar-app-service-catalog"
    s3_key = "catalog.json"
    
    endpoints = []

    for function_name in function_names:
        print("Retrieving function URL for service", function_name)
        response = lambda_client.get_function_url_config(FunctionName=function_name)
        function_url = response["FunctionUrl"]
        endpoint_details = {
            "serviceName": function_name,
            "serviceUrl": function_url
        }
        endpoints.append(endpoint_details)
    
    json_data = json.dumps(endpoints, indent=2)
    
    s3_client.put_object(Bucket=bucket_name, Key=s3_key, Body=json_data)
    print(f"Catalog successfully saved to s3://{bucket_name}/{s3_key}")

We require the following permissions on the Lambda’s Execution Role (scope this to your resources):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lambda:GetFunction",
                "lambda:GetFunctionUrlConfig"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "*"
        }
    ]
}

The resulting JSON looks like this:

[
  {
    "serviceName": "foobar-value-estimator",
    "serviceUrl": "https://vbkqy74lohmljq5msufdc6nkpei0brjrs.lambda-url.eu-west-1.on.aws/"
  },
  {
    "serviceName": "foobar-value-estimator-image",
    "serviceUrl": "https://e53gmidsdelwolujojulbizpvm0nnnwo.lambda-url.eu-west-1.on.aws/"
  }
]

Comments