Isolating Legacy Dependencies: Banish Risk to Lambda

tags: #Aws #Lambda #Architecture #Refactoring #Technical-Debt #Containers #Security
categories: Architecture Operations Cloud
published: 2024-08-22
reading time: 8 minutes

That unmaintained package with a memory leak holding back your upgrade? Isolate it. Put risky dependencies in Lambda containers where they can’t hurt your main application. A practical strategy for managing technical debt.

The Problem: Dependency Hell

You’re running Rails 6.1. Rails 7.1 is out. You want to upgrade, but there’s a problem: legacy-pdf-generator gem hasn’t been updated in 3 years, doesn’t work with Rails 7, and the maintainer has disappeared.

Or worse: it kind of works, but has a memory leak that crashes your app every few days.

Sound familiar?

Common scenarios:

Unmaintained packages blocking framework upgrades
Dependencies with memory leaks or stability issues
Libraries with security vulnerabilities (but no alternatives)
Heavy dependencies used for one small feature
Native extensions that break on new OS versions

The traditional solution: fork it, fix it, maintain it forever. Or stay stuck on old versions.

There’s a better way: isolate the risk.

The Strategy: Containment

Instead of letting risky dependencies infect your entire application, quarantine them in isolated Lambda functions. Your main app stays clean, upgradeable, and stable.

Benefits:

Upgrade your main application without dependency constraints
Memory leaks? Lambda recycles containers automatically
Security vulnerability? Isolated blast radius
Crashes? Doesn’t affect your main app
Easy to replace later without touching core code

Real-World Example: PDF Generation

Let’s say you’re using wicked_pdf (a Ruby gem wrapping wkhtmltopdf) to generate PDFs. It’s unmaintained, has memory leaks, and uses an ancient version of Qt WebKit.

Before: Tightly Coupled

# app/controllers/invoices_controller.rb
class InvoicesController < ApplicationController
  def show
    @invoice = Invoice.find(params[:id])
    
    respond_to do |format|
      format.html
      format.pdf do
        pdf = WickedPdf.new.pdf_from_string(
          render_to_string('invoices/show', layout: 'pdf'),
          page_size: 'A4'
        )
        send_data pdf, filename: "invoice-#{@invoice.id}.pdf"
      end
    end
  end
end

Problems:

wkhtmltopdf binary must be installed on every app server
Memory leaks accumulate in long-running processes
Blocks upgrade to newer Ruby/Rails versions
PDF generation can spike CPU and slow down web requests

After: Isolated in Lambda

Step 1: Create Lambda Function

# lambda/pdf_generator/handler.py
import json
import boto3
import pdfkit
from datetime import datetime

s3 = boto3.client('s3')

def generate_pdf(event, context):
    """
    Generate PDF from HTML and upload to S3
    """
    try:
        html_content = event['html']
        invoice_id = event['invoice_id']
        bucket = event['bucket']
        
        # Generate PDF
        pdf = pdfkit.from_string(html_content, False, options={
            'page-size': 'A4',
            'encoding': 'UTF-8',
            'no-outline': None,
            'quiet': ''
        })
        
        # Upload to S3
        key = f"invoices/{invoice_id}/invoice-{datetime.now().isoformat()}.pdf"
        s3.put_object(
            Bucket=bucket,
            Key=key,
            Body=pdf,
            ContentType='application/pdf'
        )
        
        # Generate presigned URL
        url = s3.generate_presigned_url(
            'get_object',
            Params={'Bucket': bucket, 'Key': key},
            ExpiresIn=3600
        )
        
        return {
            'statusCode': 200,
            'body': json.dumps({
                'pdf_url': url,
                's3_key': key
            })
        }
        
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Dockerfile for Lambda:

FROM public.ecr.aws/lambda/python:3.11

# Install wkhtmltopdf and dependencies
RUN yum install -y \
    wget \
    fontconfig \
    libX11 \
    libXext \
    libXrender \
    xorg-x11-fonts-Type1 \
    xorg-x11-fonts-75dpi

RUN wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6.1-2/wkhtmltox-0.12.6.1-2.amazonlinux2.x86_64.rpm && \
    yum install -y wkhtmltox-0.12.6.1-2.amazonlinux2.x86_64.rpm && \
    rm wkhtmltox-0.12.6.1-2.amazonlinux2.x86_64.rpm

# Install Python dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy function code
COPY handler.py ${LAMBDA_TASK_ROOT}

CMD ["handler.generate_pdf"]

requirements.txt:

boto3==1.28.0
pdfkit==1.0.0

Deploy with Terraform:

resource "aws_ecr_repository" "pdf_generator" {
  name = "pdf-generator"
}

resource "aws_lambda_function" "pdf_generator" {
  function_name = "pdf-generator"
  role          = aws_iam_role.lambda_exec.arn
  package_type  = "Image"
  image_uri     = "${aws_ecr_repository.pdf_generator.repository_url}:latest"
  
  memory_size = 1024
  timeout     = 60
  
  environment {
    variables = {
      PDF_BUCKET = aws_s3_bucket.pdfs.id
    }
  }
}

resource "aws_iam_role" "lambda_exec" {
  name = "pdf-generator-lambda"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_logs" {
  role       = aws_iam_role.lambda_exec.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_s3" {
  role = aws_iam_role.lambda_exec.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "s3:PutObject",
        "s3:GetObject"
      ]
      Resource = "${aws_s3_bucket.pdfs.arn}/*"
    }]
  })
}

Step 2: Update Rails Application

# app/services/pdf_generator_service.rb
class PdfGeneratorService
  def initialize
    @lambda = Aws::Lambda::Client.new(region: 'us-east-1')
    @function_name = 'pdf-generator'
  end

  def generate_invoice_pdf(invoice)
    html = ApplicationController.render(
      template: 'invoices/show',
      layout: 'pdf',
      assigns: { invoice: invoice }
    )

    payload = {
      html: html,
      invoice_id: invoice.id,
      bucket: ENV['PDF_BUCKET']
    }

    response = @lambda.invoke(
      function_name: @function_name,
      invocation_type: 'RequestResponse',
      payload: JSON.generate(payload)
    )

    result = JSON.parse(response.payload.read)
    
    if response.status_code == 200
      body = JSON.parse(result['body'])
      body['pdf_url']
    else
      raise "PDF generation failed: #{result['body']}"
    end
  end
end

# app/controllers/invoices_controller.rb
class InvoicesController < ApplicationController
  def show
    @invoice = Invoice.find(params[:id])
    
    respond_to do |format|
      format.html
      format.pdf do
        pdf_url = PdfGeneratorService.new.generate_invoice_pdf(@invoice)
        redirect_to pdf_url, allow_other_host: true
      end
    end
  end
end

For async generation:

# app/jobs/generate_invoice_pdf_job.rb
class GenerateInvoicePdfJob < ApplicationJob
  queue_as :default

  def perform(invoice_id)
    invoice = Invoice.find(invoice_id)
    pdf_url = PdfGeneratorService.new.generate_invoice_pdf(invoice)
    
    invoice.update!(pdf_url: pdf_url, pdf_generated_at: Time.current)
    
    # Notify user
    InvoiceMailer.pdf_ready(invoice).deliver_later
  end
end

Pattern 2: Image Processing with Memory Leaks

You’re using imagemagick via rmagick gem. It leaks memory like a sieve.

Lambda Function (Node.js)

// lambda/image_processor/index.js
const AWS = require('aws-sdk');
const sharp = require('sharp');

const s3 = new AWS.S3();

exports.handler = async (event) => {
  const { bucket, key, operations } = JSON.parse(event.body);
  
  try {
    // Download image from S3
    const image = await s3.getObject({ Bucket: bucket, Key: key }).promise();
    
    // Process with sharp (no memory leaks!)
    let processor = sharp(image.Body);
    
    // Apply operations
    if (operations.resize) {
      processor = processor.resize(operations.resize.width, operations.resize.height);
    }
    if (operations.format) {
      processor = processor.toFormat(operations.format);
    }
    if (operations.quality) {
      processor = processor.jpeg({ quality: operations.quality });
    }
    
    const processed = await processor.toBuffer();
    
    // Upload processed image
    const outputKey = key.replace(/(\.[^.]+)$/, `-processed$1`);
    await s3.putObject({
      Bucket: bucket,
      Key: outputKey,
      Body: processed,
      ContentType: `image/${operations.format || 'jpeg'}`
    }).promise();
    
    return {
      statusCode: 200,
      body: JSON.stringify({ key: outputKey })
    };
    
  } catch (error) {
    console.error('Error processing image:', error);
    return {
      statusCode: 500,
      body: JSON.stringify({ error: error.message })
    };
  }
};

Dockerfile:

FROM public.ecr.aws/lambda/nodejs:18

# Install sharp dependencies
RUN yum install -y \
    gcc-c++ \
    make \
    libpng-devel \
    libjpeg-turbo-devel \
    libwebp-devel

COPY package*.json ./
RUN npm ci --production

COPY index.js ./

CMD ["index.handler"]

Memory leak? Doesn’t matter. Lambda recycles the container after each invocation (or after a few).

Pattern 3: Legacy API Client with Security Issues

Old SOAP client library with known CVEs, but no maintained alternative.

Lambda as API Proxy

# lambda/legacy_api_client/handler.py
import json
from zeep import Client
from zeep.wsse.username import UsernameToken

def call_legacy_api(event, context):
    """
    Proxy to legacy SOAP API
    Isolated so security issues don't affect main app
    """
    try:
        action = event['action']
        params = event['params']
        
        # Initialize SOAP client (with vulnerable dependencies)
        client = Client(
            'https://legacy-api.example.com/service?wsdl',
            wsse=UsernameToken('user', 'pass')
        )
        
        # Call appropriate method
        if action == 'create_order':
            result = client.service.CreateOrder(**params)
        elif action == 'get_status':
            result = client.service.GetOrderStatus(**params)
        else:
            raise ValueError(f"Unknown action: {action}")
        
        return {
            'statusCode': 200,
            'body': json.dumps({'result': result})
        }
        
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Network isolation:

resource "aws_lambda_function" "legacy_api_client" {
  function_name = "legacy-api-client"
  # ... other config ...
  
  vpc_config {
    subnet_ids         = aws_subnet.private[*].id
    security_group_ids = [aws_security_group.lambda_legacy_api.id]
  }
}

resource "aws_security_group" "lambda_legacy_api" {
  name   = "lambda-legacy-api"
  vpc_id = aws_vpc.main.id

  # Only allow outbound to legacy API
  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["203.0.113.0/24"]  # Legacy API IP range
  }
}

Now the vulnerable library is:

Isolated in its own container
Can’t access your main database or internal services
Network-restricted to only the legacy API
Easy to replace when you find a better solution

When to Use This Pattern

Good candidates for isolation:

✅ Unmaintained dependencies blocking upgrades ✅ Libraries with known memory leaks ✅ Dependencies with security vulnerabilities ✅ Heavy processing (PDF, images, video) ✅ Infrequently used features ✅ Native extensions that break on OS upgrades ✅ Legacy protocol clients (SOAP, FTP, etc.)

Bad candidates:

❌ Core business logic ❌ High-frequency operations (>1000/sec) ❌ Operations requiring <10ms latency ❌ Stateful operations requiring session management ❌ Dependencies that are actively maintained

Cost Considerations

Lambda pricing (us-east-1):

$0.20 per 1M requests
$0.0000166667 per GB-second

Example: PDF generation

10,000 PDFs/month
1GB memory, 5 second duration
Cost: ~$8.33/month

Compare to:

Keeping old Rails version: opportunity cost of not upgrading
Forking and maintaining gem: developer time
Memory leaks crashing production: priceless

Migration Strategy

Phase 1: Identify risky dependencies

# Check for outdated gems
bundle outdated

# Check for security vulnerabilities
bundle audit

# Check for memory leaks
derailed_benchmarks:mem

Phase 2: Prioritize by risk

Blocking framework upgrades
Known security vulnerabilities
Stability issues (crashes, leaks)
Maintenance burden

Phase 3: Isolate one at a time

Build Lambda function
Add feature flag
Deploy Lambda
Route 10% of traffic to Lambda
Monitor for issues
Gradually increase to 100%
Remove old dependency

Phase 4: Upgrade main application

Now you can upgrade without the risky dependency holding you back.

Monitoring and Observability

Track Lambda performance:

# app/services/pdf_generator_service.rb
def generate_invoice_pdf(invoice)
  start_time = Time.current
  
  result = @lambda.invoke(...)
  
  duration = Time.current - start_time
  
  Rails.logger.info(
    "PDF generation completed",
    invoice_id: invoice.id,
    duration_ms: (duration * 1000).round,
    lambda_duration_ms: response.payload['duration'],
    lambda_memory_used_mb: response.payload['memory_used']
  )
  
  # Track metrics
  StatsD.increment('pdf.generated')
  StatsD.timing('pdf.generation_time', duration * 1000)
  
  result
end

Set up CloudWatch alarms:

resource "aws_cloudwatch_metric_alarm" "lambda_errors" {
  alarm_name          = "pdf-generator-errors"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = "2"
  metric_name         = "Errors"
  namespace           = "AWS/Lambda"
  period              = "300"
  statistic           = "Sum"
  threshold           = "10"
  alarm_description   = "Lambda function error rate too high"
  
  dimensions = {
    FunctionName = aws_lambda_function.pdf_generator.function_name
  }
}

The Bigger Picture

This pattern is about managing technical debt strategically. You can’t always eliminate it, but you can contain it.

Benefits beyond isolation:

Upgrade main application independently
Experiment with new solutions without risk
Scale problematic operations separately
Clear migration path when better alternatives emerge

Real-world impact:

At a previous company, we had a Rails 4 app stuck because of an unmaintained payment gateway gem. We:

Isolated the gem in Lambda (2 days)
Upgraded to Rails 7 (1 week)
Later replaced Lambda with modern payment API (1 day)

Total time: 2 weeks vs 6+ months of maintaining a fork.

Conclusion

Don’t let unmaintained dependencies hold your application hostage. Isolate them in Lambda containers where they can’t hurt you.

Your main application stays modern, secure, and maintainable. The risky code is quarantined, monitored, and easy to replace.

It’s not perfect, but it’s pragmatic. And pragmatic wins.