Dynamo: An Elegant Elixir DSL for DynamoDB

tags: #Elixir #Dynamodb #Aws #Database #Nosql
categories: Development Database
published: 2025-11-19
reading time: 8 minutes

An Ecto-inspired DSL that brings structure, type safety, and elegance to DynamoDB operations in Elixir.

If you’ve worked with DynamoDB in Elixir, you know the pain: manually managing partition and sort keys, wrestling with DynamoDB’s verbose JSON format, and constantly converting between Elixir types and DynamoDB’s type system. Enter Dynamo - an Ecto-inspired DSL that brings structure, type safety, and elegance to DynamoDB operations in Elixir.

Why Another DynamoDB Library?

DynamoDB’s schema-free nature is both a blessing and a curse. While it offers incredible flexibility and performance, it can lead to inconsistencies in your data model and verbose, error-prone code. Dynamo bridges this gap by providing:

Type Safety: Define schemas that enforce data consistency across your application
Familiar Syntax: An Ecto-inspired DSL that feels natural to Elixir developers
Automatic Key Management: No more manual partition and sort key generation
Single-Table Design Support: Built-in support for complex single-table patterns
Comprehensive GSI Support: Define and query Global Secondary Indexes with ease

Getting Started

Add Dynamo to your mix.exs:

def deps do
  [
    {:dynamo, github: "bmalum/dynamo"}
  ]
end

Define your first schema:

defmodule MyApp.User do
  use Dynamo.Schema

  item do
    table_name "users"
    
    field :id, partition_key: true
    field :email, sort_key: true
    field :name
    field :role, default: "user"
    field :active, default: true
  end
end

That’s it! You now have a fully functional schema with automatic key generation, type conversion, and default values.

Core Features

1. Schema Definition with Defaults

Dynamo schemas are clean and expressive:

defmodule MyApp.Product do
  use Dynamo.Schema, key_separator: "_"

  item do
    table_name "products"
    
    field :category_id, partition_key: true
    field :product_id, sort_key: true
    field :name
    field :price
    field :stock, default: 0
    field :active, default: true
  end
end

The key_separator option controls how composite keys are joined. You can configure this at the application, process, or schema level.

2. Automatic Key Generation

One of Dynamo’s most powerful features is automatic key generation. When you create a struct, Dynamo automatically generates the partition and sort keys based on your schema:

product = %MyApp.Product{
  category_id: "electronics",
  product_id: "prod-123",
  name: "Smartphone",
  price: 599.99
}

# After encoding, keys are automatically generated:
# pk = "product_electronics"
# sk = "prod-123"

3. CRUD Operations Made Simple

Dynamo provides clean, intuitive functions for all basic operations:

# Create
{:ok, saved_product} = MyApp.Product.put_item(product)

# Read
{:ok, retrieved_product} = MyApp.Product.get_item(
  %MyApp.Product{category_id: "electronics", product_id: "prod-123"}
)

# List with partition key
{:ok, products} = MyApp.Product.list_items(
  %MyApp.Product{category_id: "electronics"}
)

# Query with sort key conditions
{:ok, products} = MyApp.Product.list_items(
  %MyApp.Product{category_id: "electronics"},
  sort_key: "prod-",
  sk_operator: :begins_with,
  scan_index_forward: false
)

4. Global Secondary Indexes (GSIs)

GSIs are first-class citizens in Dynamo. Define them in your schema and query them with the same simple API:

defmodule MyApp.User do
  use Dynamo.Schema

  item do
    table_name "users"
    
    field :id, partition_key: true
    field :tenant
    field :email
    field :name
    field :status, default: "active"
    field :created_at, sort_key: true

    # Partition-only GSI
    global_secondary_index "EmailIndex", partition_key: :email

    # Partition + Sort GSI
    global_secondary_index "TenantIndex", 
      partition_key: :tenant, 
      sort_key: :created_at

    # GSI with custom projection
    global_secondary_index "TenantStatusIndex",
      partition_key: :tenant,
      sort_key: :status,
      projection: :include,
      projected_attributes: [:id, :email, :name]
  end
end

Querying GSIs is just as simple:

# Query by email
{:ok, users} = MyApp.User.list_items(
  %MyApp.User{email: "[email protected]"},
  index_name: "EmailIndex"
)

# Query by tenant with date range
{:ok, recent_users} = MyApp.User.list_items(
  %MyApp.User{tenant: "acme", created_at: "2023-01-01"},
  index_name: "TenantIndex",
  sk_operator: :gte
)

Dynamo automatically validates your GSI queries and provides helpful error messages:

# Missing partition key data
{:error, %Dynamo.Error{message: "GSI 'EmailIndex' requires field 'email' to be populated"}}

# Non-existent GSI
{:error, %Dynamo.Error{message: "GSI 'NonExistentIndex' not found. Available indexes: EmailIndex, TenantIndex"}}

5. Single-Table Design with `belongs_to`

One of Dynamo’s standout features is its support for single-table design patterns through the belongs_to relationship. This allows child entities to share their parent’s partition key, enabling efficient queries:

defmodule MyApp.Customer do
  use Dynamo.Schema

  item do
    table_name "app_data"
    
    field :customer_id, partition_key: true
    field :name
    field :email, sort_key: true
  end
end

defmodule MyApp.Order do
  use Dynamo.Schema

  item do
    table_name "app_data"
    
    field :customer_id  # Foreign key - auto-inferred!
    field :order_id
    field :total_amount
    field :created_at, sort_key: true

    # Child uses parent's partition key format
    belongs_to :customer, MyApp.Customer, sk_strategy: :prefix
  end
end

With this setup, your data is efficiently organized in DynamoDB:

Partition Key: "customer#cust-123"
├── Sort Key: "[email protected]"        → Customer record
├── Sort Key: "order#2024-01-15"        → Order record
└── Sort Key: "order#2024-01-20"        → Order record

Querying all orders for a customer is now a single, efficient partition query:

# Gets all orders for customer-123
{:ok, orders} = MyApp.Order.list_items(
  %MyApp.Order{customer_id: "cust-123"}
)

The sk_strategy option controls how sort keys are generated:

:prefix - Prefixes the sort key with the entity name (e.g., “order#2024-01-15”)
:use_defined - Uses the sort key as-is without prefixing

6. Batch Operations and Parallel Scans

For high-performance scenarios, Dynamo provides batch operations and parallel scanning:

# Batch write
products = [
  %MyApp.Product{category_id: "electronics", product_id: "prod-123", name: "Smartphone"},
  %MyApp.Product{category_id: "electronics", product_id: "prod-124", name: "Laptop"},
  %MyApp.Product{category_id: "electronics", product_id: "prod-125", name: "Tablet"}
]

{:ok, result} = Dynamo.Table.batch_write_item(products)

# Parallel scan for large tables
{:ok, all_products} = Dynamo.Table.parallel_scan(
  MyApp.Product,
  segments: 8,
  filter_expression: "category_id = :category",
  expression_attribute_values: %{":category" => %{"S" => "electronics"}}
)

7. Transaction Support

Dynamo supports DynamoDB transactions for atomic multi-item operations:

# Transfer money between accounts atomically
Dynamo.Transaction.transact([
  # Check source account has sufficient funds
  {:check, %Account{id: "account-123"},
    "balance >= :amount",
    %{":amount" => %{"N" => "100.00"}}},
    
  # Decrease source account balance
  {:update, %Account{id: "account-123"},
    %{balance: {:decrement, 100.00}}},
    
  # Increase destination account balance
  {:update, %Account{id: "account-456"},
    %{balance: {:increment, 100.00}}}
])

8. Flexible Configuration

Dynamo provides three levels of configuration to suit your needs:

Application-level (in config.exs):

config :dynamo,
  partition_key_name: "pk",
  sort_key_name: "sk",
  key_separator: "#",
  prefix_sort_key: false

Process-level (runtime):

Dynamo.Config.put_process_config(key_separator: "-")

Schema-level:

defmodule MyApp.User do
  use Dynamo.Schema,
    key_separator: "_",
    prefix_sort_key: true
    
  # schema definition...
end

9. Built-in Debugging

Dynamo includes a logging system to help debug DynamoDB operations:

# Enable logging
Dynamo.Logger.enable()

# Perform operations - queries are logged in JSON format
{:ok, user} = MyApp.User.get_item(%MyApp.User{id: "user-123"})

# Disable logging
Dynamo.Logger.disable()

Log output includes timestamps, operations, table names, payloads, and responses - perfect for debugging complex queries.

Mix Tasks for Table Management

Dynamo provides convenient Mix tasks for managing DynamoDB tables:

# Create a table
mix dynamo.create_table users

# Create with custom keys
mix dynamo.create_table products --partition-key category_id --sort-key product_id

# List tables
mix dynamo.list_tables

# Delete a table
mix dynamo.delete_table old_users --force

# Generate schema from existing table
mix dynamo.generate_schema users --module MyApp.User

Error Handling

Dynamo provides comprehensive error handling with meaningful error types:

case MyApp.User.get_item(%User{id: "user-123"}) do
  {:ok, user} -> 
    IO.puts("Found user: #{user.name}")
    
  {:error, %Dynamo.Error{type: :resource_not_found}} ->
    IO.puts("User not found")
    
  {:error, %Dynamo.Error{type: :provisioned_throughput_exceeded}} ->
    IO.puts("Rate limit exceeded")
    
  {:error, %Dynamo.Error{} = error} ->
    IO.puts("Error: #{error.message}")
end

Common error types include:

:resource_not_found
:provisioned_throughput_exceeded
:conditional_check_failed
:validation_error
:access_denied
:transaction_conflict

Advanced Features

Custom Encoding/Decoding

Implement the Dynamo.Encodable and Dynamo.Decodable protocols for custom types:

defimpl Dynamo.Encodable, for: MyApp.CustomType do
  def encode(value, _options) do
    %{"S" => to_string(value)}
  end
end

Custom Key Generation

Override the before_write/1 function to customize key generation:

defmodule MyApp.TimeSeries do
  use Dynamo.Schema

  item do
    field :device_id, partition_key: true
    field :timestamp, sort_key: true
    field :value
  end
  
  def before_write(item) do
    item = if is_nil(item.timestamp) do
      %{item | timestamp: DateTime.utc_now() |> DateTime.to_iso8601()}
    else
      item
    end
    
    item
    |> Dynamo.Schema.generate_and_add_partition_key()
    |> Dynamo.Schema.generate_and_add_sort_key()
    |> Dynamo.Encoder.encode_root()
  end
end

Best Practices

Use descriptive GSI names that indicate their purpose
Design GSI partition keys to distribute data evenly
Leverage single-table design with belongs_to for related entities
Use appropriate projection types to balance performance and cost
Implement pagination for large result sets
Enable logging during development for debugging
Use batch operations for bulk writes
Leverage transactions for atomic multi-item operations

Real-World Example

Here’s a complete example of a blog application using Dynamo:

defmodule Blog.User do
  use Dynamo.Schema

  item do
    table_name "blog_data"
    
    field :email, partition_key: true
    field :role, sort_key: true, default: "user"
    field :display_name
    field :bio
    field :created_at
    
    global_secondary_index "RoleIndex", 
      partition_key: :role,
      sort_key: :created_at
  end
end

defmodule Blog.Post do
  use Dynamo.Schema

  item do
    table_name "blog_data"
    
    field :email  # Foreign key
    field :title, sort_key: true
    field :body
    field :published_at
    field :status, default: "draft"
    
    belongs_to :user, Blog.User, sk_strategy: :prefix
    
    global_secondary_index "StatusIndex",
      partition_key: :status,
      sort_key: :published_at
  end
end

# Usage
user = %Blog.User{
  email: "[email protected]",
  display_name: "John Doe",
  bio: "Elixir enthusiast"
}
{:ok, _} = Blog.User.put_item(user)

post = %Blog.Post{
  email: "[email protected]",
  title: "Getting Started with Dynamo",
  body: "Dynamo makes DynamoDB easy...",
  status: "published",
  published_at: DateTime.utc_now() |> DateTime.to_iso8601()
}
{:ok, _} = Blog.Post.put_item(post)

# Get all posts by user (efficient single partition query)
{:ok, user_posts} = Blog.Post.list_items(
  %Blog.Post{email: "[email protected]"}
)

# Get all published posts (using GSI)
{:ok, published_posts} = Blog.Post.list_items(
  %Blog.Post{status: "published"},
  index_name: "StatusIndex"
)

Conclusion

Dynamo brings the elegance and developer experience of Ecto to DynamoDB. Whether you’re building a simple CRUD application or implementing complex single-table designs, Dynamo provides the tools you need to work efficiently with DynamoDB in Elixir.

Key takeaways:

Ecto-inspired DSL makes DynamoDB feel natural in Elixir
Automatic key management eliminates boilerplate
First-class GSI support with automatic validation
Single-table design patterns made easy with belongs_to
Comprehensive tooling including Mix tasks and debugging

While Dynamo is not yet published on Hex, it’s actively developed and ready for use. Check out the GitHub repository for more examples, documentation, and to contribute.

If you’re tired of wrestling with DynamoDB’s low-level API and want to bring structure and elegance to your Elixir applications, give Dynamo a try. Your future self will thank you.

Have you used Dynamo in your projects? Share your experiences in the comments below!