Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,11 @@ packages/
│ │ ├── parser.rs # SQL parsing entry point
│ │ └── context/ # Session state (statements, portals, metadata)
│ ├── proxy/ # Encryption service, schema management, config
│ └── config/ # Configuration parsing
│ ├── config/ # Configuration parsing
│ ├── cli/ # CLI argument parsing
│ ├── connect/ # Database connection management
│ ├── tls/ # TLS configuration and setup
│ └── log/ # Logging targets and configuration
├── eql-mapper/ # SQL type inference and transformation
│ └── src/
│ ├── inference/ # Type inference engine
Expand Down
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Changed

- **Log target renamed**: `KEYSET` log target renamed to `ZEROKMS`. The environment variable `CS_LOG__KEYSET_LEVEL` is now `CS_LOG__ZEROKMS_LEVEL`.
- **Dependency upgrade**: Updated `cipherstash-client` from 0.33.2 to 0.34.0-alpha.4 with new `ZeroKMSBuilder` API and `AutoStrategy` authentication.

### Removed

- **Log target removed**: `PROXY` log target and `CS_LOG__PROXY_LEVEL` environment variable have been removed.

### Added

- **Cipher cache miss metric**: New Prometheus counter `cipherstash_proxy_keyset_cipher_cache_miss_total` tracks cache misses requiring cipher initialization. This complements the `cipherstash_proxy_keyset_cipher_cache_hit_total` metric, and can be used to calculate cache hit/miss ratio.
- **Cipher cache miss metric**: New Prometheus counter `cipherstash_proxy_keyset_cipher_cache_miss_total` tracks cache misses requiring cipher initialization. This complements the `cipherstash_proxy_keyset_cipher_cache_hits_total` metric, and can be used to calculate cache hit/miss ratio.
- **Cipher init duration metric**: New Prometheus histogram `cipherstash_proxy_keyset_cipher_init_duration_seconds` tracks cipher initialization time including ZeroKMS network calls.
- **Encrypt/decrypt timing**: Debug logs for `encrypt_eql` and `decrypt_eql` now include `duration_ms`.
- **Cache eviction logging**: ScopedCipher cache eviction events are now logged under the `ZEROKMS` target.
Expand Down
11 changes: 7 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ Key capabilities:

**Core Proxy (`packages/cipherstash-proxy/`):**
- `postgresql/` - PostgreSQL wire protocol implementation, message parsing, and client handling
- `encrypt/` - Integration with CipherStash ZeroKMS for key management and encryption operations
- `proxy/zerokms/` - ZeroKMS client initialization and key management
- `config/` - Configuration management for database connections, TLS, and encryption settings
- `eql/` - EQL v2 types and encryption abstractions
- `proxy/encrypt_config/` - Encryption configuration and schema management

**EQL Mapper (`packages/eql-mapper/`):**
- SQL parsing and type inference engine
Expand All @@ -29,7 +29,9 @@ Key capabilities:

**Integration Tests (`packages/cipherstash-proxy-integration/`):**
- Comprehensive test suite covering encryption scenarios
- Language-specific integration tests (Python, Go, Elixir)

**Language Integration Tests (`tests/python/`, `tests/integration/golang/`):**
- Language-specific integration tests (Python, Go)

**Showcase (`packages/showcase/`):**
- Healthcare data model demonstrating EQL v2 encryption
Expand Down Expand Up @@ -76,7 +78,7 @@ mise run reset
# Full test suite (hygiene + unit + integration)
mise run test

# Hygiene checks only
# Hygiene checks (compilation, formatting, clippy)
mise run check

# Unit tests only
Expand Down Expand Up @@ -111,6 +113,7 @@ mise run postgres:down
### Authentication & Encryption
Proxy requires CipherStash credentials configured in `mise.local.toml`:
```toml
[env]
CS_WORKSPACE_CRN = "crn:region:workspace-id"
CS_CLIENT_ACCESS_KEY = "your-access-key"
CS_DEFAULT_KEYSET_ID = "your-keyset-id"
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ For more information see the [Code of Conduct FAQ](./CODE_OF_CONDUCT.md) or cont

## Licensing

See the [LICENSE](./LICENSE) file for our project's licensing.
See the [LICENSE](./LICENSE.md) file for our project's licensing.
18 changes: 10 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ To use a different version of EQL, set the path to the desired EQL release file

PostgreSQL port numbers are 4 digits:

- The first two digits denote non-TLS (`55`) or non-TLS (`56`)
- The first two digits denote non-TLS (`55`) or TLS (`56`)
- The last two digits denote the version of PostgreSQL

PostgreSQL latest always runs on `5532`.
Expand Down Expand Up @@ -470,7 +470,7 @@ All containers use the same credentials and database, defined in `tests/pg/commo
POSTGRES_DB="cipherstash"
POSTGRES_USER="cipherstash"
PGUSER="cipherstash"
POSTGRES_PASSWORD="password"
POSTGRES_PASSWORD="p@ssword"
```

PostgreSQL configuration files live at:
Expand All @@ -496,7 +496,7 @@ Environment files:

If you ever get confused about where your configuration is coming from, run `mise cfg` to get a list of config files in use.

Certificates are generated by `mkcert`, and live in `tests/tls/`.
Certificates are generated by `openssl`, and live in `tests/tls/`.


#### Configuration: development endpoints
Expand Down
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,15 @@ cd proxy
# Sign up, create a workspace, and generate credentials
# Visit: https://dashboard.cipherstash.com/sign-up

# Put credentials in .env.proxy.docker
# Copy the credentials from the dashboard and paste them into .env.proxy.docker using your preferred text editor.
nano .env.proxy.docker
# Create .env.proxy.docker with your CipherStash credentials
# Replace the placeholder values with your actual credentials from the dashboard
cat > .env.proxy.docker << 'EOF'
CS_WORKSPACE_CRN=crn:...your-workspace-crn...
CS_CLIENT_ACCESS_KEY=your-client-access-key
CS_ENCRYPT__DEFAULT_KEYSET_ID=your-keyset-id
CS_ENCRYPT__CLIENT_ID=your-client-id
CS_ENCRYPT__CLIENT_KEY=your-client-key
EOF

# Start the containers
docker compose up
Expand Down
11 changes: 5 additions & 6 deletions cipherstash-proxy-example.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
[tls]
type = "Path" # To provide paths to PEM files
certificate = "tests/tls/server.cert"
private_key = "tests/tls/server.key"
certificate_path = "tests/tls/server.cert"
private_key_path = "tests/tls/server.key"

# type = "Pem" # To directly provide PEM contents
# certificate = """-----BEGIN CERTIFICATE-----
# To directly provide PEM contents:
# certificate_pem = """-----BEGIN CERTIFICATE-----
# ...your certificate content here...
# -----END CERTIFICATE-----
# """
# private_key = """-----BEGIN PRIVATE KEY-----
# private_key_pem = """-----BEGIN PRIVATE KEY-----
# ...your private key content here...
# -----END PRIVATE KEY-----
# """
Expand Down
47 changes: 43 additions & 4 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
- Authentication errors:
- [Database](#authentication-failed-database)
- [Client](#authentication-failed-client)
- [ZeroKMS](#zerokms-authentication-failed)

- Mapping errors:
- [Invalid parameter](#mapping-invalid-parameter)
Expand All @@ -15,8 +16,7 @@

- Encrypt errors:
- [Column could not be encrypted](#encrypt-column-could-not-be-encrypted)
- [Column could not be encrypted](#encrypt-column-could-not-be-encrypted)
- [Could not decrypt data for keyset](#encrypt-encrypt-could-not-decrypt-data-for-keyset)
- [Could not decrypt data for keyset](#encrypt-could-not-decrypt-data-for-keyset)
- [KeysetId could not be parsed](#encrypt-keyset-id-could-not-be-parsed)
- [KeysetId could not be set](#encrypt-keyset-id-could-not-be-set)
- [KeysetName could not be set](#encrypt-keyset-name-could-not-be-set)
Expand All @@ -26,6 +26,8 @@
- [Unknown table](#encrypt-unknown-table)
- [Unknown index term](#encrypt-unknown-index-term)
- [Column configuration mismatch](#encrypt-column-config-mismatch)
- [Missing encrypt configuration](#encrypt-missing-encrypt-configuration)
- [Unexpected SET keyset](#encrypt-unexpected-set-keyset)

- Decrypt errors:
- [Column could not be deserialised](#encrypt-column-could-not-be-deserialised)
Expand Down Expand Up @@ -91,7 +93,7 @@ Client authentication failed. Check username and password.
<!-- ---------------------------------------------------------------------------------------------------- -->


## ZeroKMS <a id='authentication-failed-zerokms'></a>
## ZeroKMS <a id='zerokms-authentication-failed'></a>

Authentication failed when connecting to ZeroKMS.

Expand Down Expand Up @@ -184,7 +186,7 @@ The parameter type is not supported.
### Error message

```
Encryption of PostgreSQL {name} (OID {oid}) types is not currently supported.
Encryption of EQL column {column_type} using strategy {eql_term} is not supported.
```

### How to fix
Expand Down Expand Up @@ -218,6 +220,8 @@ When `mapping_errors_enabled` is `false` (the default), then type check errors a

When `mapping_errors_enabled` is `true`, then type check errors are raised, and statement execution halts.

Configure this setting with the environment variable `CS_DEVELOPMENT__ENABLE_MAPPING_ERRORS` or in the TOML config file under `[development] enable_mapping_errors = true`.

In our experience, most production systems have a relatively small number of columns that require protection.
As SQL is large and complex, instead of blocking statements with type check errors that are false negatives, the default behaviour of Proxy is to allow the statement.

Expand Down Expand Up @@ -577,6 +581,41 @@ If the error persists, please contact CipherStash [support](https://cipherstash.
<!-- ---------------------------------------------------------------------------------------------------- -->


## Missing encrypt configuration <a id='encrypt-missing-encrypt-configuration'></a>

The encrypted column type does not have a matching encrypt configuration.


### Error message

```
Missing encrypt configuration for column type `{plaintext_type}`.
```

### How to fix

1. Define the encrypted configuration for the column type using [EQL](https://github.com/cipherstash/encrypt-query-language).
2. If this error persists, please contact CipherStash [support](https://cipherstash.com/support) as this may indicate a bug.


<!-- ---------------------------------------------------------------------------------------------------- -->


## Unexpected SET keyset <a id='encrypt-unexpected-set-keyset'></a>

A `SET CIPHERSTASH.KEYSET` statement was used when a default keyset has already been configured.


### Error message

```
Cannot SET CIPHERSTASH.KEYSET if a default keyset has been configured.
```

### How to fix

1. Remove the `SET CIPHERSTASH.KEYSET` statement from your application code.
2. Or remove the `default_keyset_id` from the proxy configuration to allow dynamic keyset selection.


<!-- ---------------------------------------------------------------------------------------------------- -->
Expand Down
28 changes: 24 additions & 4 deletions docs/how-to/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ services:
```


For a fully-working example, go to [`docker-compose.yml`](./docker-compose.yml).
For a fully-working example, go to [`docker-compose.yml`](../../docker-compose.yml).
Follow the steps in [Getting started](../README.md#getting-started) to see it in action.

Once you have set up a `docker-compose.yml`, start the Proxy container:
Expand Down Expand Up @@ -132,7 +132,27 @@ Read the full list of configuration options and what they do in the [reference d

## Running Proxy locally

TODO: Add instructions for running Proxy locally
To run CipherStash Proxy locally for development:

```bash
# Install prerequisites
mise trust --yes && mise install

# Start PostgreSQL and install EQL
mise run postgres:up --extra-args "--detach --wait"
mise run postgres:setup

# Run Proxy as a local process
mise run proxy
```

Alternatively, run Proxy in a container:

```bash
mise run proxy:up --extra-args "--detach --wait"
```

See [Configuring Proxy](#configuring-proxy) for required environment variables and configuration options.

## Setting up the database schema

Expand Down Expand Up @@ -223,14 +243,14 @@ The first SQL statement adds a `match` index, which is used for partial matches
The second SQL statement adds an `ore` index, which is used for ordering with `ORDER BY`.


> ![IMPORTANT]
> [!IMPORTANT]
> Adding, updating, or deleting encrypted indexes on columns that already contain encrypted data will not re-index that data. To use the new indexes, you must `SELECT` the data out of the column, and `UPDATE` it again.

To learn how to use encrypted indexes for other encrypted data types like `text`, `int`, `boolean`, `date`, and `jsonb`, see the [EQL documentation](https://github.com/cipherstash/encrypt-query-language/blob/main/docs/reference/INDEX.md).

When deploying CipherStash Proxy into production environments with real data, we recommend that you apply these database schema changes with the normal tools and process you use for making changes to your database schema.

To see more examples of how to modify your database schema, check out [the example schema](./sql/schema-example.sql) from [Getting started](#getting-started).
To see more examples of how to modify your database schema, check out [the example schema](../sql/schema-example.sql) from [Getting started](#getting-started).

## Encrypting data in an existing database

Expand Down
Loading
Loading