Entity Framework

warning

Entity Framework support is still in beta and is not suitable for production databases.

info

This page refers to setting up a Bitwarden instance to develop on, for instructions on testing out our EF deployments for personal use, such as Bitwarden Unified, please see the help documentation.

Background

Entity Framework (EF) is an ORM framework that acts as a wrapper around a database. It allows us to support multiple (non-MSSQL) databases without having to maintain migration and query scripts for each.

Our EF implementations currently support Postgres, MySQL, and SQLite3.

Creating the database

The workflow here is broadly the same as with the normal MSSQL implementation: set up the Docker container, configure user secrets, and run migrations against their relating databases in chronological order.

Requirements

• A working local development server
• Docker
• A way to manage user secrets in the server project - see User Secrets
• Database management software (see tools recommendations)
• The dotnet CLI
• The dotnet CLI Entity Framework Core tool

You can have multiple databases configured and switch between them by changing the value of the globalSettings:databaseProvider user secret. You don’t have to delete your connection strings.

User secrets

Add the following values to your API, Identity, and Admin user secrets.

PostgreSQL

Be sure to change information like root password as needed. If you already have these secrets, make sure you update the existing values instead of creating new ones

"globalSettings:databaseProvider": "postgres",
"globalSettings:postgreSql:connectionString": "Host=localhost;Username=postgres;Password=example;Database=vault_dev;Include Error Detail=true",

MySQL

Be sure to change information like root password as needed. If you already have these secrets, make sure you update the existing values instead of creating new ones.

"globalSettings:databaseProvider": "mysql",
"globalSettings:mySql:connectionString": "server=localhost;uid=root;pwd=example;database=vault_dev",

MariaDB

MariaDB is generally a drop-in alternative to MySQL, so it shares the mysql configuration keys.

Be sure to change information like root password as needed. If you already have these secrets, make sure you update the existing values instead of creating new ones.

"globalSettings:databaseProvider": "mysql",
"globalSettings:mySql:connectionString": "server=localhost;uid=root;pwd=example;database=vault_dev",

SQLite

Add the following values to your API, Identity, and Admin user secrets. Note, you must set the Data Source path. Git is configured to ignore .db files in the server repository so that the sqlite database can be stored in dev. You can use any path with write permissions.

"globalSettings:databaseProvider": "sqlite",
"globalSettings:sqlite:connectionString": "Data Source=/path/to/your/server/repo/dev/bitwarden.db",

note

After making changes to your secrets.json file, remember to run pwsh setup_secrets.ps1 -clear so that the changes take effect.

Start the database server

PostgreSQL

1. Confirm that POSTGRES_PASSWORD in dev/.env matches the password in dev/secrets.json.

2. In the dev folder of your server repository, run

docker compose --profile postgres up

Confirm your database connection!

If you run into connection errors, double check that your .env and secrets.json files have matching passwords. If they do, you may have initialized your database incorrectly. Delete the Docker storage volume and initialize the database from scratch.

MySQL

1. Confirm that MYSQL_ROOT_PASSWORD in dev/.env matches the password in dev/secrets.json.

2. In the dev folder of your server repository, start your database container:

docker compose --profile mysql up

Confirm your database connection!

If you run into connection errors, double check that your .env and secrets.json files have matching passwords. If they do, you may have initialized your database incorrectly. Delete the Docker storage volume and initialize the database from scratch.

MariaDB

1. Confirm that MYSQL_ROOT_PASSWORD in dev/.env matches the password in dev/secrets.json.

2. In the dev folder of your server repository, run

docker compose --profile mariadb up

Confirm your database connection!

If you run into connection errors, double check that your .env and secrets.json files have matching passwords. If they do, you may have initialized your database incorrectly. Delete the Docker storage volume and initialize the database from scratch.

SQLite

No additional step is required for SQLite. The migrator will create the database file if it doesn't exist.

Confirm your database path!

The migrator does not create folders. If you get an error that the path doesn't exist, double check that the path exists and that the folder containing the sqlite database has write and/or create permissions.

Start all containers at once

Use the ef profile to start all EntityFramework database containers at once:

docker compose --profile ef up

Run migrations

In the dev folder, run the following command to update the database to the latest migration:

PostgreSQL

pwsh migrate.ps1 -postgres

MySQL

pwsh migrate.ps1 -mysql

MariaDB

pwsh migrate.ps1 -mariadb

SQLite

pwsh migrate.ps1 -sqlite

note

The migrator creates the database file if it doesn't exist, but it does not create folders. If you get an error that the path doesn't exist, it's referring to missing folders.

Verify changes

If you would like to verify that everything worked correctly:

• Check the database tables to make sure everything has been created
• Run the integration tests from the root of your server project using dotnet test.
  • Note: this requires a configured MSSQL database. You may also need to set up other EF providers for tests to pass.

Database integration tests

Database integration tests run for each database provider for both Dapper (MSSQL) and Entity Framework. Developers are not expected to manually test each database provider. Instead, use integration tests to ensure correctness across all supported databases.

Integration tests have their own connection strings, so that you can use separate databases to the one used by your local development server. This is recommended because integration tests produce a lot of test data over time. In the example below, this is done by using vault_test as the database name.

Configuring test databases

1. In your user secrets, find or add this block of secrets in the root of the json structure (not in GlobalSettings):

"databases:0:type": "Postgres",
"databases:0:connectionString": "Host=localhost;Username=postgres;Password=_________;Database=vault_test",
"databases:0:enabled": "true",
"databases:1:type": "Sqlite",
"databases:1:enabled": "true",
"databases:1:connectionString": "Data Source=_________",
"databases:2:type": "MySql",
"databases:2:connectionString": "server=localhost;uid=root;pwd=_________;database=vault_test",
"databases:2:enabled": "true",
"databases:3:type": "SqlServer",
"databases:3:connectionString": "Server=localhost;Database=vault_test;User Id=SA;Password=_________;Encrypt=True;TrustServerCertificate=True;",
"databases:3:enabled": "true"
"databases:4:type": "MySql",
"databases:4:connectionString": "server=localhost;port=4306;uid=maria;pwd=_________;database=vault_test;AllowUserVariables=true",
"databases:4:enabled": "true",

note

The second MySql entry refers to MariaDB.

info

The example database index + type combinations are required for the tooling to work, and to support multiple versions of the same database running tests at the same time.

This block is used for test databases for each supported provider type. These are what integration tests will connect to. Make sure that you fill in the password for each connection string.

2. Run pwsh setup_secrets.ps1 -clear to apply the updated user secrets to your local projects.

3. Ensure your databases are all migrated (see instructions above). You can use the -test flag to only migrate the integration test databases, or -all to migrate everything.

   # Migrate a specific integration test database (used by tests)
   pwsh migrate.ps1 -postgres -test
   
   # Migrate a specific development database (used by your local server)
   pwsh migrate.ps1 -postgres
   
   # Migrate all local databases
   pwsh migrate.ps1 -all

4. Run integration tests from the test/Infrastructure.IntegrationTest folder using dotnet test.

Writing integration tests

See Contributing - Database Integration Testing for more information on testing utilities and patterns.

Modifying the database

The process for modifying the database is described in Migrations.