Search...
ctrl/
Light
Dark
System
Sign in

Migrations​

Gel's baked-in migration system lets you painlessly evolve your schema over time. Just update the contents of your .gel file(s) and use the Gel CLI to create and apply migrations.

Copy
$ 
gel migration create
Created dbschema/migrations/00001.edgeql
Copy
$ 
gel migrate
Applied dbschema/migrations/00001.edgeql

Refer to the creating and applying migrations guide for more information on how to use the migration system.

This document describes how migrations are implemented.

The migrations flow​

The migration flow is as follows:

  1. The user edits the .gel files in the dbschema directory.

    This makes the schema described in the .gel files different from the actual schema in the database.

  2. The user runs the gel migration create command to create a new migration (a sequence of low-level DDL commands).

    • The CLI reads the .gel files and sends them to the Gel server, to analyze the changes.

    • The Gel server generates a migration plan and sends it back to the CLI.

    • The migration plan might require clarification from the user.

      If so, the CLI and the Gel server will go back and forth presenting the user with a sequence of questions, until the migration plan is clear and approved by the user.

  3. The CLI writes the migration plan to a new file in the dbschema/migrations directory.

  4. The user runs the gel migrate command to apply the migration to the database.

  5. The user checks in the updated .gel files and the new dbschema/migrations migration file (created by gel migration create) into version control.

Command line tools​

The two most important commands are:

  • gel migration create

  • gel migrate

Automatic migrations​

Sometimes when you're prototyping something new you don't want to spend time worrying about migrations. There's no data to lose and not much code that depends on the schema just yet.

For this use case you can use the gel watch command, which will monitor your .gel files and automatically create and apply migrations for you in the background.

Data definition language (DDL)​

The migration plan is a sequence of DDL commands. DDL commands are low-level instructions that describe the changes to the schema.

SDL and your .gel files are like a 3D printer: you design the final shape, and the system puts a database together for you. Using DDL is like building a house the traditional way: to add a window, you first need a frame; to have a frame, you need a wall; and so on.

If your schema looks like this:

Copy
type User {
  required name: str;
}

then the corresponding DDL might look like this:

Copy
create type User {
  create required property name: str;
}

There are some circumstances where users might want to use DDL directly. But in most cases you just need to learn how to read them to understand the migration plan. Luckily, the DDL and SDL syntaxes were designed in tandem and are very similar.

Most documentation pages on Gel's schema have a section about DDL commands, e.g. object types DDL.

Migration DDL commands​

Migrations themselves are a sequence of special DDL commands.

Like all DDL commands, start migration and other migration commands are considered low-level. Users are encouraged to use the built-in migration tools instead.

However, if you want to implement your own migration tools, this section will give you a good understanding of how Gel migrations work under the hood.

Start migration​

Start a migration block.

start migration to "{"
    sdl-declaration ;
    [ ... ]
"}" ;

Parameters​

sdl-declaration

Complete schema text (content of all .gel files) defined with the declarative Gel schema definition language.

Description​

The command start migration defines a migration of the schema to a new state. The target schema state is described using SDL and describes the entire schema. This is important to remember when creating a migration to add a few more things to an existing schema as all the existing schema objects and the new ones must be included in the start migration command. Objects that aren't included in the command will be removed from the new schema (which may result in data loss).

This command also starts a transaction block if not inside a transaction already.

While inside a migration block, all issued EdgeQL statements are not executed immediately and are instead recorded to be part of the migration script. Aside from normal EdgeQL commands the following special migration commands are available:

  • describe current migration – return a list of statements currently recorded as part of the migration;

  • populate migration – auto-populate the migration with system-generated DDL statements to achieve the target schema state;

  • abort migration – abort the migration block and discard the migration;

  • commit migration – commit the migration by executing the migration script statements and recording the migration into the system migration log.

Example​

Create a new migration to a target schema specified by the Gel Schema syntax:

Copy
start migration to {
    module default {
        type User {
            property username: str;
        };
    };
};

create migration​

Create a new migration using an explicit EdgeQL script.

create migration "{"
    edgeql-statement ;
    [ ... ]
"}" ;

Parameters​

edgeql-statement

Any valid EdgeQL statement, except database, branch, role, configure, migration, or transaction statements.

Description​

The command create migration executes all the nested EdgeQL commands and records the migration into the system migration log.

Example​

Create a new migration to a target schema specified by the Gel Schema syntax:

Copy
create migration {
    create type default::User {
        create property username: str;
    }
};

Abort migration​

Abort the current migration block and discard the migration.

abort migration ;

Description​

The command abort migration is used to abort a migration block started by start migration. Issuing abort migration outside of a migration block is an error.

Example​

Start a migration block and then abort it:

Copy
start migration to {
    module default {
        type User;
    };
};

abort migration;

Populate migration​

Populate the current migration with system-generated statements.

populate migration ;

Description​

The command populate migration is used within a migration block started by start migration to automatically fill the migration with system-generated statements to achieve the desired target schema state. If the system is unable to automatically find a satisfactory sequence of statements to perform the migration, an error is returned. Issuing populate migration outside of a migration block is also an error.

The statements generated by populate migration may drop schema objects, which may result in data loss. Make sure to inspect the generated migration using describe current migration before running commit migration!

Example​

Start a migration block and populate it with auto-generated statements.

Copy
start migration to {
    module default {
        type User;
    };
};

populate migration;

Describe current migration​

Describe the migration in the current migration block.

describe current migration [ as {ddl | json} ];

Description​

The command describe current migration generates a description of the migration in the current migration block in the specified output format:

as ddl

Show a sequence of statements currently recorded as part of the migration using valid DDL syntax. The output will indicate if the current migration is fully defined, i.e. the recorded statements bring the schema to the state specified by start migration.

as json

Provide a machine-readable description of the migration using the following JSON format:

{
  // Name of the parent migration
  "parent": "<parent-migration-name>",

  // Whether the confirmed DDL makes the migration complete,
  // i.e. there are no more statements to issue.
  "complete": {true|false},

  // List of confirmed migration statements
  "confirmed": [
    "<stmt text>",
    ...
  ],

  // The variants of the next statement
  // suggested by the system to advance
  // the migration script.
  "proposed": {
    "statements": [{
      "text": "<stmt text template>"
    }],
    "required-user-input": [
      {
        "placeholder": "<placeholder variable>",
        "prompt": "<statement prompt>"
      },
      ...
    ],
    "confidence": (0..1), // confidence coefficient
    "prompt": "<operation prompt>",
    "prompt_id": "<prompt id>",
    // Whether the operation is considered to be non-destructive.
    "data_safe": {true|false}
  }
}

Where:

<stmt text>

Regular statement text.

<stmt text template>

Statement text template with interpolation points using the \(name) syntax.

<placeholder variable>

The name of an interpolation variable in the statement text template for which the user prompt is given.

<statement prompt>

The text of a user prompt for an interpolation variable.

<operation prompt>

Prompt for the proposed migration step.

<prompt id>

An opaque string identifier for a particular operation prompt. The client should not repeat prompts with the same prompt id.

Commit migration​

Commit the current migration to the database.

commit migration ;

Description​

The command commit migration executes all the commands defined by the current migration and records the migration as the most recent migration in the database.

Issuing commit migration outside of a migration block initiated by start migration is an error.

Example​

Create and execute the current migration:

Copy
commit migration;

Reset schema to initial​

Reset the database schema to its initial state.

reset schema to initial ;

This command will drop all entities and, as a consequence, all data. You won't want to use this statement on a production instance unless you want to lose all that instance's data.

Migration rewrites DDL commands​

Migration rewrites allow you to change the migration history as long as your final schema matches the current database schema.

Start migration rewrite​

Start a migration rewrite.

start migration rewrite ;

Once the migration rewrite is started, you can run any arbitrary DDL until you are ready to commit your new migration history. The most useful DDL in this context will be create migration statements, which will allow you to create a sequence of migrations that will become your new migration history.

Declare savepoint​

Establish a new savepoint within the current migration rewrite.

declare savepoint savepoint-name ;

Parameters​

savepoint-name

The name which will be used to identify the new savepoint if you need to later release it or roll back to it.

Release savepoint​

Destroys a savepoint previously defined in the current migration rewrite.

release savepoint savepoint-name ;

Parameters​

savepoint-name

The name of the savepoint to be released.

Rollback to savepoint​

Rollback to the named savepoint.

rollback to savepoint savepoint-name ;

All changes made after the savepoint are discarded. The savepoint remains valid and can be rolled back to again later, if needed.

Parameters​

savepoint-name

The name of the savepoint to roll back to.

Rollback​

Rollback the entire migration rewrite.

rollback ;

All updates made within the transaction are discarded.

Commit migration rewrite​

Commit a migration rewrite.

commit migration rewrite ;