0

My system is running Ubuntu 18.04.2 LTS and Docker version 18.09.5, build e8ff056, Gitlab version gitlab-ee=12.0.3-ee.0.

I am trying to spin up a docker container with the following run command:

docker run -i \ --hostname gitlab2.michael-thompson.net \ --name gitlab \ --restart unless-stopped \ --volume /docker/gitlab/config:/etc/gitlab:Z \ --volume /docker/gitlab/log:/var/log/gitlab:Z \ --volume /docker/gitlab/data:/var/opt/gitlab:Z \ --net=bridge \ --env TZ=Australia/Perth \ gitlab/gitlab-ee:latest

The container boots for 20 seconds and then restarts (and continues in a never ending loop). Everything seems to run well until Gitlab attempts to "migrate" the database, and spits out the following error:

Recipe: gitlab::database_migrations * bash[migrate gitlab-rails database] action run [execute] rake aborted! StandardError: An error has occurred, this and all later migrations canceled: PG::DuplicateTable: ERROR: relation "audit_events" already exists : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp) /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Caused by: ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR: relation "audit_events" already exists : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp) /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Caused by: PG::DuplicateTable: ERROR: relation "audit_events" already exists /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Tasks: TOP => db:migrate (See full trace by running task with --trace) == 20141118150935 AddAuditEvent: migrating ==================================== -- adapter_name() -> 0.0000s -- adapter_name() -> 0.0000s -- create_table(:audit_events, {:id=>:integer}) ================================================================================ Error executing action `run` on resource 'bash[migrate gitlab-rails database]' ================================================================================ Mixlib::ShellOut::ShellCommandFailed ------------------------------------ Expected process to exit with [0], but received '1' ---- Begin output of "bash" "/tmp/chef-script20190704-26-1gi3jcd" ---- STDOUT: rake aborted! StandardError: An error has occurred, this and all later migrations canceled: PG::DuplicateTable: ERROR: relation "audit_events" already exists : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp) /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Caused by: ActiveRecord::StatementInvalid: PG::DuplicateTable: ERROR: relation "audit_events" already exists : CREATE TABLE "audit_events" ("id" serial NOT NULL PRIMARY KEY, "author_id" integer NOT NULL, "type" character varying NOT NULL, "entity_id" integer NOT NULL, "entity_type" character varying NOT NULL, "details" text, "created_at" timestamp, "updated_at" timestamp) /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Caused by: PG::DuplicateTable: ERROR: relation "audit_events" already exists /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20141118150935_add_audit_event.rb:6:in `change' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:56:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Tasks: TOP => db:migrate (See full trace by running task with --trace) == 20141118150935 AddAuditEvent: migrating ==================================== -- adapter_name() -> 0.0000s -- adapter_name() -> 0.0000s -- create_table(:audit_events, {:id=>:integer}) STDERR: ---- End output of "bash" "/tmp/chef-script20190704-26-1gi3jcd" ---- Ran "bash" "/tmp/chef-script20190704-26-1gi3jcd" returned 1 Resource Declaration: --------------------- # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb 53: bash "migrate gitlab-rails database" do 54: code <<-EOH 55: set -e 56: log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log" 57: umask 077 58: /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file} 59: STATUS=${PIPESTATUS[0]} 60: echo $STATUS > #{db_migrate_status_file} 61: exit $STATUS 62: EOH 63: environment env_variables unless env_variables.empty? 64: notifies :run, "execute[clear the gitlab-rails cache]", :immediately 65: dependent_services.each do |svc| 66: notifies :restart, svc, :immediately 67: end 68: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)" 69: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] } 70: end Compiled Resource: ------------------ # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:53:in `from_file' bash("migrate gitlab-rails database") do action [:run] default_guard_interpreter :default command nil backup 5 returns 0 user nil interpreter "bash" declared_type :bash cookbook_name "gitlab" recipe_name "database_migrations" code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93\n exit $STATUS\n" domain nil not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-1b1872f9d93 | grep -Fx 0)" only_if { #code block } end System Info: ------------ chef_version=13.6.4 platform=ubuntu platform_version=16.04 ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux] program_name=/opt/gitlab/embedded/bin/chef-client executable=/opt/gitlab/embedded/bin/chef-client Recipe: gitlab::gitlab-rails * execute[clear the gitlab-rails cache] action run

I've spent the best part of a day on this and can't figure out why it is crashing. I'm not using an external database engine / container, so I expect that that should be brand new with my container, and don't understand how it could be that the table already exists.

I have run the suggested docker exec -it gitlab update-permissions however given my suspicions it is database related, I had low hopes to which I seemed to be correct when trying to start the container again.

I have also tried updating my system via apt (with no luck, to no surprise), stopping and starting my container, to find the same error. I am already running 2 docker containers at this very moment on the same box, without a problem. I am also running another instance of Gitlab elsewhere (not inside a container) with the same config (although tweaked for the new system), also without a problem. It was my intention to migrate into the docker Gitlab if I could get it working.

Thanks

Improve this question
6
  • gitlab-ee being a commercial edition, you should contact gitlab support for help. Commented Jul 4, 2019 at 9:39
  • Interestingly enough, when I first installed Gitlab (previously, not as a docker container), it was ce. I did not knowingly upgrade to ee and am not paying for enterprise support, and according to the license model I read up on yesterday, it is OK to run ee without paying, and it will continue to operate in the same manner as ce. The reason I am installing ee is so that I can import my backup from my current production gitlab. In short, I do not have enterprise support for this as I am not a paying customer. Commented Jul 5, 2019 at 1:04
  • 1
    According to the error message, the DB content (within /var/opt/gitlab) is proabbly corrupted, the state of the DB doesn't match the recorded state of migration tasks, you should restore a backup before retrying Commented Jul 5, 2019 at 7:32
  • This is on the initial boot of the container. It is automatically trying to run the initial database migration. I do have a backup which I wish to restore into, but I need gitlab to be initialised before I can do that. Commented Jul 8, 2019 at 2:42
  • 1
    You are mounting /var/opt/gitlab as a volume from the host, so there's obviously something broken in those datas, just empty it to restart from scratch Commented Jul 8, 2019 at 6:11

1 Answer 1

Reset to default
1

According to the error message

PG::DuplicateTable: ERROR: relation "audit_events" already exists

The DB content (within /var/opt/gitlab) is probably corrupted, the state of the DB doesn't match the recorded state of migration tasks.

As you're mounting /var/opt/gitlab as a volume from the host with --volume /docker/gitlab/data:/var/opt/gitlab:Z, there's obviously something broken in those datas. you should just empty it to restart from scratch and restore a backup once bootstrapped properly.

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.