Upgrade from v.3.7.8 to Latest Version Failed

[0xChri2]

Deployment Type

Self-hosted

NetBox Version

v.3.7.8

Python Version

3.10

Steps to Reproduce

Upgradet from v.3.7.8 to latest

cd /opt/netbox
sudo git checkout master
sudo git pull origin master

then
sudo ./upgrade.sh

In that upgrade.sh this error accrued

Operations to perform:
Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
Applying users.0005_alter_user_table...Traceback (most recent call last):
File "/opt/netbox/netbox/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/init.py", line 442, in execute_from_command_line
utility.execute()
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/init.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 356, in handle
post_migrate_state = executor.migrate(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
self.code(from_state.apps, schema_editor)
File "/opt/netbox/netbox/users/migrations/0005_alter_user_table.py", line 17, in update_content_types
CustomField.objects.filter(related_object_type_id=netboxuser_ct.id).update(related_object_type_id=user_ct.id)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1476, in filter
return self._filter_or_exclude(False, args, kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1494, in _filter_or_exclude
clone._filter_or_exclude_inplace(negate, args, kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1501, in _filter_or_exclude_inplace
self._query.add_q(Q(*args, **kwargs))
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1613, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1645, in _add_q
child_clause, needed_inner = self.build_filter(
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1495, in build_filter
lookups, parts, reffed_expression = self.solve_lookup_type(arg, summarize)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1307, in solve_lookup_type
_, field, , lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1772, in names_to_path
raise FieldError(
django.core.exceptions.FieldError: Cannot resolve keyword 'related_object_type_id' into field. Choices are: choice_set, choice_set_id, created, default, description, filter_logic, group_name, id, is_cloneable, label, last_updated, name, object_type, object_type_id, object

i have looked into the database and there maybe some problem with older versions. Because i have updated first from v3.1.5 to v3.7.8 that worked perfekt but from there the update was not succesful.

Expected Behavior

The Migration failed of users.0005_alter_user_table

Observed Behavior

Operations to perform:
Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
Applying users.0005_alter_user_table...Traceback (most recent call last):
File "/opt/netbox/netbox/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/init.py", line 442, in execute_from_command_line
utility.execute()
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/init.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 356, in handle
post_migrate_state = executor.migrate(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
File "/home/service/.local/lib/python3.10/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
self.code(from_state.apps, schema_editor)
File "/opt/netbox/netbox/users/migrations/0005_alter_user_table.py", line 17, in update_content_types
CustomField.objects.filter(related_object_type_id=netboxuser_ct.id).update(related_object_type_id=user_ct.id)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1476, in filter
return self._filter_or_exclude(False, args, kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1494, in _filter_or_exclude
clone._filter_or_exclude_inplace(negate, args, kwargs)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/query.py", line 1501, in _filter_or_exclude_inplace
self._query.add_q(Q(*args, **kwargs))
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1613, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1645, in _add_q
child_clause, needed_inner = self.build_filter(
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1495, in build_filter
lookups, parts, reffed_expression = self.solve_lookup_type(arg, summarize)
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1307, in solve_lookup_type
_, field, , lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
File "/home/service/.local/lib/python3.10/site-packages/django/db/models/sql/query.py", line 1772, in names_to_path
raise FieldError(
django.core.exceptions.FieldError: Cannot resolve keyword 'related_object_type_id' into field. Choices are: choice_set, choice_set_id, created, default, description, filter_logic, group_name, id, is_cloneable, label, last_updated, name, object_type, object_type_id, object

[VictorPetrescu]

I'm in the same situation....upgrading from v3.7.8 to v4.1.0 and it throws the same error.

I only managed to make it work by upgrading to v4.0.8

[0xChri2]

Ok, but there is no tag for v4.0.8. Everything above 4 is only master

[VictorPetrescu]

git tag v4.0.8
git checkout v4.0.8
git pull origin v4.0.8

This worked for me.

[candlerb]

but there is no tag for v4.0.8

Use "git fetch" to update your local copy of the remote repo. Then "git checkout v4.0.8"

git tag v4.0.8

Don't do that - the chances are you've created your own tag v4.0.8 which is not the same as the upstream tag!! Best delete the tag, then "git fetch" again, and hopefully that may recreate the tag but I'm not sure

[PeterKnotek]

Try disabling plugins in configuration.py ... I had that problem and this helped

[0xChri2]

I got no Plugins installed

[Matze1224]

Had the same problem, too. Looks like an migration dependency issue, but let me explain:

Migration fails on 0113_customfield_rename_object_type. At digging in this file with blame, I've found 78dd652 which renames object_type to related_object_type. Reminds me of the error message of an unknown keyword. Inside the commit, it adds netbox/extras/migrations/0113_customfield_rename_object_type.py, unsure in which order it executes.

Tried and found out that it executes after our failing migration script. Adding it in as dependency solved the tragedy for me:

--- netbox/users/migrations/0005_alter_user_table.py.bak	2024-09-07 11:22:22.826711850 +0200
+++ netbox/users/migrations/0005_alter_user_table.py	2024-09-07 11:22:57.876836102 +0200
@@ -22,6 +22,7 @@
 
     dependencies = [
         ('users', '0002_squashed_0004'),
+        ('extras', '0113_customfield_rename_object_type')
     ]
 
     operations = [

(after patch or simply adding this line, rerun your migration and stuff so it works, finally - oh, another webui update :P)

P.S.: Dont know if I did it in an correct django manner that it can be mainlined.

[candlerb]

Nice catch. Clearly the dependencies must be done first, and if this isn't specified, they could be done in an arbitrary order.

[KodySalak]

You're the best, thanks for digging into this and finding that.

(glad that blame adds the file name because there is no way in hell i was going to be able to decipher that haha)