[Upgrade 4.21 to 4.22] java.sql.SQLSyntaxErrorException: PROCEDURE cloud.INSERT_EXTENSION_IF_NOT_EXISTS does not exist

[zidz]

problem

I was following the upgrade walk through here: https://docs.cloudstack.apache.org/en/4.22.0.0/upgrading/upgrade/upgrade-4.21.html
When there was time to start the management server again I get this error:

2026-01-19 09:53:45,366 DEBUG [c.c.u.d.ScriptRunner] (main:[]) (logid:) CALL `cloud`.`INSERT_EXTENSION_IF_NOT_EXISTS`('MaaS', 'Baremetal Extension for Canonical MaaS written in Python', 'MaaS/maas.py') 
2026-01-19 09:53:45,366 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: CALL `cloud`.`INSERT_EXTENSION_IF_NOT_EXISTS`('MaaS', 'Baremetal Extension for Canonical MaaS written in Python', 'MaaS/maas.py') 
2026-01-19 09:53:45,366 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLSyntaxErrorException: PROCEDURE cloud.INSERT_EXTENSION_IF_NOT_EXISTS does not exist
2026-01-19 09:53:45,367 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLSyntaxErrorException: PROCEDURE cloud.INSERT_EXTENSION_IF_NOT_EXISTS does not exist

Resulting in fatal error and the management server won't start.

versions

ACS Upgrade 4.21 -> 4.22

kvm --version
QEMU emulator version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18)

mariadb --version
mariadb Ver 15.1 Distrib 10.11.14-MariaDB, for debian-linux-gnu (x86_64) using EditLine wrapper

The steps to reproduce the bug

Alter /etc/apt/sources.list.d/cloudstack.list to "deb [signed-by=/usr/share/keyrings/cloudstack.gpg] https://download.cloudstack.org/ubuntu/ noble 4.22"
sudo service cloudstack-management stop
sudo apt-get update; sudo apt-get upgrade cloudstack-management
sudo service cloudstack-management start

What to do about it?

I'm trying to find the sql source of this missing procedure so I can add it. Can not find it.

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[zidz]

I now found the missing procedure so I can add it. I also saw that there are more procedures missing.
The first one missing I found in engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql

DROP PROCEDURE IF EXISTS `cloud`.`INSERT_EXTENSION_IF_NOT_EXISTS`;
CREATE PROCEDURE `cloud`.`INSERT_EXTENSION_IF_NOT_EXISTS`(
    IN ext_name VARCHAR(255),
    IN ext_desc VARCHAR(255),
    IN ext_path VARCHAR(255)
)
BEGIN
    IF NOT EXISTS (
        SELECT 1 FROM `cloud`.`extension` WHERE `name` = ext_name
    ) THEN
        INSERT INTO `cloud`.`extension` (
            `uuid`, `name`, `description`, `type`,
            `relative_path`, `path_ready`,
            `is_user_defined`, `state`, `created`, `removed`
        )
        VALUES (
            UUID(), ext_name, ext_desc, 'Orchestrator',
            ext_path, 1, 0, 'Enabled', NOW(), NULL
        )
;   END IF
;END;

This should be added by cloudstack at some point and should not need to be manually added.

I have the following procedures available now, before I'm trying to fix any issues:

MariaDB [(none)]> show procedure status where Db='cloud';
+-------+----------------------------------------------+-----------+---------+---------------------+---------------------+---------------+---------+----------------------+----------------------+--------------------+
| Db    | Name                                         | Type      | Definer | Modified            | Created             | Security_type | Comment | character_set_client | collation_connection | Database Collation |
+-------+----------------------------------------------+-----------+---------+---------------------+---------------------+---------------+---------+----------------------+----------------------+--------------------+
| cloud | ADD_GUEST_OS_AND_HYPERVISOR_MAPPING          | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_ADD_COLUMN                        | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_ADD_FOREIGN_KEY                   | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_ADD_KEY                           | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_ADD_UNIQUE_KEY                    | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_CHANGE_COLUMN                     | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_CREATE_UNIQUE_INDEX               | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_DROP_FOREIGN_KEY                  | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_DROP_INDEX                        | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_DROP_UNIQUE_KEY                   | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_INSERT_GUESTOS_HYPERVISOR_MAPPING | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
| cloud | IDEMPOTENT_UPDATE_API_PERMISSION             | PROCEDURE | cloud@% | 2026-01-19 10:28:49 | 2026-01-19 10:28:49 | DEFINER       |         | utf8mb4              | utf8mb4_general_ci   | utf8mb4_general_ci |
+-------+----------------------------------------------+-----------+---------+---------------------+---------------------+---------------+---------+----------------------+----------------------+--------------------+
12 rows in set (0.002 sec)

[zidz]

Fixing the next missing procedure:

2026-01-19 10:56:33,565 DEBUG [c.c.u.d.ScriptRunner] (main:[]) (logid:) CALL `cloud`.`INSERT_EXTENSION_IF_NOT_EXISTS`('MaaS', 'Baremetal Extension for Canonical MaaS written in Python', 'Ma
aS/maas.py') 
2026-01-19 10:56:33,567 DEBUG [c.c.u.d.ScriptRunner] (main:[]) (logid:) CALL `cloud`.`INSERT_EXTENSION_DETAIL_IF_NOT_EXISTS`('MaaS', 'orchestratorrequirespreparevm', 'true', 0) 
2026-01-19 10:56:33,570 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: CALL `cloud`.`INSERT_EXTENSION_DETAIL_IF_NOT_EXISTS`('MaaS', 'orchestratorrequirespreparevm', 'true'
, 0) 
2026-01-19 10:56:33,570 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLSyntaxErrorException: PROCEDURE cloud.INSERT_EXTENSION_DETAIL_IF_NOT_EXISTS does not exist
2026-01-19 10:56:33,571 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLSyntaxErrorException: PROCEDURE cloud.INSERT_EXTENSION_DETAIL_IF
_NOT_EXISTS does not exist

Procedure also found in: engine/schema/src/main/resources/META-INF/db/schema-42010to42100.sql

DROP PROCEDURE IF EXISTS `cloud`.`INSERT_EXTENSION_DETAIL_IF_NOT_EXISTS`;
CREATE PROCEDURE `cloud`.`INSERT_EXTENSION_DETAIL_IF_NOT_EXISTS`(
    IN ext_name VARCHAR(255),
    IN detail_key VARCHAR(255),
    IN detail_value TEXT,
    IN display TINYINT(1)
)
BEGIN
    DECLARE ext_id BIGINT
;   SELECT `id` INTO ext_id FROM `cloud`.`extension` WHERE `name` = ext_name LIMIT 1
;   IF NOT EXISTS (
        SELECT 1 FROM `cloud`.`extension_details`
        WHERE `extension_id` = ext_id AND `name` = detail_key
    ) THEN
        INSERT INTO `cloud`.`extension_details` (
            `extension_id`, `name`, `value`, `display`
        )
        VALUES (
            ext_id, detail_key, detail_value, display
        )
;   END IF
;END;

[zidz]

Adding the above procedures solved the issue and the management server is now running again.

[DaanHoogland]

@zidz thanks, I think procedures are in separate file though, in engine/schema/src/main/resources/META-INF/db/procedures.

[DaanHoogland]

@harikrishna-patnala @shwstppr , can you guys have a look?

[zidz]

A note, this cluster was installed as 4.21, so no previous upgrades have been done.

[DaanHoogland]

Doesn’t matter @zidz , any installation is done as 4.0 and then a series of upgrades. This is because there is a strong emphasis on backwards compatibility in the project. All procedures from the folder mentioned get installed in the DB. the fact that the procedure is in the upgrade file is a bug, but it would still have been run for a clean 4.21 installation. cc @shwstppr @harikrishna-patnala