feat: enhance MySQL support and improve script logging (#20)

* feat: enhance MySQL support and improve script logging

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

Author: filippolmt

CHANGELOG.md

@@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres
 to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+## [0.5.0] - 2025-11-11
+
+### Added
+
+- Introduced the `permissions_refresh_id` input plus the `null_resource.force_permissions_refresh`/`null_resource.grant_permissions` helpers so you can rerun the proxy/grant scripts without recreating users.
+
+### Changed
+
+- Reworked the resource lifecycles so proxy start/kill and permission scripts are re-executed whenever `permissions_refresh_id` changes.
+- Hardened the Cloud SQL proxy helper scripts: better logging, explicit dependency checks (mysql/nc/proxy), use of the v2 proxy CLI syntax, and safer shutdown handling.
+- Enhanced `execute_sql.sh` so MySQL 8.0/8.4 users have the `cloudsqlsuperuser` role removed, default roles cleared, and only database-scoped privileges granted.
+- Documentation updates: clarify Cloud SQL Auth Proxy v2 requirement, mention MySQL 8.4 support.
+
 ## [0.4.1] - 2025-02-24
 
 [Compare with previous version](https://github.com/sparkfabrik/terraform-google-gcp-mysql-db-and-user-creation-helper/compare/0.4.0...0.4.1)

README.md

@@ -1,13 +1,17 @@
 # Terraform module for creating database and associated user on an existing Google CloudSQL instance
 
-This module creates database and users on an existing CloudSQL instance. The structure of the input variable is designed so that the database/user ratio is 1:1, so the module not only takes care of creating the database and its user, but also sets permissions on the user so that it has access to only the database for which it is responsible.
+This module creates databases and users on an existing CloudSQL instance. The structure of the input variable enforces a 1:1 database/user ratio. The module both creates each pair and applies the required permissions so that the user can access only its database.
 
-To enforce permissions, the module executes SQL commands with the mysql cli, which is therefore a prerequisite (it must be present in the filesystem where terraform apply is executed).
+To enforce permissions, the module executes SQL commands through the MySQL CLI, which therefore must be installed on the machine running `terraform apply`. The bundled proxy helper scripts rely on the [Cloud SQL Auth Proxy **v2** command syntax](https://cloud.google.com/sql/docs/mysql/connect-auth-proxy), so make sure you install proxy version 2.x (the newer `${CLOUDSQL_PROXY_BIN} \"${CONNECTION_NAME}\" --port ...` invocation) rather than the legacy 1.x binary that used `-instances=` flags.
 
-In addition, the script must be able to connect to the CloudSQL instance. In case this is not easily accessible from the terraform cli, the module is able to:
+For MySQL 8.x instances, the module automatically removes the default `cloudsqlsuperuser` role, clears any global privileges and assigns the target database as the only default role so that new users are scoped exclusively to their database.
 
-1. Start an instance of [CloudSQL Auth Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy), for this purpose two null resources will be created for each user added to the database, enabling this option requires the [presence of the proxy executable](https://cloud.google.com/sql/docs/mysql/sql-proxy) in the filesystem where `terraform apply` is executed.
-2. Connect from a [CloudSQL Auth Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy) instance not present in the filesystem.
+If you ever need to rerun all local scripts (start proxy → grant privileges → stop proxy) without recreating the module-managed users, set a different value for the `permissions_refresh_id` variable (use `YYYYMMDD` or `YYYYMMDDHHMM`, e.g. `20251110` or `202511101030`) and run `terraform apply`; changing the value forces Terraform to recreate the null resources that execute those scripts while keeping the `google_sql_user` resources in place (see `examples/main.tf` for a ready-to-use snippet).
+
+In addition, the module must be able to connect to the CloudSQL instance. If the instance is not directly reachable from the machine running `terraform apply`, the module can:
+
+1. Start a local instance of [CloudSQL Auth Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy). This creates two null resources per user and requires the [proxy executable](https://cloud.google.com/sql/docs/mysql/sql-proxy) to be present on the machine running `terraform apply`.
+2. Connect through an existing [CloudSQL Auth Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy) instance that is already running elsewhere (outside this module).
 
 ### IMPORTANT
 
@@ -33,12 +37,13 @@ CloudSQL Auth Proxy needs the CloudSQL instance to expose a public IP address in
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_cloudsql_instance_name"></a> [cloudsql\_instance\_name](#input\_cloudsql\_instance\_name) | The name of the existing Google CloudSQL Instance name. Actually only a MySQL 5.7 or 8 instance is supported. | `string` | n/a | yes |
+| <a name="input_cloudsql_instance_name"></a> [cloudsql\_instance\_name](#input\_cloudsql\_instance\_name) | The name of the existing Google CloudSQL Instance name. MySQL 5.7, 8.0 and 8.4 are supported. | `string` | n/a | yes |
 | <a name="input_cloudsql_privileged_user_name"></a> [cloudsql\_privileged\_user\_name](#input\_cloudsql\_privileged\_user\_name) | The name of the privileged user of the Cloud SQL instance | `string` | n/a | yes |
 | <a name="input_cloudsql_privileged_user_password"></a> [cloudsql\_privileged\_user\_password](#input\_cloudsql\_privileged\_user\_password) | The password of the privileged user of the Cloud SQL instance | `string` | n/a | yes |
 | <a name="input_cloudsql_proxy_host"></a> [cloudsql\_proxy\_host](#input\_cloudsql\_proxy\_host) | The host of the Cloud SQL Auth Proxy; if a value other than localhost or 127.0.0.1 (default) is entered, it is assumed that there is a CloudSQL Auth Proxy instance defined and already configured outside this module, and therefore the proxy will not be launched. | `string` | `"127.0.0.1"` | no |
 | <a name="input_cloudsql_proxy_port"></a> [cloudsql\_proxy\_port](#input\_cloudsql\_proxy\_port) | Port of the Cloud SQL Auth Proxy | `string` | `"1234"` | no |
-| <a name="input_database_and_user_list"></a> [database\_and\_user\_list](#input\_database\_and\_user\_list) | The list with all the databases and the relative user. Please not that you can assign only a database to a single user, the same user cannot be assigned to multiple databases. `user_host` is optional, has a default value of '%' to allow the user to connect from any host, or you can specify it for the given user for a more restrictive access. | <pre>list(object({<br/>    user      = string<br/>    user_host = optional(string, "%")<br/>    database  = string<br/>  }))</pre> | n/a | yes |
+| <a name="input_database_and_user_list"></a> [database\_and\_user\_list](#input\_database\_and\_user\_list) | The list with all the databases and the relative user. Please note that you can assign only a database to a single user, the same user cannot be assigned to multiple databases. `user_host` is optional, has a default value of '%' to allow the user to connect from any host, or you can specify it for the given user for a more restrictive access. | <pre>list(object({<br/>    user      = string<br/>    user_host = optional(string, "%")<br/>    database  = string<br/>  }))</pre> | n/a | yes |
+| <a name="input_permissions_refresh_id"></a> [permissions\_refresh\_id](#input\_permissions\_refresh\_id) | Optional identifier (use format YYYYMMDD or YYYYMMDDHHMM, e.g. 20251110 or 202511101030) used only to force Terraform to rerun the proxy/grant scripts without recreating users. Change the value whenever you need to reapply permissions. | `string` | `""` | no |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The ID of the project in which the resource belongs. | `string` | n/a | yes |
 | <a name="input_region"></a> [region](#input\_region) | The region in which the resource belongs. | `string` | n/a | yes |
 | <a name="input_terraform_start_cloud_sql_proxy"></a> [terraform\_start\_cloud\_sql\_proxy](#input\_terraform\_start\_cloud\_sql\_proxy) | If `true` terraform will automatically start the Cloud SQL Proxy instance present in the filesystem at the condition that cloudsql\_proxy\_host is set to a supported value. If `false` you have to start the Cloud SQL Proxy manually. This variable is used to prevent the creation of a Cloud SQL Proxy instance even if cloudsql\_proxy\_host has a supported value. | `bool` | `true` | no |
@@ -54,6 +59,8 @@ CloudSQL Auth Proxy needs the CloudSQL instance to expose a public IP address in
 | [google_sql_database.sql_database](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database) | resource |
 | [google_sql_user.sql_user](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user) | resource |
 | [null_resource.execute_cloud_sql_proxy](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
+| [null_resource.force_permissions_refresh](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
+| [null_resource.grant_permissions](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
 | [null_resource.kill_cloud_sql_proxy](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
 | [random_password.sql_user_password](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource |
 | [google_sql_database_instance.cloudsql_instance](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/sql_database_instance) | data source |

examples/main.tf

@@ -70,11 +70,13 @@ resource "google_sql_user" "admin_user_mysql" {
 
 # Add additional user and database using this this module.
 module "mysql_additional_users_and_databases" {
-  source                            = "sparkfabrik/gcp-mysql-db-and-user-creation-helper/sparkfabrik"
-  version                           = "~> 0.1"
-  project_id                        = var.project_id
-  region                            = var.region
-  database_and_user_list            = var.database_and_user_list
+  source                 = "sparkfabrik/gcp-mysql-db-and-user-creation-helper/sparkfabrik"
+  version                = "~> 0.1"
+  project_id             = var.project_id
+  region                 = var.region
+  database_and_user_list = var.database_and_user_list
+  # Change this value (use YYYYMMDD or YYYYMMDDHHMM, e.g. 20251110 or 202511101030) whenever you need to rerun the proxy/grant scripts without recreating users.
+  permissions_refresh_id            = var.permissions_refresh_id
   cloudsql_instance_name            = google_sql_database_instance.instance.name
   cloudsql_privileged_user_name     = google_sql_user.admin_user_mysql.name
   cloudsql_privileged_user_password = google_sql_user.admin_user_mysql.password

examples/test.tfvars

@@ -17,3 +17,6 @@ database_and_user_list = [
     user      = "user4"
   }
 ]
+
+# Bump this value (YYYYMMDD or YYYYMMDDHHMM, e.g. 20251110 or 202511101030) whenever you need to rerun the proxy/grant scripts without recreating users.
+permissions_refresh_id = "202511101030"

examples/variables.tf

@@ -19,3 +19,9 @@ variable "database_and_user_list" {
     database = string
   }))
 }
+
+variable "permissions_refresh_id" {
+  type        = string
+  default     = ""
+  description = "Change this date or date-time (YYYYMMDD or YYYYMMDDHHMM, e.g. 20251110 or 202511101030) to force rerunning the proxy/grant scripts."
+}

main.tf

@@ -2,6 +2,11 @@ resource "null_resource" "execute_cloud_sql_proxy" {
   for_each = (((var.cloudsql_proxy_host == "localhost" || var.cloudsql_proxy_host == "127.0.0.1") && var.terraform_start_cloud_sql_proxy) ? {
     for u in var.database_and_user_list : u.user => u
   } : {})
+  lifecycle {
+    replace_triggered_by = [
+      null_resource.force_permissions_refresh.id
+    ]
+  }
   provisioner "local-exec" {
     command = "${path.module}/scripts/execute_cloud_sql_proxy.sh"
     environment = {
@@ -55,6 +60,31 @@ resource "google_sql_user" "sql_user" {
   name     = each.value.user
   password = random_password.sql_user_password[each.value.user].result
   host     = each.value.user_host
+  depends_on = [
+    google_sql_database.sql_database
+  ]
+}
+
+resource "null_resource" "force_permissions_refresh" {
+  triggers = {
+    refresh_id = var.permissions_refresh_id
+  }
+}
+
+resource "null_resource" "grant_permissions" {
+  for_each = { for u in var.database_and_user_list : u.user => u }
+
+  triggers = {
+    user      = each.key
+    user_host = each.value.user_host
+    database  = each.value.database
+  }
+
+  lifecycle {
+    replace_triggered_by = [
+      null_resource.force_permissions_refresh.id
+    ]
+  }
 
   provisioner "local-exec" {
     command = "${path.module}/scripts/execute_sql.sh"
@@ -74,17 +104,24 @@ resource "google_sql_user" "sql_user" {
     interpreter = [
       "/bin/sh", "-c"
     ]
-    when = create
   }
+
   depends_on = [
-    google_sql_database.sql_database
+    google_sql_database.sql_database,
+    google_sql_user.sql_user,
+    null_resource.execute_cloud_sql_proxy
   ]
 }
 
 resource "null_resource" "kill_cloud_sql_proxy" {
   for_each = (((var.cloudsql_proxy_host == "localhost" || var.cloudsql_proxy_host == "127.0.0.1") && var.terraform_start_cloud_sql_proxy) ? {
     for u in var.database_and_user_list : u.user => u
   } : {})
+  lifecycle {
+    replace_triggered_by = [
+      null_resource.force_permissions_refresh.id
+    ]
+  }
   provisioner "local-exec" {
     command = "${path.module}/scripts/kill_cloud_sql_proxy.sh"
     interpreter = [
@@ -94,6 +131,7 @@ resource "null_resource" "kill_cloud_sql_proxy" {
   }
   depends_on = [
     google_sql_database.sql_database,
-    google_sql_user.sql_user
+    google_sql_user.sql_user,
+    null_resource.grant_permissions
   ]
 }

scripts/execute_cloud_sql_proxy.sh

@@ -1,31 +1,50 @@
 #!/usr/bin/env sh
 
-if ! [ -x "$(command -v cloud_sql_proxy)" ]; then
-    echo "Error: cannot find the cloud_sql_proxy executable, please install it or add to your path." >&2
+set -eu
+
+# shellcheck disable=SC3040
+if (set -o pipefail 2>/dev/null); then
+    set -o pipefail
+fi
+
+log() {
+    printf '[sql-proxy] %s\n' "${1}"
+}
+
+CLOUDSQL_PROXY_BIN=""
+if command -v cloud_sql_proxy >/dev/null 2>&1; then
+    CLOUDSQL_PROXY_BIN="cloud_sql_proxy"
+else
+    log "Error: cannot find the Cloud SQL Auth Proxy executable cloud_sql_proxy. Please install it or add it to your PATH." >&2
     exit 1
-elif ! [ -x "$(command -v nc)" ]; then
-    echo "Error: Netcat is not installed." >&2
+fi
+
+if ! command -v nc >/dev/null 2>&1; then
+    log "Error: Netcat is not installed." >&2
     exit 1
 fi
 
-SERVICE="cloud_sql_proxy"
+CONNECTION_NAME="${CLOUDSDK_CORE_PROJECT}:${GCLOUD_PROJECT_REGION}:${CLOUDSQL_INSTANCE_NAME}"
 
-if ! pgrep -x "$SERVICE" >/dev/null
-then
-    exec cloud_sql_proxy -instances="${CLOUDSDK_CORE_PROJECT}:${GCLOUD_PROJECT_REGION}:${CLOUDSQL_INSTANCE_NAME}"="tcp:0.0.0.0:${CLOUDSQL_PROXY_PORT}" /dev/null 2>&1 &
+if ! pgrep -x "$CLOUDSQL_PROXY_BIN" >/dev/null; then
+    log "Starting Cloud SQL Auth Proxy (${CLOUDSQL_PROXY_BIN}) for ${CONNECTION_NAME} on localhost:${CLOUDSQL_PROXY_PORT}."
+    "${CLOUDSQL_PROXY_BIN}" "${CONNECTION_NAME}" --port "${CLOUDSQL_PROXY_PORT}" >/dev/null 2>&1 &
+    sleep 1s
+else
+    log "Cloud SQL Auth Proxy already running; skipping start."
 fi
 
 for j in $(seq 1 10); do
     READY=$(sh -c 'nc -v ${CLOUDSQL_PROXY_HOST} ${CLOUDSQL_PROXY_PORT} </dev/null; echo $?;' 2>/dev/null)
     if [ "$READY" -eq 0 ]; then
-        echo "Connection with with CloudSQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}."
+        log "Connection with Cloud SQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}."
         break
     fi
-    echo "Waiting for Cloud SQL Proxy to start... $j"
+    log "Waiting for Cloud SQL Proxy to start (attempt ${j}/10)..."
     sleep 1s
 done
 
-if [ "$READY" -eq 1 ]; then
-    echo "ERROR: cannot connect to the CloudSQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}, please check your settings."
+if [ "$READY" -ne 0 ]; then
+    log "ERROR: cannot connect to the Cloud SQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}, please check your settings." >&2
     exit 1
 fi