Skip to content

Commit c4feeb8

Browse files
filippolmtfilippolmt
committed
feat: enhance MySQL support and improve script logging
1 parent e6504b3 commit c4feeb8

File tree

10 files changed

+166
-40
lines changed

10 files changed

+166
-40
lines changed

‎README.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ This module creates database and users on an existing CloudSQL instance. The str
44

55
To enforce permissions, the module executes SQL commands with the mysql cli, which is therefore a prerequisite (it must be present in the filesystem where terraform apply is executed).
66

7+
For MySQL 8.x instances, the module automatically removes the default `cloudsqlsuperuser` role, clears any global privileges and assigns the target database as the only default role so that new users are scoped exclusively to their database.
8+
9+
If you ever need to rerun all local scripts (start proxy → grant privileges → stop proxy) without recreating the module-managed users, set a different value for the `permissions_refresh_id` variable (use the `YYYYMMDD` format, e.g. `20251110`) and run `terraform apply`; changing the value forces Terraform to recreate the null resources that execute those scripts while keeping the `google_sql_user` resources in place (see `examples/main.tf` for a ready-to-use snippet).
10+
711
In addition, the script must be able to connect to the CloudSQL instance. In case this is not easily accessible from the terraform cli, the module is able to:
812

913
1. Start an instance of [CloudSQL Auth Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy), for this purpose two null resources will be created for each user added to the database, enabling this option requires the [presence of the proxy executable](https://cloud.google.com/sql/docs/mysql/sql-proxy) in the filesystem where `terraform apply` is executed.
@@ -33,12 +37,13 @@ CloudSQL Auth Proxy needs the CloudSQL instance to expose a public IP address in
3337

3438
| Name | Description | Type | Default | Required |
3539
|------|-------------|------|---------|:--------:|
36-
| <a name="input_cloudsql_instance_name"></a> [cloudsql\_instance\_name](#input\_cloudsql\_instance\_name) | The name of the existing Google CloudSQL Instance name. Actually only a MySQL 5.7 or 8 instance is supported. | `string` | n/a | yes |
40+
| <a name="input_cloudsql_instance_name"></a> [cloudsql\_instance\_name](#input\_cloudsql\_instance\_name) | The name of the existing Google CloudSQL Instance name. MySQL 5.7, 8.0 and 8.4 are supported. | `string` | n/a | yes |
3741
| <a name="input_cloudsql_privileged_user_name"></a> [cloudsql\_privileged\_user\_name](#input\_cloudsql\_privileged\_user\_name) | The name of the privileged user of the Cloud SQL instance | `string` | n/a | yes |
3842
| <a name="input_cloudsql_privileged_user_password"></a> [cloudsql\_privileged\_user\_password](#input\_cloudsql\_privileged\_user\_password) | The password of the privileged user of the Cloud SQL instance | `string` | n/a | yes |
3943
| <a name="input_cloudsql_proxy_host"></a> [cloudsql\_proxy\_host](#input\_cloudsql\_proxy\_host) | The host of the Cloud SQL Auth Proxy; if a value other than localhost or 127.0.0.1 (default) is entered, it is assumed that there is a CloudSQL Auth Proxy instance defined and already configured outside this module, and therefore the proxy will not be launched. | `string` | `"127.0.0.1"` | no |
4044
| <a name="input_cloudsql_proxy_port"></a> [cloudsql\_proxy\_port](#input\_cloudsql\_proxy\_port) | Port of the Cloud SQL Auth Proxy | `string` | `"1234"` | no |
4145
| <a name="input_database_and_user_list"></a> [database\_and\_user\_list](#input\_database\_and\_user\_list) | The list with all the databases and the relative user. Please not that you can assign only a database to a single user, the same user cannot be assigned to multiple databases. `user_host` is optional, has a default value of '%' to allow the user to connect from any host, or you can specify it for the given user for a more restrictive access. | <pre>list(object({<br/> user = string<br/> user_host = optional(string, "%")<br/> database = string<br/> }))</pre> | n/a | yes |
46+
| <a name="input_permissions_refresh_id"></a> [permissions\_refresh\_id](#input\_permissions\_refresh\_id) | Optional identifier (use format YYYYMMDD, e.g. 20251110) used only to force Terraform to rerun the proxy/grant scripts without recreating users. Change the value whenever you need to reapply permissions. | `string` | `""` | no |
4247
| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The ID of the project in which the resource belongs. | `string` | n/a | yes |
4348
| <a name="input_region"></a> [region](#input\_region) | The region in which the resource belongs. | `string` | n/a | yes |
4449
| <a name="input_terraform_start_cloud_sql_proxy"></a> [terraform\_start\_cloud\_sql\_proxy](#input\_terraform\_start\_cloud\_sql\_proxy) | If `true` terraform will automatically start the Cloud SQL Proxy instance present in the filesystem at the condition that cloudsql\_proxy\_host is set to a supported value. If `false` you have to start the Cloud SQL Proxy manually. This variable is used to prevent the creation of a Cloud SQL Proxy instance even if cloudsql\_proxy\_host has a supported value. | `bool` | `true` | no |
@@ -54,6 +59,8 @@ CloudSQL Auth Proxy needs the CloudSQL instance to expose a public IP address in
5459
| [google_sql_database.sql_database](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database) | resource |
5560
| [google_sql_user.sql_user](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user) | resource |
5661
| [null_resource.execute_cloud_sql_proxy](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
62+
| [null_resource.force_permissions_refresh](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
63+
| [null_resource.grant_permissions](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
5764
| [null_resource.kill_cloud_sql_proxy](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
5865
| [random_password.sql_user_password](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) | resource |
5966
| [google_sql_database_instance.cloudsql_instance](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/sql_database_instance) | data source |

‎examples/main.tf‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,11 +70,13 @@ resource "google_sql_user" "admin_user_mysql" {
7070

7171
# Add additional user and database using this this module.
7272
module "mysql_additional_users_and_databases" {
73-
source = "sparkfabrik/gcp-mysql-db-and-user-creation-helper/sparkfabrik"
74-
version = "~> 0.1"
75-
project_id = var.project_id
76-
region = var.region
77-
database_and_user_list = var.database_and_user_list
73+
source = "sparkfabrik/gcp-mysql-db-and-user-creation-helper/sparkfabrik"
74+
version = "~> 0.1"
75+
project_id = var.project_id
76+
region = var.region
77+
database_and_user_list = var.database_and_user_list
78+
# Change this value (use YYYYMMDD, e.g. 20251110) whenever you need to rerun the proxy/grant scripts without recreating users.
79+
permissions_refresh_id = var.permissions_refresh_id
7880
cloudsql_instance_name = google_sql_database_instance.instance.name
7981
cloudsql_privileged_user_name = google_sql_user.admin_user_mysql.name
8082
cloudsql_privileged_user_password = google_sql_user.admin_user_mysql.password

‎examples/test.tfvars‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,6 @@ database_and_user_list = [
1717
user = "user4"
1818
}
1919
]
20+
21+
# Bump this value (YYYYMMDD, e.g. 20251110) whenever you need to rerun the proxy/grant scripts without recreating users.
22+
permissions_refresh_id = "20251110"

‎examples/variables.tf‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,9 @@ variable "database_and_user_list" {
1919
database = string
2020
}))
2121
}
22+
23+
variable "permissions_refresh_id" {
24+
type = string
25+
default = ""
26+
description = "Change this date (YYYYMMDD, e.g. 20251110) to force rerunning the proxy/grant scripts."
27+
}

‎main.tf‎

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@ resource "null_resource" "execute_cloud_sql_proxy" {
22
for_each = (((var.cloudsql_proxy_host == "localhost" || var.cloudsql_proxy_host == "127.0.0.1") && var.terraform_start_cloud_sql_proxy) ? {
33
for u in var.database_and_user_list : u.user => u
44
} : {})
5+
lifecycle {
6+
replace_triggered_by = [
7+
null_resource.force_permissions_refresh.id
8+
]
9+
}
510
provisioner "local-exec" {
611
command = "${path.module}/scripts/execute_cloud_sql_proxy.sh"
712
environment = {
@@ -55,6 +60,31 @@ resource "google_sql_user" "sql_user" {
5560
name = each.value.user
5661
password = random_password.sql_user_password[each.value.user].result
5762
host = each.value.user_host
63+
depends_on = [
64+
google_sql_database.sql_database
65+
]
66+
}
67+
68+
resource "null_resource" "force_permissions_refresh" {
69+
triggers = {
70+
refresh_id = var.permissions_refresh_id
71+
}
72+
}
73+
74+
resource "null_resource" "grant_permissions" {
75+
for_each = { for u in var.database_and_user_list : u.user => u }
76+
77+
triggers = {
78+
user = each.key
79+
user_host = each.value.user_host
80+
database = each.value.database
81+
}
82+
83+
lifecycle {
84+
replace_triggered_by = [
85+
null_resource.force_permissions_refresh.id
86+
]
87+
}
5888

5989
provisioner "local-exec" {
6090
command = "${path.module}/scripts/execute_sql.sh"
@@ -74,17 +104,24 @@ resource "google_sql_user" "sql_user" {
74104
interpreter = [
75105
"/bin/sh", "-c"
76106
]
77-
when = create
78107
}
108+
79109
depends_on = [
80-
google_sql_database.sql_database
110+
google_sql_database.sql_database,
111+
google_sql_user.sql_user,
112+
null_resource.execute_cloud_sql_proxy
81113
]
82114
}
83115

84116
resource "null_resource" "kill_cloud_sql_proxy" {
85117
for_each = (((var.cloudsql_proxy_host == "localhost" || var.cloudsql_proxy_host == "127.0.0.1") && var.terraform_start_cloud_sql_proxy) ? {
86118
for u in var.database_and_user_list : u.user => u
87119
} : {})
120+
lifecycle {
121+
replace_triggered_by = [
122+
null_resource.force_permissions_refresh.id
123+
]
124+
}
88125
provisioner "local-exec" {
89126
command = "${path.module}/scripts/kill_cloud_sql_proxy.sh"
90127
interpreter = [
@@ -94,6 +131,7 @@ resource "null_resource" "kill_cloud_sql_proxy" {
94131
}
95132
depends_on = [
96133
google_sql_database.sql_database,
97-
google_sql_user.sql_user
134+
google_sql_user.sql_user,
135+
null_resource.grant_permissions
98136
]
99137
}

‎scripts/execute_cloud_sql_proxy.sh‎

Lines changed: 31 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,50 @@
11
#!/usr/bin/env sh
22

3-
if ! [ -x "$(command -v cloud_sql_proxy)" ]; then
4-
echo "Error: cannot find the cloud_sql_proxy executable, please install it or add to your path." >&2
3+
set -eu
4+
5+
# shellcheck disable=SC3040
6+
if (set -o pipefail 2>/dev/null); then
7+
set -o pipefail
8+
fi
9+
10+
log() {
11+
printf '[sql-proxy] %s\n' "${1}"
12+
}
13+
14+
PROXY_BIN=""
15+
if command -v cloud_sql_proxy >/dev/null 2>&1; then
16+
PROXY_BIN="cloud_sql_proxy"
17+
else
18+
log "Error: cannot find the Cloud SQL Auth Proxy executable cloud_sql_proxy. Please install it or add it to your PATH." >&2
519
exit 1
6-
elif ! [ -x "$(command -v nc)" ]; then
7-
echo "Error: Netcat is not installed." >&2
20+
fi
21+
22+
if ! command -v nc >/dev/null 2>&1; then
23+
log "Error: Netcat is not installed." >&2
824
exit 1
925
fi
1026

11-
SERVICE="cloud_sql_proxy"
27+
CONNECTION_NAME="${CLOUDSDK_CORE_PROJECT}:${GCLOUD_PROJECT_REGION}:${CLOUDSQL_INSTANCE_NAME}"
1228

13-
if ! pgrep -x "$SERVICE" >/dev/null
14-
then
15-
exec cloud_sql_proxy -instances="${CLOUDSDK_CORE_PROJECT}:${GCLOUD_PROJECT_REGION}:${CLOUDSQL_INSTANCE_NAME}"="tcp:0.0.0.0:${CLOUDSQL_PROXY_PORT}" /dev/null 2>&1 &
29+
if ! pgrep -x "$PROXY_BIN" >/dev/null; then
30+
log "Starting Cloud SQL Auth Proxy (${PROXY_BIN}) for ${CONNECTION_NAME} on localhost:${CLOUDSQL_PROXY_PORT}."
31+
"${PROXY_BIN}" "${CONNECTION_NAME}" --port "${CLOUDSQL_PROXY_PORT}" >/dev/null 2>&1 &
32+
sleep 1s
33+
else
34+
log "Cloud SQL Auth Proxy already running; skipping start."
1635
fi
1736

1837
for j in $(seq 1 10); do
1938
READY=$(sh -c 'nc -v ${CLOUDSQL_PROXY_HOST} ${CLOUDSQL_PROXY_PORT} </dev/null; echo $?;' 2>/dev/null)
2039
if [ "$READY" -eq 0 ]; then
21-
echo "Connection with with CloudSQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}."
40+
log "Connection with Cloud SQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}."
2241
break
2342
fi
24-
echo "Waiting for Cloud SQL Proxy to start... $j"
43+
log "Waiting for Cloud SQL Proxy to start (attempt ${j}/10)..."
2544
sleep 1s
2645
done
2746

28-
if [ "$READY" -eq 1 ]; then
29-
echo "ERROR: cannot connect to the CloudSQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}, please check your settings."
47+
if [ "$READY" -ne 0 ]; then
48+
log "ERROR: cannot connect to the Cloud SQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}, please check your settings." >&2
3049
exit 1
3150
fi

‎scripts/execute_sql.sh‎

Lines changed: 40 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,65 @@
11
#!/usr/bin/env sh
22

3+
set -eu
4+
5+
# shellcheck disable=SC3040
6+
if (set -o pipefail 2>/dev/null); then
7+
set -o pipefail
8+
fi
9+
10+
log() {
11+
printf '[sql-grant] %s\n' "${1}"
12+
}
13+
314
if ! [ -x "$(command -v mysql)" ]; then
4-
echo "Error: the mysql client is not installed or is not in your path. Please add the mysql client executable." >&2
15+
log "Error: the mysql client is not installed or is not in your path. Please add the mysql client executable." >&2
516
exit 1
617
elif ! [ -x "$(command -v nc)" ]; then
7-
echo "Error: Netcat is not installed." >&2
18+
log "Error: Netcat is not installed." >&2
819
exit 1
920
fi
1021

1122
for j in $(seq 1 10); do
1223
READY=$(sh -c 'nc -v ${CLOUDSQL_PROXY_HOST} ${CLOUDSQL_PROXY_PORT} </dev/null; echo $?;' 2>/dev/null)
1324

1425
if [ "$READY" -eq 0 ]; then
15-
echo "Connection with with CloudSQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}."
26+
log "Connection with CloudSQL Auth Proxy established at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}."
1627
break
1728
fi
18-
echo "Waiting for Cloud SQL Proxy to start... $j"
29+
log "Waiting for Cloud SQL Proxy to start (attempt ${j}/10)..."
1930
sleep 1s
2031
done
2132

2233
if [ "$READY" -eq 0 ]; then
23-
if [ "${MYSQL_VERSION:0:9}" = "MYSQL_5_7" ]; then
24-
mysql --host=${CLOUDSQL_PROXY_HOST} --port=${CLOUDSQL_PROXY_PORT} --user=${CLOUDSQL_PRIVILEGED_USER_NAME} --password=${CLOUDSQL_PRIVILEGED_USER_PASSWORD} --execute="REVOKE ALL PRIVILEGES, GRANT OPTION FROM '${USER}'@'${USER_HOST}'; GRANT ALL ON ${DATABASE}.* TO ${USER}@'${USER_HOST}';"
25-
fi
34+
USER_IDENTIFIER="'${USER}'@'${USER_HOST}'"
35+
DATABASE_IDENTIFIER="\`${DATABASE}\`.*"
2636

27-
if [ "${MYSQL_VERSION:0:9}" = "MYSQL_8_0" ]; then
28-
mysql --host=${CLOUDSQL_PROXY_HOST} --port=${CLOUDSQL_PROXY_PORT} --user=${CLOUDSQL_PRIVILEGED_USER_NAME} --password=${CLOUDSQL_PRIVILEGED_USER_PASSWORD} --execute="REVOKE cloudsqlsuperuser FROM '${USER}'@'${USER_HOST}'; GRANT ALL ON ${DATABASE}.* TO ${USER}@'${USER_HOST}';"
37+
log "Preparing privilege statements for ${USER_IDENTIFIER} on database \`${DATABASE}\` (MySQL ${MYSQL_VERSION})."
38+
39+
case "${MYSQL_VERSION}" in
40+
MYSQL_5_7*)
41+
SQL_COMMANDS="REVOKE ALL PRIVILEGES, GRANT OPTION FROM ${USER_IDENTIFIER}; GRANT ALL PRIVILEGES ON ${DATABASE_IDENTIFIER} TO ${USER_IDENTIFIER};"
42+
;;
43+
MYSQL_8_0*|MYSQL_8_4*)
44+
SQL_COMMANDS="REVOKE cloudsqlsuperuser FROM ${USER_IDENTIFIER}; SET DEFAULT ROLE NONE TO ${USER_IDENTIFIER}; GRANT ALL PRIVILEGES ON ${DATABASE_IDENTIFIER} TO ${USER_IDENTIFIER};"
45+
;;
46+
*)
47+
log "ERROR: Unsupported MySQL version ${MYSQL_VERSION}." >&2
48+
exit 1
49+
;;
50+
esac
51+
52+
printf '[sql-grant] Executing SQL statements:\n%s\n' "${SQL_COMMANDS}"
53+
54+
if ! MYSQL_PWD="${CLOUDSQL_PRIVILEGED_USER_PASSWORD}" mysql --host="${CLOUDSQL_PROXY_HOST}" --port="${CLOUDSQL_PROXY_PORT}" --user="${CLOUDSQL_PRIVILEGED_USER_NAME}" --execute="${SQL_COMMANDS}"; then
55+
log "ERROR: Failed to apply privileges for ${USER_IDENTIFIER} on ${DATABASE}." >&2
56+
exit 1
2957
fi
3058

59+
log "Successfully applied privileges for ${USER_IDENTIFIER}."
60+
3161
exit 0
3262
else
33-
echo "ERROR: cannot connect to the CloudSQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}, please check your settings."
63+
log "ERROR: cannot connect to the CloudSQL Auth Proxy at ${CLOUDSQL_PROXY_HOST}:${CLOUDSQL_PROXY_PORT}, please check your settings." >&2
3464
exit 1
3565
fi

‎scripts/kill_cloud_sql_proxy.sh‎

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,23 @@
11
#!/usr/bin/env sh
22

3-
SERVICE="cloud_sql_proxy"
3+
set -eu
44

5-
if pgrep -x "$SERVICE" >/dev/null; then
6-
# It's better to take some time and to wait for the other tasks to finish
7-
# before killing the proxy; do not entering a sleep time, can lead to a
8-
# race condition error when simultaneously creating and destroying resources.
5+
# shellcheck disable=SC3040
6+
if (set -o pipefail 2>/dev/null); then
7+
set -o pipefail
8+
fi
9+
10+
log() {
11+
printf '[sql-proxy] %s\n' "${1}"
12+
}
13+
14+
PROXY_BIN="cloud_sql_proxy"
15+
if pgrep -x "$PROXY_BIN" >/dev/null; then
16+
log "Detected running ${PROXY_BIN}; waiting 5 seconds before shutdown to avoid race conditions."
917
sleep 5s
10-
PID_CLOUD_SQL_PROXY=$(pgrep -x ${SERVICE})
11-
kill "$PID_CLOUD_SQL_PROXY" || true
18+
# Obtain the PID of the running Cloud SQL Auth Proxy and terminate gently.
19+
PID="$(pgrep -x "$PROXY_BIN")"
20+
log "Stopping ${PROXY_BIN} (PID(s): ${PID})."
21+
22+
kill "${PID}" || true
1223
fi

‎variables.tf‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ variable "region" {
1010

1111
variable "cloudsql_instance_name" {
1212
type = string
13-
description = "The name of the existing Google CloudSQL Instance name. Actually only a MySQL 5.7 or 8 instance is supported."
13+
description = "The name of the existing Google CloudSQL Instance name. MySQL 5.7, 8.0 and 8.4 are supported."
1414
}
1515

1616
variable "terraform_start_cloud_sql_proxy" {
@@ -50,3 +50,14 @@ variable "database_and_user_list" {
5050
}))
5151
description = "The list with all the databases and the relative user. Please not that you can assign only a database to a single user, the same user cannot be assigned to multiple databases. `user_host` is optional, has a default value of '%' to allow the user to connect from any host, or you can specify it for the given user for a more restrictive access."
5252
}
53+
54+
variable "permissions_refresh_id" {
55+
type = string
56+
default = ""
57+
description = "Optional identifier (use format YYYYMMDD, e.g. 20251110) used only to force Terraform to rerun the proxy/grant scripts without recreating users. Change the value whenever you need to reapply permissions."
58+
59+
validation {
60+
condition = var.permissions_refresh_id == "" || can(regex("^\\d{8}$", var.permissions_refresh_id))
61+
error_message = "Set permissions_refresh_id to an 8-digit date in the form YYYYMMDD (e.g. 20251110) or leave it empty."
62+
}
63+
}

‎versions.tf‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,3 @@ terraform {
1616
}
1717
}
1818
}
19-

0 commit comments

Comments
 (0)