Skip to content

Commit 3b35bf1

Browse files
authored
docs: fix batch of upstream doc-change requests (#2688)
1 parent eb4075c commit 3b35bf1

28 files changed

Lines changed: 841 additions & 34 deletions

File tree

docs/reference/http-endpoints.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -291,6 +291,13 @@ Refer to the original Prometheus documentation for more information on the [Prom
291291
- **Description**: Compatible with Loki's API for log ingestion.
292292
- **Usage**: Send log data in Loki's format to this endpoint.
293293

294+
### Splunk HEC Compatibility
295+
296+
- **Path**: `/v1/splunk/services/collector/event`, `/v1/splunk/services/collector/raw`, `/v1/splunk/services/collector/health`
297+
- **Methods**: `POST` for the ingestion endpoints, `GET` for the health endpoint
298+
- **Description**: Compatible with the Splunk HTTP Event Collector (HEC) protocol for log ingestion.
299+
- **Usage**: Send JSON events to `/event` or plain text to `/raw`. See [Ingest Data with Splunk](/user-guide/ingest-data/for-observability/splunk.md).
300+
294301
### OpenTSDB Protocol
295302

296303
- **Path**: `/v1/opentsdb/api/put`

docs/reference/sql/alter.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,7 @@ Currently following options are supported:
194194
- `compaction.twcs.trigger_file_num`: the number of files in a specific time window to trigger a compaction.
195195
- `sst_format`: the SST format of the table. The value can be `flat` or `primary_key`. A table supports changing the format in both directions: `primary_key` to `flat` and `flat` to `primary_key`.
196196
- `write_buffer_size`: the per-region write buffer stall threshold of the table. For a positive value such as `512MB`, GreptimeDB schedules a flush when mutable memtable usage reaches half the value, stalls writes at the value, and rejects writes at twice the value. The table option overrides `region_engine.mito.default_region_write_buffer_size`. Setting it to `0` explicitly disables the per-region limit even when the engine default is nonzero. Unsetting it removes the table override and falls back to the engine default.
197+
- `auto_flush_interval`: how long a region of this table may go without a flush before one is triggered. The value is a [time duration string](/reference/time-durations.md) and must be greater than zero. The table option overrides the engine-wide `region_engine.mito.auto_flush_interval`.
197198

198199
```sql
199200
ALTER TABLE monitor SET 'ttl'='1d';
@@ -211,8 +212,22 @@ ALTER TABLE monitor SET 'sst_format'='flat';
211212
ALTER TABLE monitor SET 'sst_format'='primary_key';
212213

213214
ALTER TABLE monitor SET 'write_buffer_size'='512MB';
215+
216+
ALTER TABLE monitor SET 'auto_flush_interval'='5m';
214217
```
215218

219+
To drop the `auto_flush_interval` override and fall back to the engine-wide setting, set it
220+
to `NULL`:
221+
222+
```sql
223+
ALTER TABLE monitor SET 'auto_flush_interval' = NULL;
224+
```
225+
226+
:::warning
227+
`auto_flush_interval` does not support `UNSET`. `ALTER TABLE monitor UNSET 'auto_flush_interval'`
228+
returns an error; use `SET ... = NULL` as shown above.
229+
:::
230+
216231
### Unset table options
217232

218233
```sql

docs/reference/sql/copy.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,13 +138,40 @@ COPY tbl FROM '/path/to/file.csv' WITH (
138138
| `COMPRESSION_TYPE` | Compression algorithm for the imported file. Supported values: `gzip`, `zstd`, `bzip2`, `xz`. Only supported for CSV and JSON formats. | Optional |
139139
| `HEADERS` | Whether the CSV file has a header row. Supported values: `true`, `false`. Default is `true`. | Optional |
140140
| `SKIP_BAD_RECORDS` | Whether to skip rows with parsing or casting errors during CSV import. Supported values: `true`, `false`. Default is `false`. Only supported for CSV format. | Optional |
141+
| `STRICT_HEADERS` | Whether to validate CSV headers against the table schema before importing. Supported values: `true`, `false`. Default is `false`. Only supported for CSV format, and requires `HEADERS = 'true'`. | Optional |
141142

142143
:::tip NOTE
143144
By default, CSV files are parsed with headers and columns are matched by name.
144145

145146
To import headerless CSV files, set `HEADERS = 'false'`. In this mode, columns are mapped by position following the target table schema order. You can use `SHOW CREATE TABLE <table_name>` to inspect the table column order.
146147
:::
147148

149+
#### STRICT_HEADERS Option
150+
151+
By default, header matching is lenient: CSV columns the table does not have are ignored, and table columns the CSV does not have fall back to the normal insert and default-value handling.
152+
153+
Set `STRICT_HEADERS = 'true'` to validate the header row before any data is read. The import fails if the CSV has:
154+
155+
- a column the table does not have,
156+
- no column for a column the table does have,
157+
- the same header name more than once.
158+
159+
```sql
160+
COPY monitor FROM 'monitor.csv' WITH (FORMAT = 'CSV', STRICT_HEADERS = 'true');
161+
```
162+
163+
A mismatch reports all three categories at once:
164+
165+
```
166+
ERROR: CSV header mismatch in path: monitor.csv, unknown columns: ["extra"], missing columns: [], duplicate columns: []
167+
```
168+
169+
:::warning
170+
The missing-column check covers **every** column in the table, including nullable columns and columns with a default value. A CSV that omits any of them is rejected.
171+
172+
`STRICT_HEADERS = 'true'` cannot be combined with `HEADERS = 'false'` — a headerless CSV has no header names to validate. The combination returns `strict_headers=true requires headers=true`.
173+
:::
174+
148175
#### `CONNECTION` Option
149176

150177
`COPY FROM` also supports importing data from cloud storage services. See [connect-to-s3](#connect-to-s3), [connect-to-gcs](#connect-to-gcs), or [connect-to-azure-blob-storage](#connect-to-azure-blob-storage) for more details.

docs/reference/sql/create.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ Users can add table options by using `WITH`. The valid options contain the follo
162162
| `comment` | Table level comment | String value. |
163163
| `skip_wal` | Whether to disable Write-Ahead-Log for this table | String type. When set to `'true'`, the data written to the table will not be persisted to the write-ahead log, which can avoid storage wear and improve write throughput. However, when the process restarts, any unflushed data will be lost. Please use this feature only when the data source itself can ensure reliability. |
164164
| `write_buffer_size` | Per-region write buffer stall threshold for this table | String type, such as `'512MB'` or `'1GB'`. For a positive value, GreptimeDB schedules a flush when mutable memtable usage reaches half the value, stalls writes at the value, and rejects writes at twice the value. The table option overrides `region_engine.mito.default_region_write_buffer_size`. An explicit `'0'` disables the per-region limit even when the engine default is nonzero. Unset the option to remove the table override and fall back to the engine default. |
165+
| `auto_flush_interval` | How long a region of this table may go without a flush before one is triggered | String type, a time duration such as `'5m'` or `'1h'`. Must be greater than zero. The table option overrides the engine-wide `region_engine.mito.auto_flush_interval`. Set it to `NULL` with `ALTER TABLE` to drop the override and fall back to the engine setting. |
165166
| `index.type` | Index type | **Only for metric engine** String value, supports `none`, `skipping`. |
166167

167168
#### Create a table with TTL

docs/user-guide/deployments-administration/authentication/static.md

Lines changed: 37 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -60,13 +60,15 @@ Since v1.1, a password can use an explicit verifier format, so that plaintext pa
6060
- `plain:<password>` — plaintext. This is the default when no prefix is given.
6161
- `pbkdf2_sha256:<iterations>:<hex_salt>:<hex_hash>` — a PBKDF2-SHA256 hash stored at rest.
6262
- `mysql_native_password:<hex_sha1_sha1_password>` — a hashed verifier that still serves the MySQL `mysql_native_password` handshake.
63+
- `pg_scram_sha256:<iterations>:<hex_salt>:<hex_stored_key>:<hex_server_key>` — a SCRAM-SHA-256 verifier that serves the PostgreSQL SASL handshake. Available since v1.2.
6364

6465
Example:
6566

6667
```
6768
admin=plain:admin_pwd
6869
alice=pbkdf2_sha256:4096:73616c74:c5e478d59288c841aa530db6845c4c8d962893a001ce4e11a4963873aa98134a
6970
bob=mysql_native_password:6bb4837eb74329105ee4568dda7dc67ed2ca2ad9
71+
carol=pg_scram_sha256:4096:73616c74:53706a13f10b3c031b4c355d75ebd6500d3478062ce7d262710c3e60de02b93f:a19ce79824bd7ad68d96b8c00b0f1cc776bd0feca54d663301bb9866a860545b
7072
```
7173

7274
Permission modes combine with verifier formats. The verifier goes after the `=`:
@@ -79,18 +81,39 @@ alice:readonly=pbkdf2_sha256:4096:73616c74:c5e478d59288c841aa530db6845c4c8d96289
7981

8082
A single verifier format does not serve every protocol. Choose the format based on how clients connect:
8183

82-
| Verifier | HTTP/gRPC Basic | PostgreSQL cleartext | MySQL clear password | MySQL `mysql_native_password` |
83-
| --- | --- | --- | --- | --- |
84-
| `plain:<password>` (or legacy `user=password`) | yes | yes | yes | yes |
85-
| `pbkdf2_sha256:...` | yes | yes | yes | no |
86-
| `mysql_native_password:...` | no | no | no | yes |
84+
| Verifier | HTTP/gRPC Basic | PostgreSQL SCRAM-SHA-256 | PostgreSQL cleartext | MySQL clear password | MySQL `mysql_native_password` |
85+
| --- | --- | --- | --- | --- | --- |
86+
| `plain:<password>` (or legacy `user=password`) | yes | yes | yes | yes | yes |
87+
| `pbkdf2_sha256:...` | yes | no | yes | yes | no |
88+
| `mysql_native_password:...` | no | no | no | no | yes |
89+
| `pg_scram_sha256:...` | yes | yes | yes | yes | no |
8790

8891
`pbkdf2_sha256` protects passwords at rest; it does not change wire security. Cleartext-capable protocols still need TLS in production.
8992

9093
:::warning Breaking change
91-
Passwords are prefix-parsed. A legacy plaintext password that literally starts with `plain:`, `pbkdf2_sha256:`, or `mysql_native_password:` changes meaning. Use the `plain:` prefix to keep the literal value. For example, to keep the literal password `plain:secret`, configure it as `user=plain:plain:secret`.
94+
Passwords are prefix-parsed. A legacy plaintext password that literally starts with `plain:`, `pbkdf2_sha256:`, `mysql_native_password:`, or `pg_scram_sha256:` changes meaning. Use the `plain:` prefix to keep the literal value. For example, to keep the literal password `plain:secret`, configure it as `user=plain:plain:secret`.
9295
:::
9396

97+
#### PostgreSQL SCRAM-SHA-256
98+
99+
SCRAM-SHA-256 lets PostgreSQL clients authenticate without sending the password in cleartext.
100+
101+
PostgreSQL negotiates a single authentication method when a connection starts. The server does receive the username at that point, but picking the method per user would reveal whether that user exists and what verifier format it uses. GreptimeDB therefore decides globally: it offers SCRAM only when **every** user in the credential file can do SCRAM — that is, every verifier is `plain:` or `pg_scram_sha256:`. An unknown username is answered with a throwaway verifier and still runs the full handshake, so a failed login looks the same as a wrong password.
102+
103+
:::warning
104+
A single `pbkdf2_sha256:` or `mysql_native_password:` user makes the whole instance fall back to cleartext for PostgreSQL, including users whose own verifier supports SCRAM. If you want SCRAM, do not mix verifier formats.
105+
:::
106+
107+
Channel binding (`SCRAM-SHA-256-PLUS`) is not supported.
108+
109+
You can check which method the server offers with libpq's `require_auth` parameter, which needs libpq or `psql` 16 or newer:
110+
111+
```shell
112+
psql "host=127.0.0.1 port=4003 user=carol dbname=public require_auth=scram-sha-256"
113+
```
114+
115+
When the instance has fallen back to cleartext, that command fails with `server requested a cleartext password`.
116+
94117
### Generating Password Verifiers
95118

96119
Since v1.1, you can use the `greptime user hash-password` command to generate a verifier string. It runs standalone without starting any server component:
@@ -113,10 +136,10 @@ admin=pbkdf2_sha256:4096:<random_hex_salt>:<hex_hash>
113136

114137
Options:
115138

116-
- `--format <FORMAT>` — verifier format, `pbkdf2_sha256` (default) or `mysql_native_password`.
139+
- `--format <FORMAT>` — verifier format, `pbkdf2_sha256` (default), `mysql_native_password`, or `pg_scram_sha256`.
117140
- `--password <PASSWORD>` — plaintext password. Mutually exclusive with `--password-stdin`; exactly one is required. Prefer `--password-stdin` in scripts, since `--password` can leak through shell history or process listings.
118141
- `--password-stdin` — read the plaintext password from stdin.
119-
- `--iterations <N>` — PBKDF2-SHA256 iteration count (default `4096`, range `1..=1000000`).
142+
- `--iterations <N>` — PBKDF2-SHA256 / SCRAM-SHA-256 iteration count (default `4096`, range `1..=1000000`).
120143
- `--salt-len <N>` — random salt length in bytes (default `16`, range `1..=1024`).
121144
- `--salt-hex <HEX>` — fixed salt as hex instead of a random one, for deterministic automation.
122145

@@ -126,6 +149,12 @@ To generate a `mysql_native_password` verifier instead:
126149
./greptime user hash-password --password-stdin --format mysql_native_password
127150
```
128151

152+
To generate a PostgreSQL SCRAM-SHA-256 verifier:
153+
154+
```shell
155+
./greptime user hash-password --password-stdin --format pg_scram_sha256
156+
```
157+
129158
### Starting the Server
130159

131160
Start the server with the `--user-provider` parameter and set it to `static_user_provider:file:<path_to_file>` (replace `<path_to_file>` with the path to your user configuration file):

docs/user-guide/deployments-administration/configuration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -617,7 +617,7 @@ Available options:
617617
| `max_background_flushes` | Integer | `Auto` | Max number of running background flush jobs (default: 1/2 of cpu cores). |
618618
| `max_background_compactions` | Integer | `Auto` | Max number of running background compaction jobs (default: 1/4 of cpu cores). |
619619
| `max_background_purges` | Integer | `Auto` | Max number of running background purge jobs (default: cpu cores). |
620-
| `auto_flush_interval` | String | `1h` | Interval to auto flush a region if it has not flushed yet. |
620+
| `auto_flush_interval` | String | `1h` | Interval to auto flush a region if it has not flushed yet. Can be overridden per table with the [`auto_flush_interval` table option](/reference/sql/create.md#table-options). |
621621
| `global_write_buffer_size` | String | `1GB` | Global write buffer size for all regions. If not set, it's default to 1/8 of OS memory with a max limitation of 1GB. |
622622
| `global_write_buffer_reject_size` | String | `2GB` | Global write buffer size threshold to reject write requests. If not set, it's default to 2 times of `global_write_buffer_size` |
623623
| `default_region_write_buffer_size` | String | `0` | Default per-region write buffer stall threshold. For a positive value, GreptimeDB schedules a flush when mutable memtable usage reaches half the value, stalls writes at the value, and rejects writes at twice the value. Setting it to `0` disables the default per-region limit. A table's `write_buffer_size` overrides this value, including an explicit `0` that disables the limit for that table. |

docs/user-guide/deployments-administration/deploy-on-kubernetes/common-helm-chart-configurations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -621,7 +621,7 @@ datanode:
621621
[region_engine.mito.gc]
622622
enable = true
623623
lingering_time = "10m"
624-
unknown_file_lingering_time = "1h"
624+
unknown_file_lingering_time = "1d"
625625
```
626626

627627
Make sure the datanode `lingering_time` is longer than the metasrv `gc_cooldown_period` to avoid deleting files that may still be in use.

0 commit comments

Comments
 (0)